Backtesting, Simulation & Replay

From strategy idea to evidence

Now that we've covered Statistical Arbitrage & Latency Strategies, you can generate signals. Earlier, you also built pieces of the trading path: market data, order books, execution algorithms, market making, and OMS/FIX flow.

This lesson answers the next question:

Before risking capital, how do we test whether a strategy would have behaved well under historical market conditions?

In HFT-style systems, a good test is not just a spreadsheet P&L. It must respect:

• Event ordering from market data feeds
• Timestamps and latency from earlier lessons
• Order book state reconstructed from events
• Fill assumptions for orders that may or may not execute
• Costs, slippage, and inventory risk

Backtesting, simulation, and replay are the lab bench where your strategy becomes measurable.

Three related tools

People often use these words interchangeably, but in trading systems they have different jobs.

1. Backtesting

Run a strategy over historical data and compute results.

2. Simulation

Model things the historical feed does not directly tell you, such as whether your order filled, how much queue was ahead of you, or how latency changed your decision.

3. Replay

Feed recorded events back into your system in the same order as live trading, often using the same handlers you would use in production.

Historical data
     |
     v
Event replay  ---> Order book reconstruction
     |                    |
     v                    v
Strategy logic ----> Simulated OMS / fills
     |                    |
     v                    v
Metrics: P&L, drawdown, fills, latency, markouts

The key mindset: test the same decision path you expect to use live, but replace the exchange with a controlled simulator.

Same input, same output

A replay engine should be deterministic: if you run the same data twice, you should get the same decisions and metrics.

For beginner C++ learners, the first version can be simple:

• Store market events in memory
• Sort by exchange timestamp, then sequence number
• Feed each event to the strategy/book handler
• Log every important decision

Later systems may stream terabytes from disk, but the principle stays the same.

```cpp
#include <algorithm>
#include <iomanip>
#include <iostream>
#include <string>
#include <vector>

struct Event {
    long long ts_ns;
    int seq;
    std::string type;
    double price;
    int qty;
};

int main() {
    std::vector<Event> events = {
        {1000, 2, "TRADE", 100.02, 50},
        { 900, 1, "BID",   100.00, 200},
        {1000, 1, "ASK",   100.05, 180},
        {1100, 3, "TRADE", 100.04, 20}
    };

    std::sort(events.begin(), events.end(), [](const Event& a, const Event& b) {
        if (a.ts_ns != b.ts_ns) return a.ts_ns < b.ts_ns;
        return a.seq < b.seq;
    });

    double bestBid = 0.0;
    double bestAsk = 0.0;

    std::cout << std::fixed << std::setprecision(2);
    std::cout << "Deterministic replay:\n";

    for (const auto& e : events) {
        if (e.type == "BID") bestBid = e.price;
        if (e.type == "ASK") bestAsk = e.price;

        std::cout << "ts=" << e.ts_ns
                  << " seq=" << e.seq
                  << " type=" << e.type
                  << " price=" << e.price
                  << " qty=" << e.qty;

        if (bestBid > 0.0 && bestAsk > 0.0) {
            double mid = (bestBid + bestAsk) / 2.0;
            std::cout << " mid=" << mid;
        }
        std::cout << "\n";
    }

    return 0;
}
```

Two historical market data messages have the same exchange timestamp but different sequence numbers.

What should a deterministic replay engine usually do?

The hard part: did we actually trade?

Historical market data tells you what happened in the market. It usually does not prove exactly what would have happened to your order.

That is why backtests need a fill model.

Common fill model levels:

• Optimistic: if price touched your limit, assume full fill
• Pessimistic: require price to trade through your limit before filling
• Queue-aware: estimate quantity ahead of you at the price level
• Latency-aware: delay order arrival before deciding if it joined the queue

For HFT, optimistic fills can create fake profits. A market-making strategy from the previous lesson might look excellent if every quote magically fills only when favorable. Realistic simulation tries to remove that illusion.

Simulating queue position

Here is a small standalone C++ example of a passive buy limit order waiting at the bid.

Assumptions for this toy model:

• We place a buy limit at 100.00
• There are 15 shares ahead of us in the queue
• Incoming sell trades at or below our price consume queue
• Only leftover quantity after the queue ahead is gone fills us

This is simplified, but it captures the most important lesson: a touched price is not automatically your fill.

```cpp
#include <algorithm>
#include <iomanip>
#include <iostream>
#include <vector>

struct Trade {
    long long ts_ns;
    double price;
    int qty;
};

int main() {
    const double limitPrice = 100.00;
    const int orderSize = 10;
    int remaining = orderSize;
    int queueAhead = 15;

    std::vector<Trade> trades = {
        {1000, 100.01, 8},   // above our buy limit, no fill
        {1100, 100.00, 6},   // consumes queue ahead
        {1200,  99.99, 12},  // consumes rest of queue, then fills us
        {1300, 100.00, 20}   // may fill remaining
    };

    int filled = 0;
    double cash = 0.0;

    std::cout << std::fixed << std::setprecision(2);
    std::cout << "Buy limit order: price=" << limitPrice
              << " size=" << orderSize
              << " queueAhead=" << queueAhead << "\n\n";

    for (const auto& t : trades) {
        std::cout << "trade ts=" << t.ts_ns
                  << " price=" << t.price
                  << " qty=" << t.qty << " -> ";

        if (t.price > limitPrice || remaining == 0) {
            std::cout << "no interaction\n";
            continue;
        }

        int available = t.qty;

        int queueConsumed = std::min(queueAhead, available);
        queueAhead -= queueConsumed;
        available -= queueConsumed;

        int fillNow = std::min(remaining, available);
        remaining -= fillNow;
        filled += fillNow;
        cash -= fillNow * limitPrice;

        std::cout << "queueConsumed=" << queueConsumed
                  << " fillNow=" << fillNow
                  << " remaining=" << remaining << "\n";
    }

    double markPrice = 100.04;
    double positionValue = filled * markPrice;
    double pnl = cash + positionValue;

    std::cout << "\nFinal filled=" << filled
              << " cash=" << cash
              << " markPrice=" << markPrice
              << " toyPnL=" << pnl << "\n";

    return 0;
}
```

Using information from the future while making a historical trading decision is called ________.

Type the exact two-word phrase.

P&L is not enough

A backtest should produce a report that explains why the strategy made or lost money.

Useful metrics include:

• Gross P&L: before fees and costs
• Net P&L: after fees, rebates, and estimated slippage
• Fill ratio: submitted quantity vs executed quantity
• Average slippage: expected price vs simulated execution price
• Inventory path: especially for market making
• Drawdown: worst peak-to-trough loss
• Markout: price movement after your fill, such as 10 ms or 1 second later
• Latency sensitivity: what happens if your order arrives 50 µs, 500 µs, or 5 ms later?

For strategies from Market Making & Liquidity Provision and Statistical Arbitrage & Latency Strategies, markouts and latency sensitivity are often more informative than headline P&L.

True or false?

A historical replay can run faster than real time and still be valid, as long as it preserves event-time ordering and does not let the strategy observe future events early.

Common ways backtests lie

Before trusting a result, check for these failure modes:

1. Look-ahead bias

The strategy accidentally reads future bars, future volume, or final-day statistics.

2. Survivorship bias

Testing only symbols that still exist today ignores delisted or failed instruments.

3. Unrealistic fills

Assuming every touched limit order fills can overstate performance.

4. Missing costs

Fees, rebates, spread, slippage, and rejects all matter.

5. Bad timestamps

If exchange time, capture time, and local time are mixed incorrectly, latency analysis becomes misleading.

6. Non-determinism

If the same replay produces different results, debugging becomes painful.

The goal is not to make the backtest pessimistic. The goal is to make it honest enough to improve the strategy.

What you now have

You can now describe a practical research loop:

1. Load historical events
2. Replay them deterministically
3. Reconstruct book state using the logic from Market Data Handling & Order Book Reconstruction
4. Run strategy decisions from the execution, market-making, or stat-arb lessons
5. Simulate order arrival and fills
6. Compute net performance and risk metrics
7. Investigate bad trades, not just good averages

A strong backtesting setup becomes your safety net. It helps you reject fragile strategies before they reach production.

Next, we will move from the research lab toward live concerns: deployment, monitoring, and performance tuning.