Market Data Handling & Order Book Reconstruction

Why this lesson matters

In Market Microstructure & Data Feeds, you learned that exchanges publish quotes, trades, timestamps, and sequence numbers. In Order Management & FIX Integration, you learned how our system sends orders and receives execution reports.

Now we connect the two worlds: the trading engine needs a trustworthy view of the order book before it can make or manage orders.

A market data handler usually does three jobs:

• Decode feed messages quickly
• Maintain an in-memory order book
• Detect bad state: gaps, crossed books, stale data, and out-of-order updates

Today we focus on reconstructing a clean Level 2 price book from market data events. Later, execution algorithms and market making will consume this book.

From feed messages to book state

A market data feed is not usually sent as a full book every time. Instead, it sends events.

Common event types:

• Add / update price level: set quantity at a price
• Delete price level: remove a price from the book
• Trade print: report an execution that happened
• Snapshot: a full book image at a known sequence

For this lesson, we will model Level 2 updates like this:

seq=101 side=B price=100.10 qty=500
seq=102 side=A price=100.20 qty=300
seq=103 side=B price=100.10 qty=0      # delete level

The core rule is simple:

The book is the result of applying every valid update in exact sequence order.

If one update is missing, every later top-of-book value may be wrong.

A Level 2 order book reconstruction engine receives incremental updates from an exchange feed.

Which field is most critical for detecting that an update was missed?

A beginner-friendly C++ book

For production HFT, data structures are often custom, cache-aware, and preallocated, building on ideas from Low-Latency C++ for HFT.

But first, we need the reconstruction logic to be correct.

This example uses:

• std::map for asks, sorted low to high
• std::map with std::greater<> for bids, sorted high to low
• Quantity 0 as a delete

This is not the final low-latency design, but it is a clear reference implementation.

```cpp
#include <iostream>
#include <map>
#include <string>
#include <functional>

struct Update {
    long seq;
    char side;       // 'B' bid, 'A' ask
    double price;
    int qty;         // qty == 0 means delete level
};

class L2Book {
    std::map<double, int, std::greater<double>> bids; // best bid first
    std::map<double, int> asks;                       // best ask first

public:
    void apply(const Update& u) {
        auto& side = (u.side == 'B') ? bids : asks;
        if (u.qty == 0) {
            side.erase(u.price);
        } else {
            side[u.price] = u.qty;
        }
    }

    void printTop() const {
        std::cout << "Top of book: ";
        if (!bids.empty()) {
            std::cout << "bid " << bids.begin()->first << " x " << bids.begin()->second;
        } else {
            std::cout << "bid EMPTY";
        }

        std::cout << " | ";

        if (!asks.empty()) {
            std::cout << "ask " << asks.begin()->first << " x " << asks.begin()->second;
        } else {
            std::cout << "ask EMPTY";
        }
        std::cout << "\n";
    }
};

int main() {
    L2Book book;

    Update feed[] = {
        {101, 'B', 100.10, 500},
        {102, 'A', 100.20, 300},
        {103, 'B', 100.05, 900},
        {104, 'B', 100.10, 0},    // delete best bid
        {105, 'A', 100.18, 200}   // better ask
    };

    for (const auto& u : feed) {
        book.apply(u);
        std::cout << "After seq " << u.seq << ": ";
        book.printTop();
    }

    return 0;
}
```

Never blindly apply increments

A market data handler should not simply apply every update it sees. It should maintain an expectedSeq value.

Basic logic:

if update.seq == expectedSeq:
    apply update
    expectedSeq++
elif update.seq > expectedSeq:
    gap detected, book is unsafe
else:
    old or duplicate message, ignore or log

In a real feed handler, a gap might trigger:

• Stop publishing this book to strategies
• Request or wait for a fresh snapshot
• Replay buffered incrementals after the snapshot
• Alert monitoring

The important mindset: a fast wrong book is worse than a slightly slower correct book.

```cpp
#include <iostream>
#include <vector>

struct Msg {
    long seq;
    const char* text;
};

int main() {
    std::vector<Msg> feed = {
        {10, "bid update"},
        {11, "ask update"},
        {13, "trade print"},   // seq 12 is missing
        {14, "ask delete"}
    };

    long expectedSeq = 10;
    bool bookSafe = true;

    for (const auto& msg : feed) {
        if (msg.seq == expectedSeq) {
            std::cout << "Apply seq " << msg.seq << ": " << msg.text << "\n";
            expectedSeq++;
        } else if (msg.seq > expectedSeq) {
            std::cout << "GAP! expected " << expectedSeq
                      << " but received " << msg.seq << "\n";
            bookSafe = false;
            break;
        } else {
            std::cout << "Ignore old duplicate seq " << msg.seq << "\n";
        }
    }

    std::cout << "Book safe? " << (bookSafe ? "yes" : "no") << "\n";
    return 0;
}
```

True or false:

If an incremental market data update is missing, the handler can keep publishing the book as long as later sequence numbers arrive in order.

How handlers recover after gaps

A snapshot is a full book image at a known sequence number. Incremental updates are smaller, but they only make sense if none are missing.

A common recovery pattern:

1. Detect gap at expected sequence N
2. Mark symbol/book as not publishable
3. Obtain snapshot with sequence S
4. Reset the book to snapshot contents
5. Apply buffered incrementals with sequence > S, in order
6. Publish again only when sequence continuity is restored

This is why the feed handler often has a small buffer: messages may arrive while the snapshot is being fetched or decoded.

The exact protocol differs by venue, but the reconstruction principle stays the same.

Fill in the blank:

To apply incremental updates safely, the market data handler tracks the next expected ________.

Validate before publishing

Even with perfect sequencing, a production handler should run cheap sanity checks before publishing book state to strategy code.

Useful checks:

• Crossed book: best bid is greater than or equal to best ask
• Empty side: no bid or no ask, depending on instrument and venue
• Negative or impossible quantity
• Stale timestamp relative to local receive time
• Unexpected symbol or venue state

A crossed book may be valid for a tiny moment in some feeds because of message ordering across channels, but your strategy should know whether it is consuming a normal, delayed, or unsafe state.

```cpp
#include <iostream>
#include <map>
#include <functional>

class BookCheck {
    std::map<double, int, std::greater<double>> bids;
    std::map<double, int> asks;

public:
    void setBid(double price, int qty) { bids[price] = qty; }
    void setAsk(double price, int qty) { asks[price] = qty; }

    bool hasBothSides() const {
        return !bids.empty() && !asks.empty();
    }

    bool isCrossed() const {
        if (!hasBothSides()) return false;
        double bestBid = bids.begin()->first;
        double bestAsk = asks.begin()->first;
        return bestBid >= bestAsk;
    }

    void report() const {
        if (!hasBothSides()) {
            std::cout << "Book warning: missing one side\n";
            return;
        }

        std::cout << "Best bid=" << bids.begin()->first
                  << " best ask=" << asks.begin()->first << "\n";

        if (isCrossed()) {
            std::cout << "Book warning: crossed or locked market\n";
        } else {
            std::cout << "Book looks normal\n";
        }
    }
};

int main() {
    BookCheck normal;
    normal.setBid(100.10, 500);
    normal.setAsk(100.20, 300);
    normal.report();

    BookCheck crossed;
    crossed.setBid(100.25, 100);
    crossed.setAsk(100.20, 300);
    crossed.report();

    return 0;
}
```

Where this fits in the HFT system

You can now place order book reconstruction inside the architecture we have been building:

Network receive
  -> packet/feed decoder
  -> sequence gate
  -> snapshot/incremental merger
  -> in-memory book
  -> validated top-of-book/depth event
  -> strategy and risk consumers

This lesson gives us the market-state foundation. The next lesson, Core Execution Algorithms (TWAP, VWAP, POV), will use market data as an input while deciding how to schedule and place orders.

Before moving on, remember the operating rule:

Publish only book states that are sequenced, reconstructed, and validated.