Concurrency & Lock-Free Programming

From fast code to fast systems

In Low-Latency C++ for HFT, you learned to respect cache, allocation, and latency budgets. In Networking & Kernel Bypass, you saw why polling, batching, and packet flow matter.

Now we connect those ideas: a trading system is not one heroic function. It is a set of cooperating threads:

• one thread may read market data packets
• another may normalize events
• another may run strategy logic
• another may prepare orders

The danger: every handoff can add latency, jitter, or bugs.

This lesson focuses on concurrency and lock-free programming for HFT-style C++ systems: how threads communicate without accidentally turning microseconds into milliseconds.

Threads are lanes, not magic

Concurrency lets work happen in parallel, but it also creates coordination cost.

A useful HFT mental model:

Market data thread  --->  queue  --->  strategy thread
      owns packets                  owns decisions

The goal is usually single ownership:

• one thread writes a piece of data
• another thread reads it only after a clear handoff
• shared mutable state is minimized

Why not just use locks everywhere?

Locks are correct and useful, but hot-path locks can introduce:

• blocking when another thread owns the lock
• OS scheduling delays
• priority inversion
• unpredictable tail latency

In trading systems, the average latency matters, but the 99.9th percentile latency often matters more.

In an HFT hot path, what is the main reason a contended mutex can be dangerous?

The simplest useful lock-free queue

A common HFT building block is the SPSC ring buffer:

• Single Producer: exactly one writing thread
• Single Consumer: exactly one reading thread
• fixed capacity: no allocation in the hot path
• circular indexing: reuse memory

This is ideal for a clean handoff such as:

network/feed thread  --->  strategy thread

The key idea: producer owns tail, consumer owns head. They communicate with atomics.

```cpp
#include <atomic>
#include <array>
#include <iostream>
#include <thread>
#include <chrono>

struct Event {
    int sequence;
    double price;
};

template <size_t N>
class SpscRing {
public:
    bool push(const Event& e) {
        size_t tail = tail_.load(std::memory_order_relaxed);
        size_t next = (tail + 1) % N;

        // If advancing tail would catch head, the ring is full.
        if (next == head_.load(std::memory_order_acquire)) {
            return false;
        }

        buffer_[tail] = e;
        tail_.store(next, std::memory_order_release);
        return true;
    }

    bool pop(Event& out) {
        size_t head = head_.load(std::memory_order_relaxed);

        // If head equals tail, there is nothing to consume.
        if (head == tail_.load(std::memory_order_acquire)) {
            return false;
        }

        out = buffer_[head];
        head_.store((head + 1) % N, std::memory_order_release);
        return true;
    }

private:
    std::array<Event, N> buffer_{};
    std::atomic<size_t> head_{0};
    std::atomic<size_t> tail_{0};
};

int main() {
    SpscRing<8> queue;

    std::thread producer([&] {
        for (int i = 1; i <= 10; ++i) {
            Event e{i, 100.0 + i * 0.25};
            while (!queue.push(e)) {
                // Backpressure: consumer is behind.
                std::this_thread::yield();
            }
            std::cout << "produced seq=" << i << "\n";
        }
    });

    std::thread consumer([&] {
        int received = 0;
        while (received < 10) {
            Event e;
            if (queue.pop(e)) {
                std::cout << "  consumed seq=" << e.sequence
                          << " price=" << e.price << "\n";
                ++received;
            } else {
                std::this_thread::yield();
            }
        }
    });

    producer.join();
    consumer.join();
    std::cout << "done\n";
}
```

Fill in the blank:

A consumer using std::memory_order_acquire is commonly paired with a producer using std::memory_order_____ to publish data safely.

Atomics are about visibility and order

With multiple threads, the question is not only “did this variable change?” It is also:

When another thread sees the flag, can it safely see the data that came before it?

For beginners, start with three practical levels:

Ordering Useful intuition relaxed Atomic read/write, but no ordering promise around other data release “I am publishing everything I wrote before this point” acquire “After I see this, I can safely read what was published”

In the SPSC queue, the producer writes the event first, then stores the new tail with release. The consumer reads tail with acquire, then reads the event.

HFT rule of thumb

Use the weakest ordering you can prove correct — but correctness comes first. A faster wrong trading system is just an expensive bug.

Different variables can still fight

In Low-Latency C++ for HFT, you saw that cache layout matters. Concurrency adds a painful version of that lesson: false sharing.

False sharing happens when two threads update different variables that sit on the same cache line.

same cache line:
[counter A][counter B]
 thread 1    thread 2

Even though the variables are logically separate, the CPU cache system may bounce the whole cache line between cores.

The fix is often to align or pad hot per-thread data so independent writers do not share a cache line.

```cpp
#include <atomic>
#include <chrono>
#include <iostream>
#include <thread>

constexpr int64_t N = 20'000'000;

struct SharedLine {
    std::atomic<int64_t> a{0};
    std::atomic<int64_t> b{0};
};

struct alignas(64) PaddedCounter {
    std::atomic<int64_t> value{0};
};

struct SeparateLines {
    PaddedCounter a;
    PaddedCounter b;
};

template <typename T>
long long run_test(T& counters) {
    auto start = std::chrono::high_resolution_clock::now();

    std::thread t1([&] {
        for (int64_t i = 0; i < N; ++i) {
            counters.a.fetch_add(1, std::memory_order_relaxed);
        }
    });

    std::thread t2([&] {
        for (int64_t i = 0; i < N; ++i) {
            counters.b.fetch_add(1, std::memory_order_relaxed);
        }
    });

    t1.join();
    t2.join();

    auto end = std::chrono::high_resolution_clock::now();
    return std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count();
}

int main() {
    SharedLine shared;
    SeparateLines separated;

    auto t_shared = run_test(shared);
    auto t_separate = run_test(separated);

    std::cout << "same cache line-ish: " << t_shared << " ms\n";
    std::cout << "padded counters:      " << t_separate << " ms\n";
    std::cout << "final values: "
              << shared.a.load() << ", " << shared.b.load() << "\n";
}
```

True or false:

Lock-free code is always faster than lock-based code.

Waiting is a design choice

In latency-sensitive systems, a thread may spin instead of sleeping:

while queue is empty:
    keep checking

This can reduce wake-up latency, but it burns CPU. That tradeoff can be acceptable for dedicated HFT cores, but dangerous if applied everywhere.

Common wait strategies

• Busy spin: lowest wake-up delay, highest CPU use
• Spin then yield: good compromise for short waits
• Condition variable / blocking: efficient CPU use, higher wake-up jitter

Backpressure

When a producer is faster than a consumer, the queue fills. You must decide what happens:

• wait and slow the producer
• drop non-critical events
• coalesce updates
• alert and fail safely

For trading systems, never hide backpressure. Make it measurable.

You have exactly one market data thread producing normalized quote events and exactly one strategy thread consuming them. Which structure is usually the cleanest hot-path handoff?

What to take forward

You now have the concurrency foundation needed for low-latency trading components.

Before putting concurrent code on a hot path, ask:

• Who owns this data?
• How many producers and consumers exist?
• Is allocation avoided after startup?
• What happens when the queue is full?
• Which memory ordering proves the handoff correct?
• Could false sharing create cache-line ping-pong?
• Are latency percentiles measured, not guessed?

Practical HFT guidance

Prefer simple topologies first:

single writer -> fixed queue -> single reader

Only move to more complex lock-free structures when the design truly requires them.

Next, we will use these handoff ideas when discussing how trading systems manage outbound orders and exchange-facing protocols.