Networking & Kernel Bypass

Why this matters

In Low-Latency C++ for HFT, you learned that nanoseconds disappear into cache misses, allocation, and avoidable work. Now we zoom out one layer: the path between your process and the exchange/network feed.

For HFT systems, networking is not just “send bytes.” It is a latency pipeline:

market data packet -> NIC -> OS/kernel -> socket -> C++ parser -> strategy
order message      -> C++ OMS -> socket/kernel -> NIC -> venue

Today’s goal is to understand:

• where normal sockets spend time,
• what kernel bypass changes,
• when polling, batching, and NIC placement matter,
• and how to write a small socket baseline before reaching for advanced tools.

We will stay scoped to networking mechanics. The next lesson, Concurrency & Lock-Free Programming, will handle multi-threaded handoff patterns.

The normal Linux path

A basic UDP market-data receiver usually follows this journey:

wire
  ↓
NIC hardware queue
  ↓ interrupt or polling
kernel network stack
  ↓ copy/checks/routing/socket lookup
socket receive buffer
  ↓ recvfrom()/recvmmsg()
your C++ buffer

Each step can be useful, but each step can also cost latency.

What the kernel gives you

• device drivers and hardware support
• routing and protocol handling
• safety between processes
• familiar APIs like sendto() and recvfrom()
• tools such as tcpdump, firewalling, and OS counters

What HFT engineers worry about

• interrupt delays and scheduling jitter
• packet copies between kernel and user space
• cache pollution from generic networking paths
• queueing inside NIC, kernel, or application buffers
• unpredictable tail latency, not just average latency

The lesson is not “kernel bad.” The lesson is: measure the path, then remove layers only when they are proven to matter.

A market-data receiver has low average latency but occasional large spikes. Which networking factor is most directly associated with unpredictable tail latency?

Start with a boring baseline

Before kernel bypass, build a tiny socket test you can trust. This example sends one UDP packet over loopback and receives it with recvfrom().

In real HFT, loopback is not the same as a NIC, switch, multicast feed, or exchange gateway. But a baseline like this verifies:

• you understand the API boundary,
• your program handles byte buffers correctly,
• and you can instrument the receive path before optimizing it.

Compile on Linux/macOS-like systems:

g++ -std=c++17 udp_baseline.cpp -O2 -o udp_baseline
./udp_baseline

```cpp
#include <arpa/inet.h>
#include <netinet/in.h>
#include <sys/socket.h>
#include <unistd.h>

#include <cstring>
#include <iostream>
#include <string>

int main() {
    int rx = socket(AF_INET, SOCK_DGRAM, 0);
    int tx = socket(AF_INET, SOCK_DGRAM, 0);
    if (rx < 0 || tx < 0) {
        std::perror("socket");
        return 1;
    }

    sockaddr_in rx_addr{};
    rx_addr.sin_family = AF_INET;
    rx_addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
    rx_addr.sin_port = 0; // ask OS for an available port

    if (bind(rx, reinterpret_cast<sockaddr*>(&rx_addr), sizeof(rx_addr)) < 0) {
        std::perror("bind");
        return 1;
    }

    socklen_t len = sizeof(rx_addr);
    if (getsockname(rx, reinterpret_cast<sockaddr*>(&rx_addr), &len) < 0) {
        std::perror("getsockname");
        return 1;
    }

    std::string msg = "QUOTE AAPL bid=189.10 ask=189.11 seq=42";
    ssize_t sent = sendto(tx, msg.data(), msg.size(), 0,
                          reinterpret_cast<sockaddr*>(&rx_addr), sizeof(rx_addr));
    if (sent < 0) {
        std::perror("sendto");
        return 1;
    }

    char buffer[256]{};
    sockaddr_in from{};
    socklen_t from_len = sizeof(from);
    ssize_t n = recvfrom(rx, buffer, sizeof(buffer) - 1, 0,
                         reinterpret_cast<sockaddr*>(&from), &from_len);
    if (n < 0) {
        std::perror("recvfrom");
        return 1;
    }

    std::cout << "sent bytes: " << sent << "\n";
    std::cout << "received bytes: " << n << "\n";
    std::cout << "payload: " << buffer << "\n";

    close(tx);
    close(rx);
    return 0;
}
```

In POSIX UDP socket code, the common system call used to receive a datagram and also capture the sender address is ____.

Kernel bypass is not step one

Many production systems first tune the normal socket path. Common areas include:

• receive/send buffer sizing: reduce drops during bursts, but avoid hiding queueing too deeply;
• CPU affinity: keep the network-processing thread near the NIC interrupt/core path;
• interrupt moderation: lower latency may require fewer coalesced packets;
• RSS / hardware queues: steer flows to predictable CPU queues;
• jumbo frames: useful in some environments, not universal;
• timestamping: verify where latency is actually introduced.

This small program demonstrates inspecting and changing a socket receive buffer. Notice that the OS may round or cap the requested value.

```cpp
#include <sys/socket.h>
#include <unistd.h>

#include <cstring>
#include <iostream>

int main() {
    int fd = socket(AF_INET, SOCK_DGRAM, 0);
    if (fd < 0) {
        std::perror("socket");
        return 1;
    }

    int requested = 1 << 20; // 1 MiB
    if (setsockopt(fd, SOL_SOCKET, SO_RCVBUF, &requested, sizeof(requested)) < 0) {
        std::perror("setsockopt SO_RCVBUF");
        close(fd);
        return 1;
    }

    int actual = 0;
    socklen_t len = sizeof(actual);
    if (getsockopt(fd, SOL_SOCKET, SO_RCVBUF, &actual, &len) < 0) {
        std::perror("getsockopt SO_RCVBUF");
        close(fd);
        return 1;
    }

    std::cout << "requested receive buffer bytes: " << requested << "\n";
    std::cout << "actual receive buffer bytes:    " << actual << "\n";
    std::cout << "note: kernels may round, double, or cap this value\n";

    close(fd);
    return 0;
}
```

True or false: Kernel bypass always makes a trading system faster, safer, and easier to operate.

What kernel bypass changes

Kernel bypass moves packet I/O closer to user space. Instead of every packet traveling through the full kernel networking stack, a user-space library talks more directly to NIC queues.

Common approaches include:

• DPDK: user-space polling drivers and hugepage-backed packet buffers;
• Solarflare/Xilinx Onload-style acceleration: optimized user-space networking while preserving familiar APIs in some modes;
• AF_XDP: Linux path for high-performance packet I/O using XDP sockets;
• RDMA: direct memory access across machines, used in specialized environments.

The key idea is not magic. It is usually some combination of:

less kernel work + fewer copies + polling + pinned memory + direct NIC queues

The tradeoff

You gain control and predictability, but you may lose ordinary OS conveniences. Your application becomes responsible for more of the packet lifecycle.

Two recurring network ideas

Even without implementing DPDK today, two ideas appear everywhere in high-performance packet I/O.

Polling

Instead of waiting for the OS to wake you when packets arrive, a hot loop checks a queue repeatedly. This can reduce wakeup latency but burns CPU.

Batching

Instead of processing one packet per expensive boundary crossing, receive or handle several packets at once. This improves throughput and can reduce per-packet overhead, but too-large batches can add waiting time.

This toy program simulates batch processing. It does not use real NIC queues; it simply shows why “one packet at a time” and “all packets in huge chunks” are different design choices.

```cpp
#include <algorithm>
#include <chrono>
#include <iostream>
#include <numeric>
#include <vector>

struct Packet {
    int seq;
    int bytes;
};

long long process_batches(const std::vector<Packet>& packets, int batch_size) {
    long long checksum = 0;
    for (std::size_t i = 0; i < packets.size(); i += batch_size) {
        std::size_t end = std::min<std::size_t>(i + batch_size, packets.size());
        // One loop represents one batch pulled from a receive ring.
        for (std::size_t j = i; j < end; ++j) {
            checksum += packets[j].seq * 31 + packets[j].bytes;
        }
    }
    return checksum;
}

int main() {
    std::vector<Packet> packets;
    packets.reserve(100000);
    for (int i = 1; i <= 100000; ++i) {
        packets.push_back(Packet{i, 64 + (i % 64)});
    }

    for (int batch : {1, 8, 32, 128}) {
        auto start = std::chrono::steady_clock::now();
        long long checksum = process_batches(packets, batch);
        auto end = std::chrono::steady_clock::now();
        auto micros = std::chrono::duration_cast<std::chrono::microseconds>(end - start).count();

        std::cout << "batch=" << batch
                  << " checksum=" << checksum
                  << " time_us=" << micros << "\n";
    }

    std::cout << "In real networking, batch size affects both throughput and latency.\n";
    return 0;
}
```

A team receives 50k market-data messages per second, sees no drops, and its measured end-to-end latency budget is already met. What is the most reasonable next step?

What to remember

Networking in HFT is about controlling the path between market events and trading decisions.

Practical checklist

• Build a socket baseline first.
• Measure average, p99, and worst-case latency.
• Watch for packet drops, queue buildup, and burst behavior.
• Understand where copies occur.
• Tune buffers carefully; bigger is not always lower latency.
• Know when interrupts help and when polling is justified.
• Treat kernel bypass as an engineering tradeoff, not a badge of seriousness.

Narrative checkpoint

You now have the network mental model needed for real trading infrastructure. Next, Concurrency & Lock-Free Programming will focus on how packets move between threads without destroying the latency gains you worked to preserve here.