# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Build

Requires DPDK 24.11 built with meson at `~/share/dpdk-study/sources/dpdk-24.11/build`.

```bash
make
```

The binary is `./build/dpdk_ws_client`. The Makefile uses `pkg-config` (with `PKG_CONFIG_PATH` pointing to DPDK's uninstalled `.pc` files) to get compiler/linker flags. The binary includes an RPATH pointing to the DPDK lib directory, so no `LD_LIBRARY_PATH` is needed at runtime. Key flags: `-DALLOW_EXPERIMENTAL_API`, `-Wl,--allow-multiple-definition`, `-O2 -g`.

### Test/benchmark binaries (standalone, no DPDK needed)

```bash
gcc -O2 -o test/test_ws_protocol test/test_ws_protocol.c -lpthread
gcc -O2 -o test/bench_tcp test/bench_tcp.c -lpthread
```

## Architecture

This is a **DPDK user-space TCP/IP stack** with a **WebSocket client (RFC 6455)** layered on top. It runs entirely in userspace — no kernel TCP.

### Lcore split

- **lcore0 (main thread)**: DPDK RX/TX burst from NIC, WS receive polling, application logic (`main.c`)
- **lcore1 (pkt_process)**: Protocol processing loop — TCP state machine, ARP/ICMP, timer maintenance, TX output engine (`tcp_core.c`)

Data crosses lcores via two `rte_ring` SP/SC queues: `ring->in` (RX → pkt_process) and `ring->out` (pkt_process → TX).

### Layer stack (bottom-up)

```
DPDK rte_eth_rx_burst / rte_eth_tx_burst
  → pkt_process dispatch: ARP / ICMP / TCP (ng_tcp_process)
    → TCP 11-state FSM (tcp_input.c) + congestion control (tcp_output.c) + timers (tcp_timer.c)
      → stream->rcvbuf / stream->sndbuf (rte_ring per-TCB)
        → POSIX-like socket API: nsocket / nconnect / nsend / nrecv / nclose (tcp_socket.c)
          → WebSocket: ws_connect (HTTP Upgrade) / ws_send_frame / ws_recv_frame (ws_client.c)
            → Application (main.c)
```

### Critical design details

- **Blocking I/O via condition variables**: `nconnect()` and `nrecv()` block the app thread using `pthread_cond_wait`. The pkt_process lcore signals `stream->cond` when state changes or data arrives. `nconnect()` additionally services DPDK RX/TX inline while waiting for the handshake to complete.
- **TCB lookup is O(1)**: 256-bucket hash table on the 4-tuple (sip, dip, sport, dport). Additionally, an `fd → TCB` direct array (`fd_to_tcb[]`) provides O(1) lookup from socket fd.
- **Client→Server frames are MASKED**, Server→Client frames are unmasked (RFC 6455). Masking key is 4 random bytes XORed over the payload.
- **TCP send path**: `nsend()` segments into MSS-sized `ng_tcp_fragment` structs, enqueues to stream->sndbuf ring. `ng_tcp_out()` (called from pkt_process) dequeues them, builds Ethernet/IP/TCP headers (with option bytes from `fragment->option[]`), resolves MAC via ARP table, sends to ring->out.
- **TCP receive path**: `ng_tcp_input()` validates sequence numbers against the receive window, handles in-order data by enqueueing to `stream->rcvbuf` and waking `nrecv()`. Out-of-order segments go to `ofo_head` sorted linked list (max 64 entries), drained when the gap fills.
- **TCP timers** checked per-iteration in pkt_process: RTO with exponential backoff, delayed ACK (200ms), keepalive (2h idle), zero-window probe (1s interval, max 10).
- **Congestion control**: RFC 5681 slow start / congestion avoidance / fast retransmit (3 dupacks) / fast recovery. cwnd initialized to 2 MSS (~2920 bytes).
- **ARP**: Linked-list ARP table in `arp.h`. ARP requests rate-limited to 10/s per destination. Missing MAC causes the fragment to be re-enqueued to sndbuf.
- **Globals**: `gDpdkPortId`, `gLocalIp` (default `192.168.0.115`), `gSrcMac`. Singletons: `tcpInstance()`, `ringInstance()`.

### Key source files

| File | Purpose |
|------|---------|
| `src/main.c` | Entry point, EAL init, cmdline parsing, main loop (RX/TX + WS poll + ping) |
| `src/tcp_core.h` | All structs (TCB, fragment, hash table), constants, function declarations |
| `src/tcp_core.c` | Hash table, packet encoding, ARP/ICMP helpers, pkt_process loop, port init, `ng_tcp_out` |
| `src/tcp_input.c` | TCP option parsing, sequence validation, 11-state FSM, out-of-order queue |
| `src/tcp_output.c` | Congestion control (slow start/CA/fast retx/recovery), option building |
| `src/tcp_timer.c` | RTT estimation (RFC 6298), RTO with backoff, delayed ACK, keepalive, ZWP |
| `src/tcp_socket.c` | POSIX-like socket API with condition-variable blocking |
| `src/ws_client.c` | WebSocket handshake, frame encode/decode (masked send, unmasked recv), ping/pong |
| `src/ws_client.h` | WebSocket opcodes, API declarations |
| `src/arp.h` | ARP table (linked list), LL_ADD/LL_REMOVE macros |

## Tests

- **`test/test_ws_protocol.c`** — WebSocket protocol test using kernel TCP sockets. Connects to a WS server (Python or Java), tests handshake, echo, subscribe/push. No DPDK needed. Usage: `./test_ws_protocol [server_ip] [port]`
- **`test/bench_tcp.c`** — Microbenchmarks for the data-path hot paths: ring buffer SP/SC throughput, hash table lookup, WS frame encode/decode, memcpy baseline. CPU-only, no NIC.
- **`test/ws_test_server.py`** — Python async WS echo/subscribe server for manual testing. Dependencies: `websockets`. Usage: `python3 test/ws_test_server.py [port]`

## Runtime

```bash
# 1. Hugepages
echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
mkdir -p /mnt/huge && mount -t hugetlbfs nodev /mnt/huge

# 2. Bind NIC (use DPDK 24.11 usertools)
modprobe vfio-pci
sudo python3 ~/share/dpdk-study/sources/dpdk-24.11/usertools/dpdk-devbind.py --bind=vfio-pci <pci_addr>

# 3. Run (RPATH baked in, no LD_LIBRARY_PATH needed)
./build/dpdk_ws_client -l 0-1 -n 2 -- -a <SERVER_IP> -p <PORT> -w /ws

# With subscription and local IP:
./build/dpdk_ws_client -l 0-1 -n 2 -- \
    -a 192.168.1.100 -p 8080 -w /ws \
    -s '{"action":"subscribe","topic":"market_data"}' \
    -l 10.0.0.10
```

### CLI parameters (after `--`)

| Flag | Default | Description |
|------|---------|-------------|
| `-a, --addr` | `192.168.1.100` | Server IPv4 |
| `-p, --port` | `8080` | Server port |
| `-w, --path` | `/ws` | WebSocket path |
| `-s, --subscribe` | (none) | Subscribe JSON message |
| `-k, --keepalive` | `30` | Ping interval in seconds (0=off) |
| `-l, --local-ip` | `192.168.0.115` | Local/stub IPv4 |

## Port init

`ng_init_port()` configures a single port (index 0) with 1 RX queue (128 descriptors) and 1 TX queue (512 descriptors), promiscuous mode enabled. DPDK port ID is the global `gDpdkPortId` (default 0). In DPDK 24.11, `max_rx_pkt_len` was replaced by `mtu` in `rte_eth_rxmode`.

## AWS deployment notes

- Use `c5n.large`+ (ENA PMD) or `c5n.metal` (SR-IOV)
- Verify ENA PMD is enabled: `grep ENA $RTE_SDK/x86_64-native-linux-gcc/.config`
- Security groups must allow the target TCP port