125 lines
8.4 KiB
Markdown
Executable File
125 lines
8.4 KiB
Markdown
Executable File
# DPDK → DPU Growth Roadmap
|
||
|
||
> **Single source of truth.** Supersedes all previous Chinese roadmap docs (deleted 2026-09-04, recoverable via git history).
|
||
>
|
||
> **Goal:** top-tier network data-plane engineer (DPU / AI-networking), offer from ByteDance/Ali by mid-2027.
|
||
> **Start:** 2026-08-21 (Day 1) · **Timeline:** 8–10 months, stretch to 2027-06.
|
||
> **Effort:** weekdays 1.5–2h + weekends 4–6h (≈ 20h/week).
|
||
> **Philosophy:** done > perfect · honest timelines · quality > speed.
|
||
|
||
---
|
||
|
||
## 1. Current Focus — AI-Networking Interview Sprint (4 weeks, parallel track)
|
||
|
||
*Now (from 2026-09-02). Runs alongside the main track. Output-driven: every week produces one blog post.*
|
||
|
||
**Why this is a re-naming exercise, not a pivot** — existing knowledge maps directly:
|
||
|
||
| You already know | AI-network equivalent | Difficulty |
|
||
|---|---|---|
|
||
| TCP congestion control (RTO/cwnd, from libmtcp-v3) | DCQCN (RDMA congestion control) | ⭐ very low |
|
||
| Loss = congestion signal (TCP) | Loss = disaster (RDMA, PFC) | ⭐ very low |
|
||
| ECMP / L3 (daily work) | Fat-Tree multi-pathing | ⭐ very low |
|
||
| SAI / chip pipeline (strongest area) | PFC/ECN chip-side implementation | ⭐ low |
|
||
| DPDK data-plane mindset | DPU/SmartNIC offload, collective-comm acceleration | ⭐ low |
|
||
| TCP FSM from scratch | RoCEv2 transport semantics (QP/verbs/retransmit) | ⭐⭐ medium |
|
||
|
||
**Week 0 — Panorama.** Why AI training is network-bound (AllReduce stalls 30–50% of time in 1000-GPU clusters); topology evolution (Spine-Leaf → Fat-Tree → Rail-Optimized); 800G/CPO/LPO; RDMA; lossless Ethernet.
|
||
*Output:* blog post "Why GPU Clusters Eat Networks: Starting from AllReduce" (concept-level). *Gate:* draw the GPU-cluster communication stack and explain it to a layperson.
|
||
|
||
**Week 1 — RDMA & lossless networking (the interview core).** RDMA semantics (QP/verb, zero-copy); RoCEv2 vs InfiniBand; **PFC** (priority flow control, storms, deadlocks, PFC watchdog); **ECN + DCQCN** (explicit congestion signaling, AIMD curve vs TCP Reno — your strongest talking point).
|
||
*Output:* "DCQCN: Congestion Control's RDMA Cousin — Why It Looks Familiar" (side-by-side with libmtcp-v3 experience).
|
||
*Gate:* draw how a PFC deadlock happens; talk 10 min on DCQCN vs TCP CC.
|
||
|
||
**Week 2 — Collective communication & NCCL.** Three AllReduce generations (Ring / Tree / Hierarchical) and their bandwidth/delay models; NCCL topology awareness over Fat-Tree; compute-communication overlap.
|
||
*Output:* "Ring AllReduce × Fat-Tree: How Communication Patterns Dictate Network Design".
|
||
*Gate:* derive Ring AllReduce per-step transfer = (N−1)/N × data; explain why NCCL dislikes unordered topologies.
|
||
|
||
**Week 3 — SONiC/chip side (your home turf, where you win).** PFC/ECN config surface in SONiC; PFC watchdog; buffer model & headroom math; INT (in-band telemetry); where DPU sits in AI networking.
|
||
*Output:* "SONiC on AI Switches: Familiar Control Plane, Unfamiliar Lossless Forwarding" (chip-view author — rare position).
|
||
*Gate:* name where PFC/ECN/buffer intervene on one AllReduce flow end-to-end.
|
||
|
||
**Week 4 — Interview sprint.** Benchmark literacy (perftest/ib_write_bw/NCCL-test); 20-question self-quiz (3-min oral each, recorded); resume rewritten with AI-networking vocabulary; attach the 4 blog posts as portfolio.
|
||
|
||
**Sprint reading list (each repo = one article, three-layer peeling, never the whole repo):**
|
||
- `linux-rdma/rdma-core` `libibverbs/verbs.h` (the API contract) · `linux-rdma/perftest` `ib_write_bw.c` (RDMA "ping")
|
||
- Kernel SoftRoCE: `drivers/infiniband/sw/rxe/{rxe_verbs,rxe_req,rxe_resp}.c` — **runs real RDMA between two VMs without special NICs**
|
||
- `bobzhuyb/ns3-rdma`: DCQCN/TIMELY/HPCC implementations (few hundred lines each)
|
||
- `NVIDIA/nccl` `src/graph/` (CPU side only) · `facebookincubator/gloo` (Ring/Tree AllReduce, read first)
|
||
- `sonic-net/sonic-swss` `orchagent/pfcwd/` + `orchagent/buffer` (lossless config lives here; you read it faster than papers)
|
||
- `OpenSmartNIC` / `corundum` (SmartNIC/FPGA data-plane reference)
|
||
|
||
**Papers:** DCQCN (SIGCOMM'15) 🔴 · Revisiting Network Support for RDMA (SIGCOMM'18) 🔴 · Deadlocks in Datacenter Networks (HotNets'16) 🔴 · HPCC (SIGCOMM'19) 🟡 · TIMELY (SIGCOMM'15) 🟡 · RoCEv2 spec (scan the congestion chapter) ⚪
|
||
|
||
**Sprint rules:** study only what is "interview-asked + article-writable" (no IB protocol minutiae, no optical-layer physics) · every concept gets a hand-drawn diagram · ≤3 reads per paper or switch to easier material · Friday self-quiz: 5 random kb/ cards, 3-min oral each.
|
||
|
||
---
|
||
|
||
## 2. Main Track — 6 Phases (≈ 252 days)
|
||
|
||
*Dates are cumulative from 2026-08-21; they slide 1:1 with slips — the monthly review (section 4) is the correction mechanism.*
|
||
|
||
| Phase | Days | Duration | Core deliverable |
|
||
|---|---|---|---|
|
||
| **P0 Foundations** | 1–14 | 2 wk | Hardware grounding + MPSC lock-free queue passing 2M-enqueue test (hard gate) |
|
||
| **P1 DPDK internals** | 15–42 | 4 wk | Deep understanding of the 5 subsystems (mempool/ring/mbuf/PMD/EAL/RSS-flow) vs kernel counterparts |
|
||
| **P2 libmtcp-v3** | 43–126 | 10–12 wk | Userspace TCP stack: v0.1 single-conn echo → v0.2 multi-conn + basic FSM → v0.3 perf tuning → v0.4 API + RSS + release |
|
||
| **P3 SAI-DPDK Mini** | 127–168 | 6 wk | switch/port + FDB/VLAN + route/neighbor + simplified L2/L3 pipeline + mock control plane |
|
||
| **P4 PCIe + DPU theory** | 169–196 | 4 wk | PCIe driver on X540 (probe/ioremap/MSI-X/DMA) + DMA/IOMMU/VFIO + virtio/vhost/vDPA theory |
|
||
| **P5 Job-search loop** | 197–252 | 8 wk | Resume v3 → small-company batch → feedback → H3C/云豹/中科驭数 batch → big-company batch (Ali/Byte/Tencent) → offer |
|
||
|
||
**Scope discipline (what was cut from the naive plan):**
|
||
- P2: **no** congestion control / SACK / PAWS / epoll / reordering — interview answer: "next steps". TCP corner cases (zero-window probe, TIME_WAIT assassination, timer interactions) are why 10–12 weeks is honest.
|
||
- P3: **no** ACL / QoS / ECMP(kept simplified) / real syncd integration — mock control plane only.
|
||
- P4: X540 is a plain NIC, not a DPU — claim "read mlx5 vdpa code, wrote a PCIe driver", never "DPU hands-on experience".
|
||
- P5: first batch after P3 completes (not during P4) — coding + interviewing in parallel breaks both.
|
||
|
||
**P0 completion gate (hard):** MPSC queue 2P1C, 2M enqueues, zero loss · can draw NIC→CPU packet path · perf stat/record/flamegraph basics · buddy/slab vs mempool.
|
||
|
||
---
|
||
|
||
## 3. Weekly Rhythm
|
||
|
||
**Weekdays (1.5–2h):** 15 min review yesterday → the day's ONE core task → 30 min notes + git commit.
|
||
**Weekends (4–6h):** Sat AM big coding block · Sat PM test/debug · Sun AM notes + blog post · Sun PM buffer (catch-up or rest).
|
||
**Rules:** one core task per day · if missed, mark slipped — never chase with all-nighters · Sunday is the buffer · update progress at week end, adjust next week.
|
||
|
||
---
|
||
|
||
## 4. Monthly Stop-Loss Review
|
||
|
||
- ≥70% of plan done → keep pace.
|
||
- 30–50% done → find why, cut scope or extend next month.
|
||
- Two consecutive months ≤30% → stop, re-evaluate the whole direction.
|
||
|
||
Checklist: days completed vs plan · what slipped and why · next-month adjustments · health (sleep/exercise/mood).
|
||
|
||
---
|
||
|
||
## 5. Testing Strategy
|
||
|
||
- **P2 libmtcp-v3:** unit tests per function (scapy-crafted packets) · integration with dperf · corner cases via iptables drop/reorder/delay · perf stat + flame graphs.
|
||
- **P3 SAI-DPDK Mini:** control-plane script tests (SAI API returns) · scapy data-plane verification · 1000-flow soak for leaks · benchmark vs kernel bridge/OVS.
|
||
- **Sprint experiments:** SoftRoCE (rxe) + perftest between two VMs — real RDMA traffic without RDMA NICs; reproduce PFC on/off difference with tc rate limiting.
|
||
|
||
---
|
||
|
||
## 6. Principles
|
||
|
||
1. Direction right, timeline honest, scope halved, done > perfect.
|
||
2. This is an 8–10 month marathon, not a 6-month sprint.
|
||
3. Interview feedback > closed-door polishing: ship early, fix from feedback.
|
||
4. A project you can explain beats a rushed one you can't.
|
||
5. Health > progress. Long-termism.
|
||
|
||
---
|
||
|
||
## 7. Resources
|
||
|
||
- `dpdk-roadmap-v3/books/` (13 ebooks) · `dpdk-roadmap-v3/references/` (5 papers + 9 tech docs) · `dpdk-roadmap-v3/code/` (artifacts)
|
||
- `src/` — 6 vendored source trees, each its own git repo (DPDK 25.11.3, OVS 4.0.0, VPP 26.06, rdma-core 64.0, perftest, tcp-stack)
|
||
- `kb/` — knowledge cards (3/day during sprints) · `blog-site/` — all output posts
|
||
- Career docs: `career-planning/` (decision report, Gallup profiles)
|
||
|
||
*Execution artifacts live in git commits — the roadmap above is the only planning document.*
|