Repository navigation
feat(dpdk-stdlib-tcp): real-NIC engine driver so DPDK TCP DUTs actually run - #99
Merged
Merged
Conversation
…ly run The TCP apps called TcpListener::bind / TcpStream::connect, which require a process-wide TcpContext + a running engine — but nothing built either, so an IPv4 bind failed with "TCP context not initialized". This wires up the missing runtime and fixes two engine bugs the work surfaced. - dpdk-stdlib-net: add DpdkBackend::new_real_nic(port_id, eal_args) which inits EAL WITHOUT --no-pci (mirrors dpdk-stdlib-udp's get_or_init_dpdk), so a real vfio-pci ENI is actually probed. DpdkBackend::new (--no-pci) is a null device that never sees the NIC — using it for TCP/QUIC was the root cause of "real DPDK" paths touching nothing. Serialize the EAL-lifecycle backend tests (#[serial]) since the stub's EAL state is process-global. - dpdk-stdlib-tcp: new runtime module with init_dpdk_tcp_context() — builds the backend (new_real_nic) + ArpResolver(gateway_mac) + engine + spawns the single engine-driver thread, then publishes the global TcpContext. - Engine bug 1 (zeroed MACs): the driver now extracts the Ethernet src/dst MAC from the raw frame and calls on_segment_with_macs, so a server's SYN-ACK is addressed to the real peer (gateway) MAC instead of 00:..:00. Same fix applied to stream.rs::engine_loop. - Engine bug 2 (dead command sender): accept-side ConnectionHandles got a throwaway dummy CommandSender, so Close/Shutdown/SetOption from accepted streams went nowhere and writes never woke the engine. Add TcpEngine::with_cmd_tx and thread the live sender through handle_syn / handle_syn_with_macs. - apps: tcp-echo, tokio-tcp-echo, tcp-test-client gain --gateway-mac / --eal-args (+ server --perf-interval, client --local-ip) and call init_dpdk_tcp_context before bind/connect. plain-tcp-echo (kernel) is unchanged. Tests: cargo build --workspace clean; cargo test --workspace = 1031 passed / 0 failed. New unit tests: drive_once SYN-ACK targets the peer MAC (MAC regression), with_cmd_tx threads the live sender to the accept handle, new_real_nic smoke test. Perf-runner token wiring (rust-dpdk-tcp / tokio-dpdk-tcp) is deferred to the perf-runner PR so the shared run-perf-tests.sh doesn't collide across PRs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Synthetic Performance Results — Graviton (run)Commit: ✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01) Synthetic UDP Performance ResultsMeasures framework overhead: sync IPv4 Baseline
IPv6
IPv6 vs IPv4 Comparison (sync path)
IPv4 avg sync/async ratio: 0.9x, worst: 1.0x | IPv6 vs IPv4 worst ratio: 1.35x (OK)
|
Synthetic Performance Results (run)Commit: ✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01) Synthetic UDP Performance ResultsMeasures framework overhead: sync IPv4 Baseline
IPv6
IPv6 vs IPv4 Comparison (sync path)
IPv4 avg sync/async ratio: 0.9x, worst: 1.1x | IPv6 vs IPv4 worst ratio: 1.36x (OK)
|
[CI] Stage: DeployInfrastructure ready.
|
[CI] Stage: DeployInfrastructure ready.
|
[CI] Stage: SummaryAll tests PASSED. ARP seeding: kernel /proc/net/arp (automatic)
|
1 similar comment
[CI] Stage: SummaryAll tests PASSED. ARP seeding: kernel /proc/net/arp (automatic)
|
✅ Integration Tests Passed (Run 28446092989)Branch: Test Results
Application Logs (last 20 lines)receiver-echo-server.log sender-echo-server.log sender-test-client.log receiver-test-client-iperf.log sender-test-client-iperf.log Full Application Logs (last 200 lines each)receiver-echo-server.logsender-echo-server.logsender-test-client.logreceiver-test-client-iperf.logsender-test-client-iperf.log
|
✅ Integration Tests Passed — Graviton (run)Branch: Test Results
Application Logs (last 20 lines)receiver-echo-server.log sender-echo-server.log sender-test-client.log |
gspivey
added a commit
that referenced
this pull request
Jul 1, 2026
) ## Summary Makes QUIC benchmarkable over a **real DPDK NIC** (not just in-process loopback) and keeps the existing loopback bench intact. **Stacked on #99** (for `DpdkBackend::new_real_nic`). GitHub will retarget this PR to `development` when #99 merges. Merge #99 first. ### The bug this fixes The QUIC provider built its backend via `create_backend`/`DpdkBackend::new`, which init EAL with `--no-pci` — so it **never probes the vfio-pci ENI** (a null device). Every "native-dpdk" QUIC path (including the existing `quic-echo-server`/`quic-test-client` integration tier) was therefore running against a null device, and `ProviderConfig.eal_args` was dead config that never reached EAL. ### Changes - **`provider.rs`** — `start()` now builds the backend via `DpdkBackend::new_real_nic(port_id, eal_args)` for `Dpdk`/`Auto` backend types (real PCI probe; honors `eal_args`). `backend_override` (loopback bench) still takes precedence, so `bench.rs` is unaffected. New `ProviderBuilder::with_dpdk_port`. - **`quic-echo-server`** — `--eal-args`/`--dpdk-port`, and a `--throughput` mode that stream-echoes each chunk (for long-lived sustained streams). Cert-PEM stdout protocol + `QUIC_SERVER_READY` marker unchanged. - **`quic-perf-client`** (new bin) — drives N bidi streams for a fixed duration and prints one machine-parseable `PERF_RESULT gbps=.. bytes=.. hs_us=.. rx_bursts=.. tx_bursts=.. rx_drops=.. tx_drops=..` line. ### Validation (toolchain 1.91.0, warm cache) - `cargo build --workspace` — clean - `cargo test -p dpdk-stdlib-quic` — **63 passed / 0 failed** (45 lib unit incl. new real-backend-path test; 1 perf-client `gbps` math; loopback handshake/backend/ecn/gso/provider-init integration tests all green → loopback bench unaffected) ### Notes / follow-ups - v1 measures request/response goodput (one payload in flight per stream), parallelised across `--streams`. Full pipelining + TX batching/GSO are future work; the backend also allocs/copies one mbuf per datagram, which bounds line rate. - The 2-instance orchestration (`run-quic-perf.sh`) and the `run-perf-tests.sh` `quic-*` tokens are intentionally in the **next** PR (perf-runner wiring), to keep the shared script out of the parallel feature PRs. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gspivey
added a commit
that referenced
this pull request
Jul 1, 2026
…nfigs tokens (no YAML) (#101) ## Summary Wires **QUIC IPv4**, **TCP IPv4**, and **UDP IPv6** performance tests into the existing perf runner so they're dispatchable through `perf-tests.yml` **with zero `.github/workflows` edits** — the workflow is `workflow_dispatch` with a free-text `configs` input, so new `--configs` tokens decoded inside `run-perf-tests.sh` are all that's needed. This deliberately sidesteps the `workflow`-scope push blocker (the same wall PR #97 is stuck behind). All new tokens are **purely additive** — existing UDP behavior and the default `CONFIGS` are unchanged. ## New `--configs` tokens | Token | Suite | DUT | Benchmark | |---|---|---|---| | `rust-dpdk-tcp` | TCP v4 | `tcp-echo` (DPDK) | TRex ASTF | | `tokio-dpdk-tcp` | TCP v4 | `tokio-tcp-echo` (DPDK) | TRex ASTF | | `plain-rust-tcp` | TCP v4 | `plain-tcp-echo` (kernel) | TRex ASTF | | `native-dpdk-v6` | UDP v6 | testpmd 5tswap | TRex STL (v6) | | `rust-dpdk-v6` | UDP v6 | `echo` (DPDK) | TRex STL (v6) | | `quic-stock` | QUIC | `quic-bench` loopback (stock) | in-process | | `quic-native-dpdk` | QUIC | `quic-bench` loopback (native) | in-process | | `quic-native-dpdk-nic` | QUIC | `quic-echo-server`/`quic-perf-client` | real-NIC, 2-instance | ## How it works - **TCP**: `run_benchmark_for_config` branches on the `-tcp` suffix to the ASTF (stateful) `run_tcp_benchmark.py` (`--payload-sizes`/`--cps-rates`) instead of the STL/UDP `run_benchmark.py`. New `start_dut_*_tcp` launchers reuse `TREX_GATEWAY_MAC` for the DUT `--gateway-mac` (same VPC subnet → same router MAC). - **UDP IPv6**: any `-v6` token auto-enables `IP_VERSION=6` so the existing Phase 2c assigns v6 addresses; `run_benchmark.py` already auto-detects v6 from `--dst-ip`. `native-dpdk-v6` reuses testpmd (IP-agnostic); `rust-dpdk-v6` runs `echo` with `--gateway-mac`/`--peer-ip` NDP seed. - **QUIC**: an early shim in `main()` short-circuits before any PerfTestStack deploy (QUIC is app-to-app — TRex can't speak QUIC) and delegates to `run-quic-benchmarks.sh` (loopback) / the new **`scripts/run-quic-perf.sh`** (real-NIC, DpdkTestStack, cert transfer + `PERF_RESULT` scrape). - **Reporting**: v6 tokens added to the UDP tables; a new **TCP (ASTF)** results table; `aggregate_results` made dict-safe for the list-shaped TCP/QUIC `results`. ## Run it ```bash gh workflow run perf-tests.yml --ref <branch> -f configs=tcp-native-dpdk # ⇒ rust-dpdk-tcp etc. gh workflow run perf-tests.yml --ref <branch> -f configs=rust-dpdk-tcp,tokio-dpdk-tcp,plain-rust-tcp gh workflow run perf-tests.yml --ref <branch> -f configs=native-dpdk-v6,rust-dpdk-v6 gh workflow run perf-tests.yml --ref <branch> -f configs=quic-stock,quic-native-dpdk,quic-native-dpdk-nic ``` ## Validation - `bash -n` clean on `run-perf-tests.sh` + `run-quic-perf.sh` - `python3 -m py_compile` clean on `run_benchmark.py` + `run_tcp_benchmark.py` - All embedded Python heredocs extracted and compiled OK - (Real perf runs need AWS — validated by dispatch after merge.) ## Dependencies Needs **#98** (IPv6/NDP), **#99** (TCP engine driver), **#100** (real-NIC QUIC) merged for the DUT binaries/args these tokens invoke. Scripts-only PR — no `.github/workflows` edit, no `workflow`-scope push required. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the DPDK TCP DUT apps actually runnable on a real NIC. They called
TcpListener::bind/TcpStream::connect, which require a process-wideTcpContextand a running engine — but nothing built either, so an IPv4 bind failed with "TCP context not initialized". This is the keystone that unblocksrust-dpdk-tcp/tokio-dpdk-tcpperf.Changes
dpdk-stdlib-net— the real-NIC backend (--no-pcifix)DpdkBackend::new_real_nic(port_id, eal_args)that initialises EAL without--no-pci(mirrorsdpdk-stdlib-udp'sget_or_init_dpdk), so a real vfio-pci ENI is actually probed.DpdkBackend::newhardcodes--no-pci→ a null device that never sees the NIC. Using it for TCP/QUIC was why "real DPDK" paths touched nothing. (QUIC's real-NIC PR will reusenew_real_nic.)#[serial]— the stub's EAL state is process-global, so concurrent init/cleanup races mempool creation.dpdk-stdlib-tcp— the engine driver + two bug fixesruntimemodule:init_dpdk_tcp_context(cfg)builds backend (new_real_nic) +ArpResolver(gateway_mac)+TcpEngine, spawns the single engine-driver thread (RX→engine, commands→engine, timers→engine), and publishes the globalTcpContext.on_segment_with_macs, so a server's SYN-ACK targets the real peer (gateway) MAC instead of00:..:00. Same fix applied tostream.rs::engine_loop.ConnectionHandles got a throwaway dummyCommandSender— Close/Shutdown/SetOption from accepted streams went nowhere and writes never woke the engine. AddedTcpEngine::with_cmd_txand threaded the live sender throughhandle_syn/handle_syn_with_macs.apps
tcp-echo,tokio-tcp-echo,tcp-test-clientgain--gateway-mac/--eal-args(+ server--perf-interval, client--local-ip) and callinit_dpdk_tcp_contextbefore bind/connect.plain-tcp-echo(kernel) is unchanged.Tests
cargo build --workspace— cleancargo test --workspace— 1031 passed / 0 faileddrive_onceSYN-ACK targets the peer MAC (MAC regression),with_cmd_txthreads the live sender to the accept handle,new_real_nicsmoke test.Notes
rust-dpdk-tcp/tokio-dpdk-tcpinrun-perf-tests.sh) is deferred to the perf-runner PR so the shared script doesn't collide across the parallel feature PRs..github/workflowschanges.🤖 Generated with Claude Code