Repository navigation
Add pre-built DPDK AMI support to reduce deployment time - #8
Merged
Merged
Conversation
DPDK compilation (3196 objects) in EC2 user-data was exceeding the 20-minute CloudFormation CreationPolicy timeout, causing CREATE_FAILED on integration test instances. This adds a Packer-based AMI builder that pre-installs DPDK 22.11.6, Rust, and all dependencies. The AMI ID is stored in SSM Parameter Store and fetched by the integration test workflow before CDK deploy. Changes: - Add packer/dpdk-ami.pkr.hcl: Packer template for DPDK AMI - Add .github/workflows/build-dpdk-ami.yml: weekly AMI builder workflow - Add scripts/store-ami-id.sh: SSM Parameter Store helper - Modify CDK stack: accept amiId context, bifurcate user-data into pre-built (10 min timeout) vs full bootstrap (25 min timeout) paths - Fix: add CreationPolicy to DpdkReceiver (was previously missing) - Fix: separate userData per instance for correct cfn-signal resources - Modify orchestrator: pass DPDK_AMI_ID env var as CDK context - Modify integration workflow: fetch AMI ID from SSM before deploy - Fix: correct DPDK extracted directory name (dpdk-stable-22.11.6) Estimated time savings: ~13-20 min down to ~4-6 min per instance. Graceful fallback to stock AL2023 when no pre-built AMI exists. https://claude.ai/code/session_013zajD4LZj2bYsY9p6EynWH
Reorder workflow steps so tool installation (Node.js, CDK, SSM plugin) runs before AWS credentials, ensuring the safety net teardown always has CDK dependencies available. Add fail-on-empty: false to test reporter to prevent confusing secondary failures when no XML results exist. Scope safety net teardown to only run when the test step itself fails, not on credential failures where no infrastructure was deployed. https://claude.ai/code/session_013zajD4LZj2bYsY9p6EynWH
package-lock.json is gitignored for deploy/cdk, so npm ci fails in CI. Switch to npm install which works without a lockfile. https://claude.ai/code/session_013zajD4LZj2bYsY9p6EynWH
…path The full DPDK-from-source bootstrap exceeds 25 minutes in CI, causing CloudFormation CreationPolicy timeout. Bumped to 35 minutes for the non-AMI path and increased the GHA job timeout to 55 minutes to match. https://claude.ai/code/session_013zajD4LZj2bYsY9p6EynWH
gspivey
pushed a commit
that referenced
this pull request
Apr 11, 2026
Run #8 of the performance suite showed tokio-dpdk dropping 100% of packets at every PPS step despite the process running and emitting [PERF] samples. Root cause: cargo feature unification across separate invocations. The perf-test-stack user-data script ran two cargo builds back to back: 1) cargo build --release --features dpdk-sys/bindgen 2) cargo build --release -p tokio-echo --features dpdk The second invocation does not include `dpdk-sys/bindgen`, so cargo rebuilds dpdk-stdlib-sys with the `dpdk_stubs` cfg active. The final tokio-echo binary therefore links against stub DPDK — bind succeeds, but rx_burst always returns zero packets, matching the symptoms exactly. Fix: - Make `dpdk` a default feature of tokio-echo so a single workspace build with `--features dpdk-stdlib-sys/bindgen` produces a real-DPDK binary, with no second cargo invocation needed. - Drop the redundant second `cargo build -p tokio-echo` from perf-test-stack.ts. - Skip the DPDK code path entirely in the dpdk-tokio compat layer and in `bind_udp_with_config` when `dpdk_udp::is_stub()` is true. A stub-backed socket binds successfully but produces no packets, so loopback I/O would silently hang. With this guard, plain `cargo test` (which now unifies tokio-echo's `dpdk` default into the workspace) keeps passing on machines without real DPDK installed. - Re-export `is_stub()` from dpdk-stdlib-udp so callers don't need a direct dependency on dpdk-sys. Local builds without DPDK still work — the workspace falls back to stubs and the compat layer transparently uses tokio sockets.
gspivey
pushed a commit
that referenced
this pull request
Apr 11, 2026
Captures the perf-test run that validated the tokio-dpdk fix from Run #8 (100% drops -> real DPDK linkage) and exposes a new finding: the current dpdk-tokio compat shim caps single-socket throughput at ~40K pps because each recv_from/send_to round-trips through tokio::task::spawn_blocking + a Mutex acquisition. App Drops is 0 across the board, confirming the bottleneck is the polling rate (NIC RX ring drops) rather than the dpdk-udp socket buffer. rust-dpdk continues to track native-dpdk within 0.25% at all packet sizes and rates; the App Drops column proves the residual ~3% loss at 64B/700K is upstream of our stack (AWS network), not in the Rust user-space stack.
gspivey
added a commit
that referenced
this pull request
May 20, 2026
## Roadmap item addressed IPv6 task 7: **ICMPv6 echo reply** — auto-respond to `ping6`, parallel to our existing IPv4 ICMP echo reply. ## Changes New module `dpdk-udp/src/icmpv6.rs` providing: - `Icmpv6Packet` — parsed ICMPv6 echo request/reply struct - `parse_icmpv6_packet()` — parse ICMPv6 echo from raw Ethernet frames (handles extension headers) - `build_icmpv6_frame()` / `build_echo6_request()` — frame construction with IPv6 pseudo-header checksum - `icmpv6_checksum()` — RFC 4443 §2.3 checksum using IPv6 pseudo-header - `Icmpv6Handler` — stateful handler that auto-responds to echo requests for registered local IPs Integration into `UdpSocket`: - Added `icmpv6_handler` field, initialized on bind - RX path handles ICMPv6 echo requests when `auto_icmp` is enabled and ethertype is IPv6 ## Tests added (24) - Build/parse roundtrip (echo request, echo reply, empty payload) - Checksum computation, verification, and corruption detection - `make_echo_reply()` address/field swapping - Handler: responds to local IPs, ignores others, supports multiple IPs - Edge cases: rejects IPv4, truncated frames, UDP-over-IPv6, too-short frames - Wire format: ethertype, next header, payload length - Link-local and loopback address handling - Synthetic performance benchmark (10K build+parse cycles) ## Tradeoffs - Only handles echo request/reply (types 128/129). ICMPv6 error handling (type 1-4) is a separate roadmap item (#8). - Handler is initialized with `Ipv6Addr::UNSPECIFIED` since the socket is currently IPv4-only. Once `SocketAddrV6` support lands (task 3), the handler will be initialized with the actual bound IPv6 address. - Reuses the existing `auto_icmp` flag to control ICMPv6 responses (same toggle for v4 and v6). Co-authored-by: Agent Router <agent@agent-router.dev>
gspivey
added a commit
that referenced
this pull request
Jun 8, 2026
## Roadmap Item `dpdk-stdlib-quic`: Event loop (item #8 in ROADMAP.md) ## Changes Implements the core `event_loop` function in `dpdk-stdlib-quic/src/event_loop.rs` (spec tasks 9.1, 9.2): - **Shutdown check**: Reads `AtomicBool` flag each iteration (Relaxed ordering) - **poll_wakeups**: Services application wakeups; breaks on `CloseError` (secondary shutdown when all app handles are dropped) - **RX path**: `recv_frames` → ICMP dispatch via `IcmpHandler::process_icmp_full()` → `parse_to_rx_datagram` → `endpoint.receive()` - **TX path**: `endpoint.transmit()` → `drain()` → `send_frame()` - **Busy-poll-with-cooldown**: Tracks idle cycles, sleeps until `min(endpoint.timeout(), now + 1ms)` after budget exceeded; 100µs fallback when no timeout Also provides `event_loop_with_tx_queue` variant for the provider to supply a pre-configured TxQueue with the correct gateway MAC. ## Tests Added 6 unit tests using a `MockEndpoint` and `LoopbackBackend`: - `shutdown_flag_stops_loop` — verifies AtomicBool shutdown terminates the loop - `close_error_stops_loop` — verifies CloseError from poll_wakeups terminates - `rx_path_delivers_datagrams` — valid UDP frames reach the endpoint - `rx_path_drops_wrong_port` — frames for wrong port are silently discarded - `stats_counters_increment` — all counter fields increment correctly - `icmp_dispatch_sends_reply` — ICMP echo requests dispatched (not delivered as UDP) - `event_loop_with_tx_queue_uses_provided_queue` — variant works ## Verification - `cargo build && cargo test` passes (756+ tests, 0 failures) - No changes to existing crates or public APIs - No regressions in existing test suite --------- Co-authored-by: Agent Router <agent@agent-router.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces support for using a pre-built DPDK AMI to significantly reduce EC2 instance startup time during integration tests. Instead of always building DPDK from source (25 minutes), deployments can now optionally use a pre-built AMI that has DPDK, Rust, and system dependencies pre-installed (10 minutes for cargo build only).
Key Changes
CDK Stack Refactoring (
deploy/cdk/lib/dpdk-test-stack.ts)amiIdto optionally specify a pre-built AMIcreateUserData()helper function that conditionally includes:Packer Template for AMI Building (
packer/dpdk-ami.pkr.hcl)GitHub Actions Workflow (
/.github/workflows/build-dpdk-ami.yml)/dpdk-stdlib-rust/ami/latest)Integration Test Updates
integration-tests.ymlto fetch pre-built AMI ID from SSM Parameter Storerun-integration-tests.shto passamiIdcontext to CDK when availableUtility Script (
scripts/store-ami-id.sh)put,get, andcheckoperations for local testingDPDK Install Script Enhancement (
scripts/install_dpdk_amazon_linux.sh)Implementation Details
https://claude.ai/code/session_013zajD4LZj2bYsY9p6EynWH