chore(antithesis): Parallel drivers retry sends by blt · Pull Request #1950 · DataDog/saluki

blt · 2026-06-30T01:17:37Z

Summary

We modify parallel driver to tolerate both backpressure and network
faults. The goal of this work is to ensure we do not delver to one SUT
and not the other. All payloads rip through or the driver exits with notice.

Change Type

Bug fix
New feature
Non-functional (chore, refactoring, docs)
Performance

How did you test this PR?

References

blt · 2026-06-30T01:17:52Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

pr-commenter · 2026-06-30T01:25:50Z

Binary Size Analysis (Agent Data Plane)

Baseline: 1d0f918 · Comparison: a23df6b · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 41.63 MiB (baseline) vs 41.63 MiB (comparison)
Size Change: -8 B (-0.00%)

✅ Binary size difference within threshold

Changes by Module

Module	File Size	Symbols
`anon.c332a4a573d0a9a1bf7c71b0a1f57672.1.llvm.18056271019418515100`	-130 B	1
`anon.c332a4a573d0a9a1bf7c71b0a1f57672.1.llvm.4618899617986136913`	+129 B	1
`anon.c332a4a573d0a9a1bf7c71b0a1f57672.4.llvm.18056271019418515100`	-115 B	1
`anon.c332a4a573d0a9a1bf7c71b0a1f57672.4.llvm.4618899617986136913`	+114 B	1
`anon.c332a4a573d0a9a1bf7c71b0a1f57672.3.llvm.18056271019418515100`	-109 B	1
`anon.c332a4a573d0a9a1bf7c71b0a1f57672.3.llvm.4618899617986136913`	+108 B	1
`anon.c332a4a573d0a9a1bf7c71b0a1f57672.0.llvm.18056271019418515100`	-97 B	1
`anon.c332a4a573d0a9a1bf7c71b0a1f57672.0.llvm.4618899617986136913`	+96 B	1
`anon.c332a4a573d0a9a1bf7c71b0a1f57672.2.llvm.18056271019418515100`	-95 B	1
`anon.c332a4a573d0a9a1bf7c71b0a1f57672.2.llvm.4618899617986136913`	+94 B	1
`[Unmapped]`	-3 B	1

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW]    +129  [NEW]     +40    anon.c332a4a573d0a9a1bf7c71b0a1f57672.1.llvm.4618899617986136913
  [NEW]    +114  [NEW]     +25    anon.c332a4a573d0a9a1bf7c71b0a1f57672.4.llvm.4618899617986136913
  [NEW]    +108  [NEW]     +19    anon.c332a4a573d0a9a1bf7c71b0a1f57672.3.llvm.4618899617986136913
  [NEW]     +96  [NEW]      +7    anon.c332a4a573d0a9a1bf7c71b0a1f57672.0.llvm.4618899617986136913
  [NEW]     +94  [NEW]      +5    anon.c332a4a573d0a9a1bf7c71b0a1f57672.2.llvm.4618899617986136913
  -0.1%      -3  [ = ]       0    [Unmapped]
  [DEL]     -95  [DEL]      -5    anon.c332a4a573d0a9a1bf7c71b0a1f57672.2.llvm.18056271019418515100
  [DEL]     -97  [DEL]      -7    anon.c332a4a573d0a9a1bf7c71b0a1f57672.0.llvm.18056271019418515100
  [DEL]    -109  [DEL]     -19    anon.c332a4a573d0a9a1bf7c71b0a1f57672.3.llvm.18056271019418515100
  [DEL]    -115  [DEL]     -25    anon.c332a4a573d0a9a1bf7c71b0a1f57672.4.llvm.18056271019418515100
  [DEL]    -130  [DEL]     -40    anon.c332a4a573d0a9a1bf7c71b0a1f57672.1.llvm.18056271019418515100
  -0.0%      -8  [ = ]       0    TOTAL

pr-commenter · 2026-06-30T01:40:51Z

Regression Detector (Agent Data Plane)

Run ID: b2368b34-06e8-4c38-97dd-f1b95815524e
Baseline: 1d0f9180 · Comparison: a23df6ba · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment	goal	Δ mean %	links
quality_gates_rss_dsd_low	memory	⚪ +0.07	metrics profiles logs
quality_gates_rss_dsd_heavy	memory	⚪ -0.10	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory	⚪ -0.11	metrics profiles logs
quality_gates_rss_idle	memory	⚪ -0.11	metrics profiles logs
quality_gates_rss_dsd_medium	memory	⚪ -0.27	metrics profiles logs

Bounds Checks: ✅ Passed (5)

experiment	check	replicates	observed	links
quality_gates_rss_dsd_heavy	memory_usage	10/10	✅ 133 MiB ≤ 140 MiB	metrics profiles logs
quality_gates_rss_dsd_low	memory_usage	10/10	✅ 43.2 MiB ≤ 50 MiB	metrics profiles logs
quality_gates_rss_dsd_medium	memory_usage	10/10	✅ 64.8 MiB ≤ 75 MiB	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory_usage	10/10	✅ 192 MiB ≤ 200 MiB	metrics profiles logs
quality_gates_rss_idle	memory_usage	10/10	✅ 29 MiB ≤ 40 MiB	metrics profiles logs

Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 72a0072d45

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 091f1f41fd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: da39e20e09

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 345a39cece

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

We modify parallel driver to tolerate both backpressure and network faults. The goal of this work is to ensure we do not delver to one SUT and not the other. All payloads rip through or the driver fails loudly.

Copilot

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.

dd-octo-sts Bot added the area/test All things testing: unit/integration, correctness, SMP regression, etc. label Jun 30, 2026

This was referenced Jun 30, 2026

chore(antithesis): Differential comparison between ADP and Datadog Agent context egress #1891

Open

chore(antithesis): LogLevel set only to 'error' #1937

Merged

This was referenced Jun 30, 2026

chore(antithesis): Generate configs for scenario purposes #1938

Merged

chore(antithesis): Track dropped transactions in as sometimes! assertion #1949

Open

This comment has been minimized.

Sign in to view

blt force-pushed the blt/chore_antithesis_parallel_drivers_retry_sends branch from a2c5426 to 45fa303 Compare June 30, 2026 01:55

blt force-pushed the blt/chore_antithesis_track_dropped_transactions_in_as_sometimes_assertion branch from 07709cc to 349aec9 Compare June 30, 2026 01:56

blt changed the base branch from blt/chore_antithesis_track_dropped_transactions_in_as_sometimes_assertion to graphite-base/1950 June 30, 2026 14:22

blt force-pushed the graphite-base/1950 branch from 349aec9 to 67ec949 Compare June 30, 2026 14:22

blt force-pushed the blt/chore_antithesis_parallel_drivers_retry_sends branch from 45fa303 to 285e5f1 Compare June 30, 2026 14:22

blt changed the base branch from graphite-base/1950 to blt/chore_antithesis_generate_configs_for_scenario_purposes June 30, 2026 14:22

blt requested a review from Copilot June 30, 2026 14:27

Copilot AI reviewed Jun 30, 2026

blt force-pushed the blt/chore_antithesis_generate_configs_for_scenario_purposes branch from 67ec949 to da4c048 Compare June 30, 2026 14:35

blt force-pushed the blt/chore_antithesis_parallel_drivers_retry_sends branch from 285e5f1 to 72a0072 Compare June 30, 2026 14:35

blt marked this pull request as ready for review June 30, 2026 14:46

blt requested review from a team as code owners June 30, 2026 14:46

Copilot AI review requested due to automatic review settings June 30, 2026 14:46

Copilot AI reviewed Jun 30, 2026

chatgpt-codex-connector Bot reviewed Jun 30, 2026

View reviewed changes

Comment thread test/antithesis/harness/src/driver.rs

blt force-pushed the blt/chore_antithesis_parallel_drivers_retry_sends branch from 72a0072 to 091f1f4 Compare June 30, 2026 15:05

chatgpt-codex-connector Bot reviewed Jun 30, 2026

View reviewed changes

Comment thread test/antithesis/harness/src/driver.rs Outdated

blt force-pushed the blt/chore_antithesis_generate_configs_for_scenario_purposes branch 2 times, most recently from c4bd83e to 51d3177 Compare June 30, 2026 15:21

blt force-pushed the blt/chore_antithesis_parallel_drivers_retry_sends branch from 091f1f4 to 04c5b5d Compare June 30, 2026 15:21

blt force-pushed the blt/chore_antithesis_parallel_drivers_retry_sends branch from 04c5b5d to da39e20 Compare July 1, 2026 15:56

Copilot AI review requested due to automatic review settings July 1, 2026 15:56

blt force-pushed the blt/chore_antithesis_generate_configs_for_scenario_purposes branch from 51d3177 to c0e4ead Compare July 1, 2026 15:56

Copilot started reviewing on behalf of blt July 1, 2026 15:56 View session

This was referenced Jul 1, 2026

chore(antithesis): Add two new Pyld properties #1960

Open

chore(antithesis): Add Pyld25, series are not empty #1961

Open

Copilot AI reviewed Jul 1, 2026

View reviewed changes

Comment thread test/antithesis/harness/src/driver.rs

chatgpt-codex-connector Bot reviewed Jul 1, 2026

View reviewed changes

Comment thread test/antithesis/scenarios/general/src/bin/parallel_driver_send_dogstatsd.rs

blt mentioned this pull request Jul 2, 2026

chore(antithesis): dogstatsd generation with byte limit #1974

Open

4 tasks

blt force-pushed the blt/chore_antithesis_parallel_drivers_retry_sends branch from da39e20 to 9e132a2 Compare July 2, 2026 13:44

blt force-pushed the blt/chore_antithesis_generate_configs_for_scenario_purposes branch from c0e4ead to 9ceef13 Compare July 2, 2026 13:44

blt changed the base branch from blt/chore_antithesis_generate_configs_for_scenario_purposes to graphite-base/1950 July 2, 2026 16:55

Copilot AI review requested due to automatic review settings July 2, 2026 16:55

blt force-pushed the graphite-base/1950 branch from 9ceef13 to 1d0f918 Compare July 2, 2026 16:55

graphite-app Bot changed the base branch from graphite-base/1950 to main July 2, 2026 16:55

blt force-pushed the blt/chore_antithesis_parallel_drivers_retry_sends branch 2 times, most recently from b109532 to 9284e75 Compare July 2, 2026 16:55

Copilot AI reviewed Jul 2, 2026

View reviewed changes

Comment thread test/antithesis/harness/src/driver.rs

blt force-pushed the blt/chore_antithesis_parallel_drivers_retry_sends branch 2 times, most recently from 9899f1c to 345a39c Compare July 2, 2026 22:08

chatgpt-codex-connector Bot reviewed Jul 2, 2026

View reviewed changes

Comment thread test/antithesis/harness/src/driver.rs

chore(antithesis): Parallel drivers retry sends

a23df6b

We modify parallel driver to tolerate both backpressure and network faults. The goal of this work is to ensure we do not delver to one SUT and not the other. All payloads rip through or the driver fails loudly.

blt force-pushed the blt/chore_antithesis_parallel_drivers_retry_sends branch from 345a39c to a23df6b Compare July 2, 2026 22:41

blt requested a review from Copilot July 2, 2026 22:45

Copilot started reviewing on behalf of blt July 2, 2026 22:46 View session

Copilot AI reviewed Jul 2, 2026

View reviewed changes

Comment thread test/antithesis/harness/src/driver.rs

Comment thread test/antithesis/harness/src/driver.rs

Comment thread test/antithesis/harness/src/driver.rs

Uh oh!

Conversation

blt commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Change Type

How did you test this PR?

References

Uh oh!

blt commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

pr-commenter Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Binary Size Analysis (Agent Data Plane)

✅ Binary size difference within threshold

Uh oh!

pr-commenter Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Regression Detector (Agent Data Plane)

Optimization Goals: ✅ No significant changes detected

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

blt commented Jun 30, 2026 •

edited

Loading

blt commented Jun 30, 2026 •

edited

Loading

pr-commenter Bot commented Jun 30, 2026 •

edited

Loading

pr-commenter Bot commented Jun 30, 2026 •

edited

Loading