Skip to content

chore(antithesis): Pin all rig container dependencies#2017

Draft
blt wants to merge 2 commits into
blt/chore_antithesis_dogstatsd_line_and_payload_generatorsfrom
blt/chore_antithesis_pin_all_rig_container_dependencies
Draft

chore(antithesis): Pin all rig container dependencies#2017
blt wants to merge 2 commits into
blt/chore_antithesis_dogstatsd_line_and_payload_generatorsfrom
blt/chore_antithesis_pin_all_rig_container_dependencies

Conversation

@blt

@blt blt commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Summary

This commit modifies the rig so that we are always using Datadog Agent +
ADP as the SUT. In the general case this is ADP-on and in the
differential case this is between ADP-off and ADP-on. We make one small
modification to the Datadog Agent container in that we do not restart
ADP if it faults, giving antithesis a cleaner exit signal.

I have tried to make all our containers sync'ed properly with the rest
of the project, as they had drifted. If I understand the auto-update
proecess correctly this should all stay square from now on. From Vector
I have ported the launch script and, hopefully, this makes it a little
more clear how to launch these things. Of interest I have changed the
endpoint and turned OFF node faults for both scenarios as we cannot
ensure that, especially in the differential case, that faults happen at
the same instant and are fair.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

How did you test this PR?

References

@datadog-datadog-prod-us1

This comment has been minimized.

@pr-commenter

pr-commenter Bot commented Jul 6, 2026

Copy link
Copy Markdown

Regression Detector (Agent Data Plane)

Run ID: bcb4bfdd-17d6-4ae0-802e-d8498f2bcc6b
Baseline: dff16546 · Comparison: bc500ec5 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
quality_gates_rss_dsd_medium memory ⚪ +0.16 metrics profiles logs
quality_gates_rss_idle memory ⚪ -0.02 metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory ⚪ -0.22 metrics profiles logs
quality_gates_rss_dsd_heavy memory ⚪ -0.32 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ -0.46 metrics profiles logs
Bounds Checks: ✅ Passed (5)
experiment check replicates observed links
quality_gates_rss_dsd_heavy memory_usage 10/10 ✅ 137 MiB ≤ 140 MiB metrics profiles logs
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 43 MiB ≤ 50 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 64.9 MiB ≤ 75 MiB metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 ✅ 191 MiB ≤ 200 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 29 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from f116dac to c2c153c Compare July 7, 2026 12:35
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch from c545860 to a41b667 Compare July 7, 2026 12:36
@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from c2c153c to eb51b1e Compare July 7, 2026 12:48
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch from a41b667 to 60b0777 Compare July 7, 2026 12:49

@tobz tobz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, do you have any examples of what things looked like before this?

It would make sense to me if we had an out-of-date pinned version of ca-certificates, but I'm not seeing how the versionless dependency reference would end up giving us an out-of-date version... and given that, pinning everything feels a bit labored/fragile.

@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from eb51b1e to 672554d Compare July 7, 2026 13:14
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch from 60b0777 to 321d5b0 Compare July 7, 2026 13:14
@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from 672554d to e9fb9a3 Compare July 7, 2026 22:36
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch from 321d5b0 to 2dbcec5 Compare July 7, 2026 22:36
@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from e9fb9a3 to c5cbc5a Compare July 7, 2026 22:51
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch from 2dbcec5 to 83e3b6f Compare July 7, 2026 22:51
@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from c5cbc5a to 732b36f Compare July 7, 2026 23:06
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch from 83e3b6f to f820609 Compare July 7, 2026 23:06
@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from 732b36f to 65c8858 Compare July 8, 2026 00:59
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch 2 times, most recently from 0a91d85 to 8bfacdd Compare July 8, 2026 01:28
@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from 65c8858 to 896334b Compare July 8, 2026 01:28
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch from 8bfacdd to 4b25d9b Compare July 8, 2026 02:09
@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch 2 times, most recently from ec739c1 to 5b529b3 Compare July 8, 2026 02:11
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch from 4b25d9b to 58adc65 Compare July 8, 2026 02:11
@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from 5b529b3 to c727931 Compare July 8, 2026 19:01
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch from 58adc65 to 342357b Compare July 8, 2026 19:01
@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from c727931 to e8b9907 Compare July 8, 2026 19:01
@dd-octo-sts dd-octo-sts Bot added the area/test All things testing: unit/integration, correctness, SMP regression, etc. label Jul 8, 2026
@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from e8b9907 to 4b59000 Compare July 8, 2026 19:02
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch from 342357b to dabe522 Compare July 8, 2026 19:09
@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from 4b59000 to 3800920 Compare July 8, 2026 20:28
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch from dabe522 to 174ee7e Compare July 8, 2026 20:28
@dd-octo-sts dd-octo-sts Bot removed the area/test All things testing: unit/integration, correctness, SMP regression, etc. label Jul 8, 2026
This commit updates the harness driver to build dogstatsd to a byte
limit, in the manner of datadog/lading. I have been unable to
investigate SMPTNG-7611 well owing to the error log emission by Datadog
Agent. I've added property tests to assert the payload limit is obeyed,
accepting that this means making the dogstatsd generator pure with
regard to Rng and may not use antithesis SDK's random_* directly.
@blt blt force-pushed the blt/chore_antithesis_dogstatsd_line_and_payload_generators branch from 3800920 to 9fe3971 Compare July 8, 2026 21:26
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch 2 times, most recently from 3e468fb to 51b0168 Compare July 8, 2026 22:53
@dd-octo-sts dd-octo-sts Bot added the area/test All things testing: unit/integration, correctness, SMP regression, etc. label Jul 8, 2026
@blt

blt commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Hmmm, do you have any examples of what things looked like before this?

It would make sense to me if we had an out-of-date pinned version of ca-certificates, but I'm not seeing how the versionless dependency reference would end up giving us an out-of-date version... and given that, pinning everything feels a bit labored/fragile.

@tobz Ah yeah so this is me misunderstanding the deal with AWS-LC, agreed on your read. I'm going to rework this PR to address the artifact mismatch under test, will ping when this is ready to go.

@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch 3 times, most recently from e9baf6d to 4eb20eb Compare July 8, 2026 23:10
@pr-commenter

pr-commenter Bot commented Jul 8, 2026

Copy link
Copy Markdown

Binary Size Analysis (Agent Data Plane)

Baseline: dff1654 · Comparison: bc500ec · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 41.59 MiB (baseline) vs 41.59 MiB (comparison)
Size Change: -8 B (-0.00%)

✅ Binary size difference within threshold

Changes by Module
Module File Size Symbols
anon.104b1f618a14d6d60f2ebe1b651d81f5.1.llvm.10234976937094269617 -130 B 1
anon.104b1f618a14d6d60f2ebe1b651d81f5.1.llvm.1510176363995787189 +129 B 1
anon.104b1f618a14d6d60f2ebe1b651d81f5.4.llvm.10234976937094269617 -115 B 1
anon.104b1f618a14d6d60f2ebe1b651d81f5.4.llvm.1510176363995787189 +114 B 1
anon.104b1f618a14d6d60f2ebe1b651d81f5.3.llvm.10234976937094269617 -109 B 1
anon.104b1f618a14d6d60f2ebe1b651d81f5.3.llvm.1510176363995787189 +108 B 1
anon.104b1f618a14d6d60f2ebe1b651d81f5.0.llvm.10234976937094269617 -97 B 1
anon.104b1f618a14d6d60f2ebe1b651d81f5.0.llvm.1510176363995787189 +96 B 1
anon.104b1f618a14d6d60f2ebe1b651d81f5.2.llvm.10234976937094269617 -95 B 1
anon.104b1f618a14d6d60f2ebe1b651d81f5.2.llvm.1510176363995787189 +94 B 1
[Unmapped] -3 B 1
Detailed Symbol Changes
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW]    +129  [NEW]     +40    anon.104b1f618a14d6d60f2ebe1b651d81f5.1.llvm.1510176363995787189
  [NEW]    +114  [NEW]     +25    anon.104b1f618a14d6d60f2ebe1b651d81f5.4.llvm.1510176363995787189
  [NEW]    +108  [NEW]     +19    anon.104b1f618a14d6d60f2ebe1b651d81f5.3.llvm.1510176363995787189
  [NEW]     +96  [NEW]      +7    anon.104b1f618a14d6d60f2ebe1b651d81f5.0.llvm.1510176363995787189
  [NEW]     +94  [NEW]      +5    anon.104b1f618a14d6d60f2ebe1b651d81f5.2.llvm.1510176363995787189
  -0.0%      -3  [ = ]       0    [Unmapped]
  [DEL]     -95  [DEL]      -5    anon.104b1f618a14d6d60f2ebe1b651d81f5.2.llvm.10234976937094269617
  [DEL]     -97  [DEL]      -7    anon.104b1f618a14d6d60f2ebe1b651d81f5.0.llvm.10234976937094269617
  [DEL]    -109  [DEL]     -19    anon.104b1f618a14d6d60f2ebe1b651d81f5.3.llvm.10234976937094269617
  [DEL]    -115  [DEL]     -25    anon.104b1f618a14d6d60f2ebe1b651d81f5.4.llvm.10234976937094269617
  [DEL]    -130  [DEL]     -40    anon.104b1f618a14d6d60f2ebe1b651d81f5.1.llvm.10234976937094269617
  -0.0%      -8  [ = ]       0    TOTAL

@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch 2 times, most recently from f23db7c to 6f51ee0 Compare July 9, 2026 00:10
This commit modifies the rig so that we are always using Datadog Agent +
ADP as the SUT. In the general case this is ADP-on and in the
differential case this is between ADP-off and ADP-on. We make one small
modification to the Datadog Agent container in that we do not restart
ADP if it faults, giving antithesis a cleaner exit signal.

I have tried to make all our containers sync'ed properly with the rest
of the project, as they had drifted. If I understand the auto-update
proecess correctly this should all stay square from now on. From Vector
I have ported the launch script and, hopefully, this makes it a little
more clear how to launch these things. Of interest I have changed the
endpoint and turned OFF node faults for both scenarios as we cannot
ensure that, especially in the differential case, that faults happen at
the same instant and are fair.
@blt blt force-pushed the blt/chore_antithesis_pin_all_rig_container_dependencies branch from 6f51ee0 to bc500ec Compare July 9, 2026 01:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/test All things testing: unit/integration, correctness, SMP regression, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants