Skip to content

fix(test): use single-underscore env for ADP listen addresses#1964

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits into
mainfrom
jszwedko/fix-integration-dataplane-port-isolation-single-underscore
Jul 1, 2026
Merged

fix(test): use single-underscore env for ADP listen addresses#1964
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits into
mainfrom
jszwedko/fix-integration-dataplane-port-isolation-single-underscore

Conversation

@jszwedko

@jszwedko jszwedko commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

With the addition of the data_plane keys to the Agent schema, ADP was reading the defaults rather than the test configuration due to the Agent not recognizing the __ variants of environment variables like DD_DATA_PLANE__API_LISTEN_ADDRESS. This PR updates the test config to use the single-underscore variants.

For standalone mode, saluki-config already supports falling back to reading the single-underscore variants (like DD_DATA_PLANE_API_LISTEN_ADDRESS) if the double-underscore variant isn't set. A test is added to assert this.

Converged mode — the Core Agent reads DD_DATA_PLANE_API_LISTEN_ADDRESS (single-underscore, viper form) and propagates it as the authoritative data_plane.api_listen_address via the config stream. ADP resolves the nested key on the first try.

Standalone mode — no config stream; from_environment("DD_") splits on __ and produces the flat figment key data_plane_api_listen_address. get() retries on MissingField with the flat form and resolves it. A regression test (added to saluki-config) documents and protects this fallback.

Note

DD_DATA_PLANE__API_LISTEN_ADDRESS (double-underscore) remains fully functional — the nested key is checked before the flat fallback, so existing scripts using that form continue to work.

Test plan

  • Added test_underscore_fallback_on_get_multi_segment_key to saluki-config, confirming try_get_typed("data_plane.api_listen_address") resolves from a single-underscore env var via the existing get() fallback.
  • All 5 previously-failing converged tests pass against agent-dev:master-py3-full with base-config ADP (no config.rs change): privileged-api-endpoints, unprivileged-api-endpoints, telemetry-endpoint, adp-config-dynamic-basic. (dogstatsd-forwarding forwarding all passes; only remaining failure is the separate regression Bump pin-project from 1.1.4 to 1.1.5 #1 tls classifier bug.)
  • 3 standalone-mode tests (basic-startup, adp-memory-mode-strict-within-limit, otlp-traces-enabled) pass — ADP resolves the shifted listen address via the flat-key fallback.

🤖 Generated with Claude Code

@datadog-datadog-us1-prod

This comment has been minimized.

@pr-commenter

pr-commenter Bot commented Jul 1, 2026

Copy link
Copy Markdown

Binary Size Analysis (Agent Data Plane)

Baseline: 4b4f85a · Comparison: e40ac8e · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 40.85 MiB (baseline) vs 40.85 MiB (comparison)
Size Change: -2.02 KiB (-0.00%)

✅ Binary size difference within threshold

Changes by Module
Module File Size Symbols
anon.1ba911e98cbb868a3910d01d81cd64e3.368.llvm.7557474435476924149 +3.37 KiB 1
anon.1ba911e98cbb868a3910d01d81cd64e3.368.llvm.2532428867121011575 -3.37 KiB 1
anon.8f296a3c82e85bd0d7746bc6848d9ce4.845.llvm.11000487001760752350 -2.38 KiB 1
anon.8f296a3c82e85bd0d7746bc6848d9ce4.845.llvm.16743963508081758158 +2.38 KiB 1
core -1.86 KiB 1015
anon.fbb1dc57653c0fdbc830c6923948c66d.512.llvm.2857851886350349888 +1.77 KiB 1
anon.fbb1dc57653c0fdbc830c6923948c66d.512.llvm.441118841613340164 -1.77 KiB 1
anon.07c4bb9ce5ce61a5c762f558e396468b.71.llvm.14285062802314426476 -1.69 KiB 1
anon.ed68a82a9d28fb52a623006f1bfcf487.510.llvm.838646141026691578 +1.69 KiB 1
std +1.47 KiB 35
anon.9af47ef3bc1f734053efc1a81f309c81.30.llvm.17299067839495397339 +1.42 KiB 1
anon.27bb284c8b1ed68ed68b22f69b936876.736.llvm.15117208460917206197 -1.42 KiB 1
anon.0f0acd27514c68ee01ad19c408265557.94.llvm.17649990376026517311 -1.26 KiB 1
anon.3e8abb0d43609c327bb6ada2c11ec275.72.llvm.10747916521025211526 -1.26 KiB 1
anon.42e3673cf5fb968fa55278145836f476.85.llvm.10225881335234927411 +1.26 KiB 1
anon.f83bcaa464063749734ddb56ddb3647a.145.llvm.8450999564668775366 +1.26 KiB 1
anon.e777ac5f6f7c99cb724a59ab8f35dc54.25.llvm.17909052499754804510 -1.16 KiB 1
anon.8f833c134f13c50302a91565a1218676.1202.llvm.712691169012533130 +1.16 KiB 1
anon.b691578cf4f15388151d7ecef487b24d.335.llvm.15002232744562391779 +1.14 KiB 1
anon.260484999791c0d9044b988483a1bd33.106.llvm.2470759439810408397 -1.14 KiB 1
Detailed Symbol Changes
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW] +5.28Ki  [NEW]     +33    core::ptr::drop_in_place<http_body_util::combinators::map_err::MapErr<http_body_util::combinators::map_err::MapErr<http_body_util::combinators::map_frame::MapFrame<tonic::service::interceptor::ResponseBody<tonic::body::Body>,tonic::codec::decode::Streaming<datadog_protos::agent_include::datadog::model::v1::FetchEntityResponse>::new<tonic::service::interceptor::ResponseBody<tonic::body::Body>,tonic_prost::codec::ProstDecoder<datadog_protos::agent_include::datadog::model::v1::FetchEntityResponse>>::{{closure}}>,tonic::codec::decode::Streaming<datadog_protos::agent_include::datadog::model::v1::FetchEntityResponse>::new<tonic::service::interceptor::ResponseBody<tonic::body::Body>,tonic_prost::codec::ProstDecoder<datadog_protos::agent_include::datadog::model::v1::FetchEntityResponse>>::{{closure}}>,tonic::status::Status::map_error<tonic::status::Status>>>::hf2b4abfcbde41aa8
  [NEW] +4.72Ki  [NEW]     +49    core::ptr::drop_in_place<tower::util::map_future::MapFuture<axum::util::MapIntoResponse<tower::util::map_request::MapRequest<otlp_protos::otlp_include::opentelemetry::proto::collector::logs::v1::logs_service_server::LogsServiceServer<saluki_components::common::otlp::GrpcServiceImpl<saluki_components::relays::otlp::RelayHandler>>,tonic::service::router::Routes::add_service<otlp_protos::otlp_include::opentelemetry::proto::collector::logs::v1::logs_service_server::LogsServiceServer<saluki_components::common::otlp::GrpcServiceImpl<saluki_components::relays::otlp::RelayHandler>>>::{{closure}}>>,tower::util::boxed_clone_sync::BoxCloneSyncService<http::request::Request<axum_core::body::Body>,http::response::Response<axum_core::body::Body>,core::convert::Infallible>::new<axum::util::MapIntoResponse<tower::util::map_request::MapRequest<otlp_protos::otlp_include::opentelemetry::proto::collector::logs::v1::logs_service_se
  [NEW] +3.37Ki  [NEW] +3.28Ki    anon.1ba911e98cbb868a3910d01d81cd64e3.368.llvm.7557474435476924149
  [NEW] +2.38Ki  [NEW]     +76    anon.8f296a3c82e85bd0d7746bc6848d9ce4.845.llvm.16743963508081758158
  [NEW] +2.32Ki  [NEW]     +47    _<http_body_util::combinators::map_err::MapErr<B,F> as http_body::Body>::size_hint::h69664899300ec2cb
  [NEW] +2.04Ki  [NEW] +1.62Ki    _<hyper_timeout::TimeoutConnector<T> as tower_service::Service<http::uri::Uri>>::call::_{{closure}}::h0819d82129f8d524
  [NEW] +1.77Ki  [NEW] +1.68Ki    anon.fbb1dc57653c0fdbc830c6923948c66d.512.llvm.2857851886350349888
  [NEW] +1.69Ki  [NEW]     +90    anon.ed68a82a9d28fb52a623006f1bfcf487.510.llvm.838646141026691578
  [NEW] +1.54Ki  [NEW]     +36    saluki_core::runtime::supervisor::Supervisable::shutdown_strategy::hc583455dedd5e9a7
  [NEW] +1.42Ki  [NEW]     +90    anon.9af47ef3bc1f734053efc1a81f309c81.30.llvm.17299067839495397339
  [DEL] -1.42Ki  [DEL]     -90    anon.27bb284c8b1ed68ed68b22f69b936876.736.llvm.15117208460917206197
  [DEL] -1.54Ki  [DEL]     -36    saluki_core::runtime::supervisor::Supervisable::shutdown_strategy::h10b24582feeb7c48
  [DEL] -1.69Ki  [DEL]     -90    anon.07c4bb9ce5ce61a5c762f558e396468b.71.llvm.14285062802314426476
  [DEL] -1.77Ki  [DEL] -1.68Ki    anon.fbb1dc57653c0fdbc830c6923948c66d.512.llvm.441118841613340164
  -0.1% -2.01Ki  [ = ]       0    [7729 Others]
  [DEL] -2.05Ki  [DEL] -1.62Ki    _<hyper_timeout::TimeoutConnector<T> as tower_service::Service<http::uri::Uri>>::call::_{{closure}}::hd3c5b5e8b2a3b2a8
  [DEL] -2.32Ki  [DEL]     -47    _<http_body_util::combinators::map_err::MapErr<B,F> as http_body::Body>::size_hint::h0ca514cb30748a6a
  [DEL] -2.38Ki  [DEL]     -76    anon.8f296a3c82e85bd0d7746bc6848d9ce4.845.llvm.11000487001760752350
  [DEL] -3.37Ki  [DEL] -3.28Ki    anon.1ba911e98cbb868a3910d01d81cd64e3.368.llvm.2532428867121011575
  [DEL] -4.72Ki  [DEL]     -49    core::ptr::drop_in_place<tower::util::map_future::MapFuture<axum::util::MapIntoResponse<tower::util::map_request::MapRequest<otlp_protos::otlp_include::opentelemetry::proto::collector::metrics::v1::metrics_service_server::MetricsServiceServer<saluki_components::common::otlp::GrpcServiceImpl<saluki_components::relays::otlp::RelayHandler>>,tonic::service::router::Routes::add_service<otlp_protos::otlp_include::opentelemetry::proto::collector::metrics::v1::metrics_service_server::MetricsServiceServer<saluki_components::common::otlp::GrpcServiceImpl<saluki_components::relays::otlp::RelayHandler>>>::{{closure}}>>,tower::util::boxed_clone_sync::BoxCloneSyncService<http::request::Request<axum_core::body::Body>,http::response::Response<axum_core::body::Body>,core::convert::Infallible>::new<axum::util::MapIntoResponse<tower::util::map_request::MapRequest<otlp_protos::otlp_include::opentelemetry::proto::collector::metrics
  [DEL] -5.28Ki  [DEL]     -33    core::ptr::drop_in_place<http_body_util::combinators::map_err::MapErr<http_body_util::combinators::map_err::MapErr<http_body_util::combinators::map_frame::MapFrame<tonic::service::interceptor::ResponseBody<tonic::body::Body>,tonic::codec::decode::Streaming<datadog_protos::agent_include::datadog::remoteagent::v1::RegisterRemoteAgentResponse>::new<tonic::service::interceptor::ResponseBody<tonic::body::Body>,tonic_prost::codec::ProstDecoder<datadog_protos::agent_include::datadog::remoteagent::v1::RegisterRemoteAgentResponse>>::{{closure}}>,tonic::codec::decode::Streaming<datadog_protos::agent_include::datadog::remoteagent::v1::RegisterRemoteAgentResponse>::new<tonic::service::interceptor::ResponseBody<tonic::body::Body>,tonic_prost::codec::ProstDecoder<datadog_protos::agent_include::datadog::remoteagent::v1::RegisterRemoteAgentResponse>>::{{closure}}>,tonic::status::Status::map_error<tonic::statu
  -0.0% -2.02Ki  [ = ]       0    TOTAL

…rt isolation

The Datadog Agent (7.82+) owns data_plane.api_listen_address and
secure_api_listen_address and propagates them to ADP via the config
stream in converged mode, overriding env vars set only on ADP.
Panoramic's port-isolation env used double-underscore figment forms
(DD_DATA_PLANE__*) which the Core Agent does not recognize, so the
Agent pushed its own defaults (5100/5101) through the stream and
overrode the shifted ports.

Switching to single-underscore Agent-style vars (DD_DATA_PLANE_*) fixes
both modes:
  - Converged: the Core Agent reads and propagates the shifted ports via
    the config stream.
  - Standalone: GenericConfiguration::get() already falls back from the
    nested dotted key to the flat underscore form on MissingField
    (lib/saluki-config/src/lib.rs), so single-underscore vars resolve
    directly with no additional plumbing.

The double-underscore forms are removed since the existing get() fallback
makes them redundant in both modes. A regression test is added to
saluki-config to document and protect this fallback.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@jszwedko jszwedko force-pushed the jszwedko/fix-integration-dataplane-port-isolation-single-underscore branch from bd1f400 to 6c5d8bd Compare July 1, 2026 18:31
@dd-octo-sts dd-octo-sts Bot added the area/config Configuration. label Jul 1, 2026
@jszwedko jszwedko marked this pull request as ready for review July 1, 2026 18:36
@jszwedko jszwedko requested a review from a team as a code owner July 1, 2026 18:36
@jszwedko jszwedko changed the title fix(config,integration): accept single-underscore env for ADP listen addresses [alt approach] fix(test): use single-underscore env for ADP listen addresses Jul 1, 2026

@webern webern left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much less scary than #1958 and #1959!

Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
@pr-commenter

pr-commenter Bot commented Jul 1, 2026

Copy link
Copy Markdown

Regression Detector (Agent Data Plane)

Run ID: faa39e28-90ee-4217-8301-c460b84d40b1
Baseline: 4b4f85ac · Comparison: e40ac8e5 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
quality_gates_rss_dsd_ultraheavy memory ⚪ +0.13 metrics profiles logs
quality_gates_rss_idle memory ⚪ -0.06 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ -0.29 metrics profiles logs
quality_gates_rss_dsd_medium memory ⚪ -0.37 metrics profiles logs
quality_gates_rss_dsd_heavy memory ⚪ -0.42 metrics profiles logs
Bounds Checks: ✅ Passed (5)
experiment check replicates observed links
quality_gates_rss_dsd_heavy memory_usage 10/10 ✅ 138 MiB ≤ 140 MiB metrics profiles logs
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 42.7 MiB ≤ 50 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 64.5 MiB ≤ 75 MiB metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 ✅ 191 MiB ≤ 200 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 28.5 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit 3e2ba4d into main Jul 1, 2026
85 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the jszwedko/fix-integration-dataplane-port-isolation-single-underscore branch July 1, 2026 19:34
dd-octo-sts Bot pushed a commit that referenced this pull request Jul 1, 2026
## Summary

With the addition of the `data_plane` keys to the Agent schema, ADP was reading the defaults rather than the test configuration due to the Agent not recognizing the `__` variants of environment variables like `DD_DATA_PLANE__API_LISTEN_ADDRESS`. This PR updates the test config to use the single-underscore variants.

For standalone mode, `saluki-config` already supports falling back to reading the single-underscore variants (like `DD_DATA_PLANE_API_LISTEN_ADDRESS`) if the double-underscore variant isn't set. A test is added to assert this.

**Converged mode** — the Core Agent reads `DD_DATA_PLANE_API_LISTEN_ADDRESS` (single-underscore, viper form) and propagates it as the authoritative `data_plane.api_listen_address` via the config stream. ADP resolves the nested key on the first try.

**Standalone mode** — no config stream; `from_environment("DD_")` splits on `__` and produces the flat figment key `data_plane_api_listen_address`. `get()` retries on `MissingField` with the flat form and resolves it. A regression test (added to saluki-config) documents and protects this fallback.

> [!NOTE]
> `DD_DATA_PLANE__API_LISTEN_ADDRESS` (double-underscore) remains fully functional — the nested key is checked before the flat fallback, so existing scripts using that form continue to work.

## Test plan

- [x] Added `test_underscore_fallback_on_get_multi_segment_key` to saluki-config, confirming `try_get_typed("data_plane.api_listen_address")` resolves from a single-underscore env var via the existing `get()` fallback.
- [x] All 5 previously-failing converged tests pass against `agent-dev:master-py3-full` with base-config ADP (no `config.rs` change): `privileged-api-endpoints`, `unprivileged-api-endpoints`, `telemetry-endpoint`, `adp-config-dynamic-basic`. (`dogstatsd-forwarding` forwarding all passes; only remaining failure is the separate regression #1 tls classifier bug.)
- [x] 3 standalone-mode tests (`basic-startup`, `adp-memory-mode-strict-within-limit`, `otlp-traces-enabled`) pass — ADP resolves the shifted listen address via the flat-key fallback.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: jesse.szwedko <jesse.szwedko@datadoghq.com> 3e2ba4d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants