chore(config): integrate and test the config system

[webern]

Human Summary

Integrates the typed config system into agent-data-plane even though no components are yet reading from it. We decided to take a migration approach to this, so components will be cut over to using typed config in isolated PRs where we can focus on correctness for each component.

This PR introduces a privileged API endpoint /config/internal which is a serialization of SalukiConfig. This is going to be important for end-to-end integration testing, and the first such test is added in this PR.

Also noteworthy is that all integration and correctness tests are passing here, which was not the case prior to #2093's improvement of how the type config system handles environment variables.

AI Summary

Integrate the typed configuration system into the Agent Data Plane runtime and expose its current value through the internal configuration endpoint.

• Load the configuration system after the initial configuration snapshot is available.
• Thread the configuration-system handle through runtime setup and the internal supervisor.
• Serve the current configuration through /config/internal.
• Use fallback environment-overlay behavior so Agent-provided values remain authoritative.
• Add integration coverage for runtime configuration updates through the internal endpoint.

Change Type

• Non-functional (chore, refactoring, docs)

How did you test this PR?

• make build-schema-overlay
• make fmt
• make check-docs
• make check-all
• make test

All checks passed; 1,830 tests passed and 31 were skipped.

References

Stacked on #2093.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[pr-commenter]

Binary Size Analysis (Agent Data Plane)

Baseline: b7de287 · Comparison: 096d1e7 · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 41.64 MiB (baseline) vs 43.25 MiB (comparison)
Size Change: +1.61 MiB (+3.87%)

✅ Binary size difference within threshold

Changes by Module

Module	File Size	Symbols
core	+278.29 KiB	9432
serde_json	+255.58 KiB	528
figment	+145.44 KiB	596
datadog_agent_config::generated::datadog_configuration	+113.89 KiB	55
serde_core	+85.76 KiB	1030
alloc	+77.46 KiB	1528
[sections]	+64.23 KiB	8
agent_data_plane_config_system::saluki_only::_	+38.83 KiB	7
otlp_protos::otlp_include::opentelemetry	-34.15 KiB	133
anyhow	+29.85 KiB	1151
axum	+26.81 KiB	310
serde_yaml	+25.26 KiB	60
agent_data_plane_config_system::translators::datadog_translator	+24.79 KiB	22
saluki_common::task::instrument	+24.54 KiB	55
hyper	+23.97 KiB	292
hashbrown	+20.71 KiB	861
hyper_util	-18.50 KiB	40
anon.45df57af26d64948470a716a4dc22ad3.783.llvm.9042083944440476780	+18.24 KiB	1
anon.fc8e370c0e42eec6379fb44c26631c45.1631.llvm.16529597611186777515	+18.04 KiB	1
prost	+17.57 KiB	271

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
   +10% +1.45Mi   +10% +1.11Mi    [35718 Others]
  [NEW] +57.9Ki  [NEW] +57.8Ki    saluki_components::common::datadog::io::run_endpoint_io_loop::_{{closure}}::h04bb0a9b21e007fd
  [NEW] +53.6Ki  [NEW] +53.2Ki    _<datadog_agent_config::generated::datadog_configuration::_::<impl serde_core::de::Deserialize for datadog_agent_config::generated::datadog_configuration::DatadogConfiguration>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::he7d50082300f7f0b
  [NEW] +43.4Ki  [NEW] +43.3Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h51e81e52049f41c3
  [NEW] +43.3Ki  [NEW] +43.1Ki    _<saluki_components::forwarders::otlp::OtlpForwarder as saluki_core::components::forwarders::Forwarder>::run::_{{closure}}::hf025f6405521b68a
  +3.7% +41.8Ki  +3.7% +41.8Ki    [section .gcc_except_table]
  [NEW] +35.7Ki  [NEW] +35.5Ki    _<saluki_components::transforms::apm_stats::ApmStats as saluki_core::components::transforms::Transform>::run::_{{closure}}::h4441c01b4416e7be
  [NEW] +34.4Ki  [NEW] +34.1Ki    _<agent_data_plane_config_system::saluki_only::_::<impl serde_core::de::Deserialize for agent_data_plane_config_system::saluki_only::SalukiOnly>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::ha97ee5a261f6887b
  [NEW] +32.6Ki  [NEW] +32.5Ki    saluki_components::sources::otlp::metrics::translator::OtlpMetricsTranslator::translate_metrics::hc785bcbcb692f512
  [NEW] +30.7Ki  [NEW] +30.6Ki    agent_data_plane::cli::dogstatsd::handle_dogstatsd_command::_{{closure}}::had003efd1563f3dc
  [NEW] +29.5Ki  [NEW] +29.3Ki    _<saluki_components::transforms::aggregate::Aggregate as saluki_core::components::transforms::Transform>::run::_{{closure}}::hedcff8de2f282b2e
  [NEW] +29.4Ki  [NEW] +29.2Ki    _<saluki_components::sources::dogstatsd::_::<impl serde_core::de::Deserialize for saluki_components::sources::dogstatsd::DogStatsDConfiguration>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::h785c681d04d5e213
  [NEW] +28.2Ki  [NEW] +28.1Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h15aa50dc45cb4950
  [DEL] -27.7Ki  [DEL] -27.6Ki    saluki_components::sources::dogstatsd::drive_stream::_{{closure}}::hd69617ec6298c4df
  [DEL] -28.1Ki  [DEL] -27.9Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h29bc4bf05508540a
  [DEL] -29.1Ki  [DEL] -28.9Ki    _<saluki_components::transforms::aggregate::Aggregate as saluki_core::components::transforms::Transform>::run::_{{closure}}::ha8992029ccc3e467
  [DEL] -30.1Ki  [DEL] -30.0Ki    agent_data_plane::cli::dogstatsd::handle_dogstatsd_command::_{{closure}}::h9a00c8b722d068b2
  [DEL] -39.4Ki  [DEL] -39.2Ki    _<saluki_components::forwarders::otlp::OtlpForwarder as saluki_core::components::forwarders::Forwarder>::run::_{{closure}}::h95f34135b15b6453
  [DEL] -40.8Ki  [DEL] -40.6Ki    saluki_components::sources::otlp::metrics::translator::OtlpMetricsTranslator::translate_metrics::hf98cf32915d606dd
  [DEL] -43.9Ki  [DEL] -43.8Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h0fb925ad706e10ff
  [DEL] -57.5Ki  [DEL] -57.3Ki    saluki_components::common::datadog::io::run_endpoint_io_loop::_{{closure}}::h8b7f8c5d97ba6139
  +3.9% +1.61Mi  +3.8% +1.27Mi    TOTAL

[pr-commenter]

Regression Detector (Agent Data Plane)

Run ID: 62d4c396-296a-41b6-936c-7a951d0f51db
Baseline: b7de2875 · Comparison: 096d1e77 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (3)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment	goal	Δ mean %	links
quality_gates_rss_idle	memory	⚪ +3.72	metrics profiles logs
quality_gates_rss_dsd_low	memory	⚪ +2.85	metrics profiles logs
quality_gates_rss_dsd_medium	memory	⚪ +2.22	metrics profiles logs

Bounds Checks: ✅ Passed (3)

experiment	check	replicates	observed	links
quality_gates_rss_dsd_low	memory_usage	10/10	✅ 44.1 MiB ≤ 50 MiB	metrics profiles logs
quality_gates_rss_dsd_medium	memory_usage	10/10	✅ 66.5 MiB ≤ 75 MiB	metrics profiles logs
quality_gates_rss_idle	memory_usage	10/10	✅ 29.8 MiB ≤ 40 MiB	metrics profiles logs

Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[webern]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• chore(config): cutover adp bin components to typed config system #1995
• chore(config): cutover saluki-components to the new typed configuration system #1987
• chore(config): build the first components from the typed configuration model #1979
• chore(config): integrate and test the config system #2094 👈 (View in Graphite)
• chore(config): support environment variables in the config system #2093
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.