chore(config): cutover adp bin components to typed config system by webern · Pull Request #1995 · DataDog/saluki

webern · 2026-07-05T19:06:43Z

Human Summary

Cut over some of the agent-data-plane binary components to the new typed config system.

As it turns out, many of the GenericConfiguration-reading components and sites in agent-data-plane are difficult to cut over without further work. Many are involved in bootstrapping and remote agent set-up. Some are pretty complicated (#1998). But for those that seemed pretty reasonable to cut over to typed config, this PR does so.

AI Summary

Continues the agent-data-plane cutover from the raw GenericConfiguration map to the typed
SalukiConfiguration model. Two components move over, plus a supporting consolidation of duration
parsing.

OTTL transform processor. OttlTransformConfiguration::from_configuration now takes the
resolved domains::traces::OttlTransform settings and copies them, instead of deserializing an
OttlTransformConfig out of GenericConfiguration. The component-local config module (its
ErrorMode enum and OttlTransformConfig struct) is removed in favor of the model's shared
OttlErrorMode. This was the last raw-config consumer in
add_baseline_traces_pipeline_to_blueprint, so its now-unused GenericConfiguration parameter is
dropped. The ottl_transform_config seed source in saluki_only.rs is aligned with the OTTL
filter's: error_mode deserializes directly into OttlErrorMode (replacing a loose
Option<String> and a hand-rolled parse helper) and it rejects unknown fields.
Host tags. expected_tags_duration is witnessed and already modeled at
shared.tags.expected_tags_duration, so HostTagsConfiguration reads it directly from the typed
slice rather than parsing a DurationString off GenericConfiguration; the hand-written zero
default is dropped in favor of the generated schema default. RemoteAgentClientConfiguration is
still shared by not-yet-migrated remote-agent components, so it stays on GenericConfiguration,
built at the call site and passed in. This removes GenericConfiguration from the component
entirely.
Duration parsing consolidation. The viper/cast coercion that saluki_config::DurationString
performed (trim whitespace, parse the Go duration grammar, else treat a bare integer as
nanoseconds; reject negatives and values past the time.Duration i64::MAX-nanosecond bound) now
lives once in the go-duration crate as parse_viper_duration plus checked numeric conversions.
DurationString and the typed DatadogConfiguration duration deserializer both build on these, so
the typed path accepts and rejects exactly what DurationString did before the migration. Notably
the typed path no longer silently clamps negative numeric durations to zero, which would have
disabled duration-gated features like host-tag enrichment.
Docs. A note in AGENTS.md points at the config ingest crates
(lib/agent-data-plane-config, lib/agent-data-plane-config-system).

ottl_transform_config is absent from the vendored Datadog schema, so it remains a Saluki-only key
with no witnessed alias.

Change Type

Non-functional (chore, refactoring, docs)

How did you test this PR?

Rewrote the OTTL transform tests to build from typed traces settings, and added an
ottl_transform_config round-trip rejection test mirroring the filter's.
Added duration tests covering the viper coercion rules (bare-integer nanoseconds, whitespace
trimming, negative and overflow rejection) on both go-duration and the typed deserializer.
Host-tags translation of expected_tags_duration is covered by existing translator and
config-system tests.
make build-schema-overlay leaves the tree unchanged; make fmt, make check-all, make test,
and make check-docs all pass.

References

pr-commenter · 2026-07-05T19:12:37Z

Binary Size Analysis (Agent Data Plane)

Baseline: 33d6ac6 · Comparison: 719e11d · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 41.58 MiB (baseline) vs 41.82 MiB (comparison)
Size Change: +249.66 KiB (+0.59%)

✅ Binary size difference within threshold

Changes by Module

Module	File Size	Symbols
`figment`	-572.11 KiB	355
`core`	+272.97 KiB	9253
`serde_json`	+234.31 KiB	514
`tracing`	-143.42 KiB	98
`datadog_agent_config::generated::datadog_configuration`	+120.54 KiB	52
`saluki_common::resource_tracking::groups`	+109.51 KiB	35
`serde`	-90.67 KiB	62
`agent_data_plane_config_system::saluki_env_overlay::PathRecorder`	+64.08 KiB	24
`otlp_protos::otlp_include::opentelemetry`	+59.17 KiB	100
`hyper_util`	+52.22 KiB	41
`saluki_components::sources::dogstatsd`	-42.30 KiB	358
`saluki_components::common::datadog`	-41.95 KiB	524
`prost`	-33.20 KiB	256
`agent_data_plane_config::SalukiConfiguration`	+28.75 KiB	1
`agent_data_plane_config_system::translators::datadog_translator`	+26.00 KiB	23
`serde_with`	-23.44 KiB	32
`&mut serde_json`	-22.52 KiB	94
`anon.906046189c51c143e7c52e4ee69c5016.30.llvm.13653499532151601150`	+21.68 KiB	1
`anon.72c3353df52191b74612acde54b2ebe7.240.llvm.4152876991496683762`	-21.15 KiB	1
`saluki_components::sources::otlp`	+20.46 KiB	228

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW] +59.0Ki  [NEW] +58.8Ki    saluki_components::common::datadog::io::run_endpoint_io_loop::_{{closure}}::h953e93e63a6c6d06
  [NEW] +54.6Ki  [NEW] +54.3Ki    _<datadog_agent_config::generated::datadog_configuration::_::<impl serde_core::de::Deserialize for datadog_agent_config::generated::datadog_configuration::DatadogConfiguration>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::hb8fedc8e9123324e
  [NEW] +51.9Ki  [NEW] +51.8Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h843abaf601c274c7
  [NEW] +39.3Ki  [NEW] +39.1Ki    _<saluki_components::forwarders::otlp::OtlpForwarder as saluki_core::components::forwarders::Forwarder>::run::_{{closure}}::hbd39ffd4a67d3783
  [NEW] +38.7Ki  [NEW] +38.6Ki    saluki_components::sources::otlp::metrics::translator::OtlpMetricsTranslator::translate_metrics::h9f4566c7947da3eb
  [NEW] +36.2Ki  [NEW] +36.1Ki    _<saluki_components::transforms::apm_stats::ApmStats as saluki_core::components::transforms::Transform>::run::_{{closure}}::h4e418b43a94a7896
  [NEW] +33.5Ki  [NEW] +33.3Ki    agent_data_plane::internal::env::workload::RemoteAgentWorkloadProvider::from_configuration::_{{closure}}::hb4a73336611e8fea
  [NEW] +33.1Ki  [NEW] +33.0Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h893bca651321a92e
  [NEW] +32.8Ki  [NEW] +32.6Ki    saluki_components::sources::dogstatsd::drive_stream::_{{closure}}::ha3e2b79e47e0087b
  [NEW] +31.2Ki  [NEW] +31.0Ki    _<saluki_components::transforms::aggregate::Aggregate as saluki_core::components::transforms::Transform>::run::_{{closure}}::ha5e884574e1004fa
  [NEW] +30.8Ki  [NEW] +30.7Ki    agent_data_plane::cli::dogstatsd::handle_dogstatsd_command::_{{closure}}::h6c10f433314dfb39
  [NEW] +28.8Ki  [NEW] +28.6Ki    _<agent_data_plane_config::SalukiConfiguration as core::clone::Clone>::clone::hf26eda056b9f0915
  [DEL] -28.1Ki  [DEL] -27.9Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::hde8f26a6ddd4e78f
  [DEL] -30.0Ki  [DEL] -29.9Ki    agent_data_plane::cli::dogstatsd::handle_dogstatsd_command::_{{closure}}::h0eac9e6f0e88d7cc
  [DEL] -30.5Ki  [DEL] -30.4Ki    saluki_components::sources::otlp::metrics::translator::OtlpMetricsTranslator::translate_metrics::hab9dc9f6904919be
  [DEL] -30.6Ki  [DEL] -30.4Ki    _<saluki_components::transforms::apm_stats::ApmStats as saluki_core::components::transforms::Transform>::run::_{{closure}}::h9fdc414799192cda
  [DEL] -30.6Ki  [DEL] -30.4Ki    _<saluki_components::transforms::aggregate::Aggregate as saluki_core::components::transforms::Transform>::run::_{{closure}}::hc9062a2a4246a7d4
  [DEL] -38.4Ki  [DEL] -38.2Ki    _<saluki_components::forwarders::otlp::OtlpForwarder as saluki_core::components::forwarders::Forwarder>::run::_{{closure}}::hadf9b85f315763cf
  [DEL] -41.5Ki  [DEL] -41.4Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h82b0f5413542bf87
  [DEL] -56.3Ki  [DEL] -56.2Ki    saluki_components::common::datadog::io::run_endpoint_io_loop::_{{closure}}::h49375b69dcfb6f02
  +0.4% +65.6Ki  -0.6% -73.4Ki    [33792 Others]
  +0.6%  +249Ki  +0.3%  +109Ki    TOTAL

chatgpt-codex-connector · 2026-07-05T19:19:18Z

💡 Codex Review

saluki/bin/agent-data-plane/src/cli/run.rs

Line 180 in 831aeca

    
           let config_sys = ConfigurationSystem::load(config.clone(), EnvOverlayMode::Fallback)

Use override overlay for bootstrap env precedence

When the new config stream is not in use, config is the normal bootstrap merge of file config plus DD_ env vars, so env vars should still override YAML. Passing EnvOverlayMode::Fallback here makes the typed model keep any nested YAML value and ignore the flat env key for non-aliased nested settings such as cluster_agent.url or autoscaling.failover.enabled, so DD_CLUSTER_AGENT_URL no longer overrides a cluster_agent.url set in the file for components that now read config_sys.

saluki/lib/agent-data-plane-config-system/src/system.rs

Line 224 in 831aeca

let datadog = DatadogConfiguration::deserialize(&merged)?;

Normalize numeric byte-size settings before deserializing

This deserialize still rejects wider input forms before the translator can parse them; for example, dogstatsd_log_file_max_size: 10485760 in YAML was accepted by the deleted ByteSize component config and is documented as a byte value, but it reaches merged as a JSON number while the generated Datadog field is String, causing the entire configuration load to fail. Convert numeric byte-size values such as dogstatsd_log_file_max_size to strings in normalize_datadog_input_forms before calling DatadogConfiguration::deserialize.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

pr-commenter · 2026-07-05T19:28:10Z

Regression Detector (Agent Data Plane)

Run ID: 721b8a70-1788-42c0-95e5-f8f64aa8f678
Baseline: 33d6ac63 · Comparison: 719e11d2 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment	goal	Δ mean %	links
quality_gates_rss_idle	memory	⚪ +1.12	metrics profiles logs
quality_gates_rss_dsd_low	memory	⚪ +1.10	metrics profiles logs
quality_gates_rss_dsd_medium	memory	⚪ +0.76	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory	⚪ +0.46	metrics profiles logs
quality_gates_rss_dsd_heavy	memory	⚪ +0.29	metrics profiles logs

Bounds Checks: ✅ Passed (5)

experiment	check	replicates	observed	links
quality_gates_rss_dsd_heavy	memory_usage	10/10	✅ 131 MiB ≤ 140 MiB	metrics profiles logs
quality_gates_rss_dsd_low	memory_usage	10/10	✅ 43.6 MiB ≤ 50 MiB	metrics profiles logs
quality_gates_rss_dsd_medium	memory_usage	10/10	✅ 65.7 MiB ≤ 75 MiB	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory_usage	10/10	✅ 192 MiB ≤ 200 MiB	metrics profiles logs
quality_gates_rss_idle	memory_usage	10/10	✅ 29.3 MiB ≤ 40 MiB	metrics profiles logs

Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

datadog-datadog-us1-prod · 2026-07-09T09:45:06Z

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 1 Pipeline job failed

DataDog/saluki | test-integration

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 719e11d | Docs | Datadog PR Page | Give us feedback!}

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

) Cut the OTTL transform processor over from the raw `GenericConfiguration` map to the typed `SalukiConfiguration` model, following the pattern already applied to the sibling OTTL filter processor. - `OttlTransformConfiguration::from_configuration` now takes the resolved `domains::traces::OttlTransform` settings and copies them directly, instead of deserializing an `OttlTransformConfig` out of `GenericConfiguration`. The component-local config module (its `ErrorMode` enum and `OttlTransformConfig` struct) is removed; the component reuses the model's shared `OttlErrorMode`. - The call site in `run.rs` reads `saluki.domains.traces.ottl_transform`. Since this was the last consumer of the raw config in `add_baseline_traces_pipeline_to_blueprint`, the now-unused `GenericConfiguration` parameter is dropped from that function. - The seed source `OttlTransformConfig` in `saluki_only.rs` is aligned with the filter's: its `error_mode` deserializes directly into `OttlErrorMode` (rather than a loose `Option<String>` fed through a hand-rolled parse helper) and it now rejects unknown fields, restoring the strict deserialization the original component had. `ottl_transform_config` is absent from the vendored Datadog schema, so it is a Saluki-only key on the seed track; there is no witnessed alias for it. - [x] Non-functional (chore, refactoring, docs) - Rewrote the component tests to build from typed traces settings; all OTTL transform tests pass. - Added an `ottl_transform_config` round-trip rejection test mirroring the filter's. - `make build-schema-overlay` leaves the tree unchanged; `make fmt`, `make check-all`, `make test`, and `make check-docs` all pass. - Progresses #1788 - Merges into #1987

It thrashed a bit on preserving the exact behavior that `DurationString` was providing and on viper `time.Duration` parsing correctness. Overall the changes it made seem fine to me. Migrates `HostTagsConfiguration` from the raw `GenericConfiguration` map to the typed `SalukiConfiguration` model, continuing the ADP config cutover. `expected_tags_duration` is witnessed and already modeled at `shared.tags.expected_tags_duration`, so the component now reads it directly from the typed slice instead of parsing a `DurationString` off `GenericConfiguration`. The component's hand-written zero default is dropped; the generated schema default (`0s`) is authoritative. `RemoteAgentClientConfiguration` is a shared struct still consumed by several not-yet-migrated remote-agent components, so it stays on `GenericConfiguration` for now. It is built at the call site and passed into `HostTagsConfiguration`, which removes `GenericConfiguration` from the component entirely. - [x] Non-functional (chore, refactoring, docs) `make check-all`, `make test`, and `make check-docs` all pass. Translation of `expected_tags_duration` into the typed model is already covered by existing translator and config-system tests. - Progresses #1788 - Merges into #1987

webern · 2026-07-14T13:15:20Z

Closing this branch. Like its parent, it builds on the typed-configuration system that changed substantially underneath it (redesigned environment-variable handling and validate-then-commit runtime update semantics), which invalidated this ADP-binary cutover approach. The work is preserved as a frozen snapshot on my fork at archive/pr1995-adp-bin-cutover for future reference.

webern requested a review from a team as a code owner July 5, 2026 19:06

webern changed the base branch from main to m/pr5-cutover July 5, 2026 19:08

webern changed the title ~~M/pr6 adp cutover~~ chore(config): cutover adp bin components to typed config system Jul 5, 2026

webern mentioned this pull request Jul 6, 2026

chore(config): migrate OttlTransformConfiguration to typed config #1997

Merged

1 task

Copilot AI review requested due to automatic review settings July 6, 2026 09:54

dd-octo-sts Bot removed area/components Sources, transforms, and destinations. source/dogstatsd DogStatsD source. labels Jul 6, 2026

webern mentioned this pull request Jul 9, 2026

chore(config): cutover saluki-components to the new typed configuration system #1987

Closed

1 task

Copilot AI review requested due to automatic review settings July 9, 2026 12:02

webern force-pushed the m/pr6-adp-cutover branch from 1989208 to 856aef2 Compare July 9, 2026 12:02

webern force-pushed the m/pr5-cutover branch from ff33889 to 45d671a Compare July 9, 2026 12:03

Copilot AI reviewed Jul 9, 2026

dd-octo-sts Bot removed the area/config Configuration. label Jul 9, 2026

webern force-pushed the m/pr6-adp-cutover branch from 856aef2 to e5cd64a Compare July 9, 2026 13:31

webern force-pushed the m/pr5-cutover branch from 45d671a to c9c2a29 Compare July 9, 2026 13:31

Copilot AI review requested due to automatic review settings July 9, 2026 14:27

webern force-pushed the m/pr5-cutover branch from c9c2a29 to 65e4fdd Compare July 9, 2026 14:27

webern force-pushed the m/pr6-adp-cutover branch from e5cd64a to 60fe2c9 Compare July 9, 2026 14:27

Copilot AI reviewed Jul 9, 2026

Copilot AI review requested due to automatic review settings July 9, 2026 15:41

webern force-pushed the m/pr6-adp-cutover branch from 60fe2c9 to 47a5a4f Compare July 9, 2026 15:41

webern force-pushed the m/pr5-cutover branch from 65e4fdd to 169132c Compare July 9, 2026 15:41

Copilot AI reviewed Jul 9, 2026

Copilot AI review requested due to automatic review settings July 10, 2026 10:26

webern force-pushed the m/pr6-adp-cutover branch from 47a5a4f to f638633 Compare July 10, 2026 10:26

webern force-pushed the m/pr5-cutover branch from 169132c to d0e5c8d Compare July 10, 2026 10:26

Copilot AI reviewed Jul 10, 2026

webern marked this pull request as draft July 13, 2026 09:25

webern force-pushed the m/pr5-cutover branch from d0e5c8d to 4ebacf7 Compare July 13, 2026 10:06

webern force-pushed the m/pr6-adp-cutover branch from f638633 to c892b06 Compare July 13, 2026 10:06

webern added 3 commits July 13, 2026 13:59

chore(docs): add the config system to agents md

984d3ac

This was referenced Jul 14, 2026

chore(config): support environment variables in the config system #2093

Open

chore(config): integrate and test the config system #2094

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore(config): cutover adp bin components to typed config system#1995

chore(config): cutover adp bin components to typed config system#1995
webern wants to merge 3 commits into
m/pr5-cutoverfrom
m/pr6-adp-cutover

webern commented Jul 5, 2026 •

edited

Loading

Uh oh!

pr-commenter Bot commented Jul 5, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot commented Jul 5, 2026

Uh oh!

pr-commenter Bot commented Jul 5, 2026 •

edited

Loading

Uh oh!

datadog-datadog-us1-prod Bot commented Jul 9, 2026 •

edited by datadog-official Bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

webern commented Jul 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

webern commented Jul 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Human Summary

AI Summary

Change Type

How did you test this PR?

References

Uh oh!

pr-commenter Bot commented Jul 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Binary Size Analysis (Agent Data Plane)

✅ Binary size difference within threshold

Uh oh!

chatgpt-codex-connector Bot commented Jul 5, 2026

💡 Codex Review

Uh oh!

pr-commenter Bot commented Jul 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Regression Detector (Agent Data Plane)

Optimization Goals: ✅ No significant changes detected

Uh oh!

datadog-datadog-us1-prod Bot commented Jul 9, 2026 • edited by datadog-official Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

webern commented Jul 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

webern commented Jul 5, 2026 •

edited

Loading

pr-commenter Bot commented Jul 5, 2026 •

edited

Loading

pr-commenter Bot commented Jul 5, 2026 •

edited

Loading

datadog-datadog-us1-prod Bot commented Jul 9, 2026 •

edited by datadog-official Bot

Loading