Move V3 protocol into a separate crate#1920
Draft
StephenWakely wants to merge 13 commits into
Draft
Conversation
|
dd-octo-sts
Bot
added
area/components
Binary Size Analysis (Agent Data Plane)Baseline: 7b98c6f · Comparison: 2f8686b · diff ✅ Binary size difference within thresholdChanges by Module
Detailed Symbol Changes |
Regression Detector (Agent Data Plane)Run ID: Optimization Goals: ✅ No significant changes detectedFine details of change detection per experiment (5)Experiments configured
Bounds Checks: ❌ Failed (5)
ExplanationA change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression ( |
StephenWakely
force-pushed
the
stephen/v3_vector
branch
from
e04ed5f to
4a56ec9
Compare
dd-octo-sts
Bot
added
area/core
StephenWakely
changed the base branch from
main
to
tobz/datadog-metrics-v3-payload-support
dd-octo-sts
Bot
removed
area/core
tobz
force-pushed
the
tobz/datadog-metrics-v3-payload-support
branch
from
6b21d5a to
b86c488
Compare
…ics-v3 crate Extract the pure-protocol V3 codec (interner, types, writer, serializer) from lib/saluki-components into a zero-dependency lib/datadog-metrics-v3 crate. The serializer is reimplemented using raw protobuf wire-format encoding with no external deps. saluki-components now re-exports from this crate. fix(metrics): fix V3 codec correctness issues found in code review Three issues addressed: 1. compact_values() skipped for Sketch metrics. Sketch summary values (sum/min/max) must stay in vals_float64. The old code could move them to vals_float32 or vals_sint64 based on magnitude, corrupting the payload — the decoder reads these from vals_float64 by position, not by type tag. 2. V3MetricBuilder Drop guard. Without a Drop impl, forgetting close() silently left value-type bits unset in the types column. Now panics in debug builds if close() is not called before drop. 3. MetricsV3EndpointEncoder::encode() now writes to buffer. The EndpointEncoder contract requires each encode() call to write to buffer so the request builder can track sizes. Previously the V3 encoder accumulated into self.writer but wrote nothing to buffer, producing empty payloads. Fixed by serializing a single-metric V3 payload per encode() call. Full cross-metric deduplication requires a batch-flush hook on EndpointEncoder (tracked via TODO comment). chore(metrics): remove unused WIRE_VARINT constant and redundant field_numbers module field_numbers was a duplicate of the private mod field in serializer.rs and was not referenced anywhere. WIRE_VARINT (protobuf wire type 0) is not used by the current serializer.
…ics-v3 crate After rebasing stephen/v3_vector on tobz/datadog-metrics-v3-payload-support (PR #1175), extract the codec internals into lib/datadog-metrics-v3 so Vector can reuse them without depending on saluki-core or saluki-common. Extracted into standalone crate (zero external deps): - Interner: Borrow-trait based get_or_insert — no clone on cache hits - Types: V3MetricType, V3ValueType, PointKind, value_type_for_values (the precision-safe compaction algorithm from PR #1175) - Writer: unit support (FLAG_HAS_UNIT / dict_unit_bytes / unit_refs), FLAG_NO_INDEX, set_unit(), set_no_index() - Serializer: fields 25 (dictUnitStr) and 26 (unitRefs) Removed from saluki-components (now re-exported from standalone crate): - v3/interner.rs → deleted; use datadog_metrics_v3::Interner - v3/types.rs → deleted; use datadog_metrics_v3::{V3MetricType, V3ValueType, ...} - v3/writer.rs → kept (saluki-specific output types + protobuf serialization) but now imports Interner, V3MetricType, V3ValueType from the standalone crate The saluki-components v3/writer.rs retains its own value_type_for_values (identical algorithm) and protobuf::CodedOutputStream serialization since those are appropriate for saluki. The standalone crate uses a zero-dep hand-rolled serializer for Vector compatibility.
…dOutputStream Replace the zero-dep hand-rolled protobuf serializer with protobuf::CodedOutputStream. This eliminates the duplicate serialization implementation — the crate now uses the same serialization path as saluki-components' v3/writer.rs. API change: V3Writer::close() now returns Vec<u8> (serialized bytes) instead of V3EncodedData. Callers no longer need a separate serialize_v3_payload() call. finalize_inner() is kept as pub(crate) for testing internal columnar state. serializer.rs is deleted; the field constants move to writer.rs.
StephenWakely
force-pushed
the
stephen/v3_vector
branch
from
a802d8c to
990f584
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Change Type
How did you test this PR?
References