Repository navigation
fix: protobuf parser resyncs past leading garbage instead of stalling - #169
Conversation
The parser previously bailed after 3 failed parse attempts via a MaxRetryAttempts cap. Once that limit hit, the loop exited without advancing past unrecognizable bytes at the head of the buffer. Each subsequent read appended more bytes onto a buffer the parser refused to drain, so QueuedMessageCount climbed indefinitely and downstream consumers stopped receiving messages. This bit the desktop streaming flow on serial connect: the DTR pulse on Microchip USB-to-Serial chips emits boot-time garbage that lands at the head of the consumer buffer. The first valid streaming frame after that never gets parsed, so Live Graph shows no data. The fix removes the retry cap and resyncs by advancing one byte per failed attempt until the buffer drains or a valid frame is found. Added a regression test that prepends 20 junk bytes to a valid frame and asserts the parser recovers the frame and consumes the full input. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Review Summary by QodoRemove retry cap and implement single-byte resync in protobuf parser
WalkthroughsDescription• Removes MaxRetryAttempts cap that caused parser to stall on garbage bytes • Implements single-byte resync on every parse failure to drain buffer • Adds regression test for leading garbage recovery scenario • Fixes desktop Live Graph receiving zero messages after stream start Diagramflowchart LR
A["Malformed/Invalid Bytes"] -->|Old: 3 retries then exit| B["Buffer Stalls"]
A -->|New: Advance 1 byte| C["Continue Parsing"]
C -->|Valid Frame Found| D["Message Recovered"]
C -->|Buffer Empty| E["Wait for More Data"]
File Changes1. src/Daqifi.Core/Communication/Consumers/ProtobufMessageParser.cs
|
Code Review by Qodo
1.
|
Use ParseFrom(ReadOnlySpan<byte>) (Google.Protobuf 3.18+) instead of Slice(...).ToArray() so the byte-by-byte resync over a garbage buffer no longer allocates a candidate payload on each attempt. Addresses Qodo's performance concern on PR #169. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Re: Qodo "Resync allocates per attempt" — addressed in d9aa65b Fixed the allocation by switching to Not addressing the second part of Qodo's note (exception-as-control-flow) in this PR — short of reimplementing protobuf parsing, the only way to know whether a candidate length-prefixed slice is valid is to attempt to parse it. The garbage-recovery path is already a cold/exceptional case (real streaming traffic parses the first try and never hits the catch); the allocation fix removes the dominant cost. |
Summary
MaxRetryAttempts = 3cap inProtobufMessageParser. After 3 failed parse attempts the loop exited without advancing past unrecognizable bytes — the consumer buffer stayed clogged forever and downstreamMessageReceivedevents stopped firing.ProtobufMessageParser_ParseMessages_RecoversFromLeadingGarbagethat prepends 20 zero bytes to a validWriteDelimitedToframe and asserts the parser recoversMsgTimeStamp = 42and consumes the full input.Why this matters
This regression manifests as the desktop Live Graph receiving zero stream messages after
StartStreaming. The DTR pulse on Microchip USB-to-Serial chips emits a few boot-time bytes when the port opens; those land at the head of the consumer buffer. Pre-fix,QueuedMessageCount(which is byte count, not message count) climbed from 66 → 8202 in 2 seconds while no frames were dispatched. Post-fix, the parser advances past the boot garbage and the first valid streaming frame is recovered.Test plan
dotnet test --filter "FullyQualifiedName~ProtobufMessageParser" -f net9.0— 8/8 pass (new test green)dotnet test --filter "FullyQualifiedName~Consumers" -f net9.0— 41/41 pass (no regressions)/dev/cu.usbmodem101after the new Core release lands🤖 Generated with Claude Code