fix: validate rootfs diff-ids before use to prevent SBOM panic on digest-less images - #854
Conversation
|
Warning Review limit reached
Next review available in: 33 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthrough
ChangesImage diff-ID validation
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…est-less images Node-agent crashed with `panic: no ':' separator in digest ""` when the SBOM manager processed images without a resolvable digest (e.g. images referenced only by tag, such as `:latest` on OpenShift/CRI-O). The empty diff-id string reached go-digest's Digest.Algorithm()/Hex(), which panic on malformed input instead of returning an error. Add validateDiffIDs, using the digest library's own non-panicking Validate() method, and call it in NewSource before any diff-id is touched. Invalid images now surface as a reported SBOM-generation failure through the existing error path instead of crashing the process. Fixes #853 Docs-exempt: pure bug fix, no existing doc describes diff-id validation or SBOM source construction behavior Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com>
02238bc to
1b15b32
Compare
…top reprocessing loop SBOM-generation failures (sidecar scan error, invalid image source, syft cataloging error) reported via reportFailure but never marked the reserved SBOM object with a terminal status. Since the reprocessing switch only special-cases TooLarge and Learning, a permanently-failing image fell through to the "processing was interrupted, retrying" default case and was silently reprocessed on every subsequent container start for that image, producing repeated identical error logs and backend failure reports forever. Add markSBOMStatus, generalizing the existing TooLarge-marking pattern, and use it to persist an Incomplete status on all three generic-failure call sites. Add a matching Incomplete case to the reprocessing switch, version-gated exactly like the existing Learning case, so a later node-agent build still retries images that previously failed. This is a pre-existing defect independent of #853/#854 (it already affected ordinary syft SBOM-generation failures, unrelated to digests); splitting it into its own change keeps each fix minimal and reviewable. Docs-exempt: pure bug fix, no existing doc describes SBOM reprocessing or terminal-status behavior Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com>
Performance Benchmark ResultsNode-Agent Resource Usage
Dedup EffectivenessNo data available. |
Review: no blockersVerified locally on
Scope is tight and the Non-blocking notes
|
Performance Benchmark ResultsNode-Agent Resource Usage
Dedup EffectivenessNo data available. |
…top reprocessing loop (#855) * fix: persist terminal status on generic SBOM-generation failures to stop reprocessing loop SBOM-generation failures (sidecar scan error, invalid image source, syft cataloging error) reported via reportFailure but never marked the reserved SBOM object with a terminal status. Since the reprocessing switch only special-cases TooLarge and Learning, a permanently-failing image fell through to the "processing was interrupted, retrying" default case and was silently reprocessed on every subsequent container start for that image, producing repeated identical error logs and backend failure reports forever. Add markSBOMStatus, generalizing the existing TooLarge-marking pattern, and use it to persist an Incomplete status on all three generic-failure call sites. Add a matching Incomplete case to the reprocessing switch, version-gated exactly like the existing Learning case, so a later node-agent build still retries images that previously failed. This is a pre-existing defect independent of #853/#854 (it already affected ordinary syft SBOM-generation failures, unrelated to digests); splitting it into its own change keeps each fix minimal and reviewable. Docs-exempt: pure bug fix, no existing doc describes SBOM reprocessing or terminal-status behavior Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com> * fix: never overwrite existing SBOM content on a failed reprocess Addresses review feedback: markSBOMStatus persisted a wipSbom fetched via GetSBOMMeta, which the storage layer returns without its Spec (metadata- only fetch). Reprocessing a previously-successful, content-bearing SBOM (e.g. after a node-agent version bump) that then fails would silently overwrite its real content with an empty Spec and pin it to a terminal status -- permanently losing vulnerability-scan coverage for that image. Track whether the SBOM being reprocessed had prior successful content (wipSbomHadContent, set only in the Learning-case version-mismatch branch) and skip the destructive persist whenever it did, across every path that can reach it: the generic-failure branches (handleGenericFailure, now with a bounded failureRetries counter so a single transient error doesn't permanently pin an image either), the scanner-crash branch (handleScannerCrash), and the ErrImageTooLarge branch, whose totalSize is computed from the currently-mounted layer paths rather than being a fixed property of the image and so is equally reachable while reprocessing. Also extracts shouldRetryAtCurrentVersion, shared by the Learning and Incomplete switch cases, removing the near-duplicate version-gating logic and the comment that had drifted between them. Docs-exempt: pure bug fix, no existing doc describes SBOM reprocessing or terminal-status behavior Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com> * fix: mark SBOM terminal status via annotation-only patch, not full replace Addresses non-blocking review feedback: the previous fix's hadContent guard meant a previously-successful SBOM that started failing permanently (not just transiently) was reprocessed on every container start forever, since nothing was ever persisted to stop the loop for that class of image -- the retry-bounding only applied to images that never had content. Add storage.SbomClient.PatchSBOMAnnotations, a JSON merge patch on metadata.annotations only, which never sends spec regardless of what the caller does or doesn't know about the object's content. Route all SBOM status marking (Incomplete, TooLarge) through it instead of a full ReplaceSBOM, so it's always safe to persist a terminal status -- the hadContent tracking, and every guard built on it across handleGenericFailure, handleScannerCrash and the ErrImageTooLarge branch, is removed entirely. Retries are now bounded uniformly for all images via the same failureRetries counter, switched from a plain map to a bounded+TTL'd expirable.LRU so short-lived images don't leak entries either (the second non-blocking item). markSBOMStatus now also records the current tool version alongside the status, since the version check that gates future reprocessing depends on it and the object is no longer implicitly carrying an in-memory version bump the way the old ReplaceSBOM-based code did. Docs-exempt: pure bug fix, no existing doc describes SBOM reprocessing, terminal-status, or storage patch behavior Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com> * fix: never mark a content-bearing SBOM TooLarge, a storage-layer one-way door Addresses review feedback on d95955c: TooLarge is special-cased in the storage layer's GuaranteedUpdate, which silently drops every future write (patch or replace) to an object once its status annotation is TooLarge. Every other TooLarge writer in this codebase explicitly clears Spec first because of this, but PatchSBOMAnnotations never touches Spec at all -- so patching a content-bearing SBOM to TooLarge left its real Spec permanently frozen in storage: unfixable by any future reprocess, and defeating the point of TooLarge in the first place (avoiding a bloated stored object). Reintroduce wipSbomHadContent, scoped narrowly to the two TooLarge write sites (the ErrImageTooLarge branch and handleScannerCrash's post-maxScanRetries marking): a content-bearing SBOM now falls back to the retryable Incomplete path instead, which has no such short-circuit. Incomplete continues to flow entirely through the annotation-only patch introduced in d95955c, unaffected. Docs-exempt: pure bug fix, no existing doc describes SBOM status transitions or storage patch semantics Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com> --------- Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Overview
Node-agent crashed with
panic: no ':' separator in digest ""when the SBOM manager processed a container image without a resolvable digest (e.g. an image referenced only by tag, such as:latest, which CRI-O/OpenShift can report with an empty layer diff-id). The empty digest string reachedgo-digest'sDigest.Algorithm()/Digest.Hex(), which panic on malformed input instead of returning an error, taking down the whole node-agent process.This adds a
validateDiffIDshelper (using the digest library's own non-panickingValidate()method) and calls it inNewSourcebefore any diff-id is touched. Images with an invalid/empty diff-id now surface as a reported SBOM-generation failure through the existingreportFailureerror path instead of crashing the process — the agent keeps running.Additional Information
This change is deliberately scoped to
pkg/sbommanager/v1/syftutil/source.go+ its test file only — no changes tosbom_manager.go. During root-cause investigation a separate, pre-existing defect was found (SBOM-generation-failure objects aren't marked terminal and get silently reprocessed/re-logged on every container restart); that is out of scope here and is addressed in a follow-up PR.How to Test
Test_NewSource_InvalidDiffIDreproduces the exact crash input (diff_ids: [""]) and asserts no panic + a wrapped error.Related issues/PRs
Checklist before requesting a review
Summary by CodeRabbit