Skip to content

SBOM sidecar path misclassifies oversized images as Incomplete instead of TooLarge #877

Description

@matthyx

Description

When an image exceeds maxImageSize, the in-process SBOM path correctly classifies the failure as terminal, but the sidecar path does not. On the sidecar path the same deterministic size failure is retried maxScanRetries (3) times, is eventually pinned Incomplete rather than TooLarge, and is reported to the platform as ReasonSBOMGenerationFailed instead of ReasonImageTooLarge.

This is the same class of bug as kubescape/kubevuln#391 (fixed in kubescape/kubevuln#392), just in node-agent's sidecar transport rather than its scanner logic.

Where it breaks

The scanner server correctly detects the condition and signals it with a dedicated gRPC code:

// pkg/sbomscanner/v1/server.go:59-60
if errors.Is(err, syftutil.ErrImageTooLarge) {
    return nil, status.Error(codes.FailedPrecondition, "image size exceeds maximum allowed size")
}

But the client only translates Unavailable / Aborted into a sentinel, and passes everything else through raw:

// pkg/sbomscanner/v1/client.go:92-93
if ok && (st.Code() == codes.Unavailable || st.Code() == codes.Aborted) {
    return nil, fmt.Errorf("%w: %v", ErrScannerCrashed, err)
}

So SbomManager never recognises the condition and falls into the generic failure branch (pkg/sbommanager/v1/sbom_manager.go:512-521):

s.handleGenericFailure(sbomName)
s.reportFailure(notif, imageTag, imageID, scanfailure.ReasonSBOMGenerationFailed, scanErr)

handleGenericFailure only marks the SBOM after maxScanRetries consecutive failures, and marks it Incomplete.

codes.FailedPrecondition is currently produced in exactly one place and consumed nowhere outside tests:

pkg/sbomscanner/v1/server.go:60          <- produced
pkg/sbomscanner/v1/server_test.go:116    <- asserted
pkg/sbomscanner/v1/integration_test.go:135 <- asserted

Expected behaviour

Same as the in-process path already does (pkg/sbommanager/v1/sbom_manager.go:562-570):

if errors.Is(srcErr, syftutil.ErrImageTooLarge) {
    if wipSbomHadContent {
        // don't let a content-bearing SBOM reach the TooLarge one-way door; treat
        // it as a generic (retryable, eventually Incomplete) failure instead.
        s.handleGenericFailure(sbomName)
    } else {
        s.markSBOMStatus(sbomName, helpersv1.TooLarge, nil)
    }
    s.reportFailure(notif, imageTag, imageID, scanfailure.ReasonImageTooLarge, srcErr)
}

An oversized image should be terminal on the first attempt, marked TooLarge, and reported as ReasonImageTooLarge.

Impact

  • Wasted work. Three full pull attempts per oversized image, each aborting only once the layer stream crosses maxImageSize, repeated on every scan cycle that re-enters the retry budget.
  • Wrong user-facing reason. The platform shows "Failed to generate software inventory (SBOM) for this image" instead of "Image exceeds the maximum size limit for vulnerability scanning", so operators have no signal that raising maxImageSize is the remedy.
  • Inconsistent status between transports. The same image yields TooLarge in-process and Incomplete via the sidecar, so downstream consumers see different terminal states depending on deployment mode.

Suggested fix

  1. Map codes.FailedPrecondition to a sentinel in pkg/sbomscanner/v1/client.go — e.g. an exported ErrImageTooLarge alongside the existing ErrScannerCrashed.
  2. In sbom_manager.go, handle that sentinel next to the existing ErrScannerCrashed check (around line 504), reusing the in-process branch's logic — including the wipSbomHadContent guard, since TooLarge is a one-way door in the storage layer and must never be applied to a content-bearing SBOM.

A test asserting that a FailedPrecondition from the sidecar produces TooLarge + ReasonImageTooLarge (and does not consume the retry budget) would lock this in.

Environment

  • Repo: kubescape/node-agent @ 8866b6c (main)
  • Affects the sidecar SBOM-scanner path only; the in-process path is correct.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions