Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ internal/selector/ Generic +/- selector resolver (Resolve, Catalog
internal/progress/ Live spinner + buffered completed-step renderer (Progress, Child)
internal/detectors/ Detector contracts (Detector, DetectorDescriptor, ResolveGraphRequest)
internal/detectors/* Concrete per-ecosystem detectors (gomod, gradle, maven, npm,
pnpm, yarn, python, ruby, composer, githubactions, sbom, syft)
pnpm, yarn, bun, python, ruby, composer, githubactions, sbom, syft)
internal/engine/ Pipeline core (pipeline.go, engine.go), Registry wrapper, scope,
graph-container helpers, explain orchestration, diff orchestration
internal/engine/consolidation/ Cross-subproject graph consolidation, manifest dedup, enrichment
Expand Down
16 changes: 15 additions & 1 deletion dev-docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,12 @@ Reachability data lives on `sdk.Vulnerability.Reachability` rather than on `Find

Pipeline plumbing: `engine.PipelineResult` exposes `Graph`, `Registry`, `Findings`, and `RiskScores`. The registry is built right after consolidation (`consolidation.BuildPackageRegistry`) and threaded through match/analyze/audit requests; output helpers (`BuildScanResponse`, `WriteSARIF`, `FindingsFromScan`, `PackagesFromGraph`) all accept `*sdk.PackageRegistry` and re-enrich their projections by resolving `PackageRef` and `VulnerabilityID`. See [`MODELS.md`](MODELS.md) for the full schema reference.

### Decision: registry matching eligibility is an occurrence-level engine boundary

Detection keeps every dependency occurrence and every PURL-backed package artifact, including application roots, workspace members, local sources, and unknown relationships. Immediately before matcher selection and execution, `engine.registryMatchRequest` clones only occurrences for which `Dependency.RegistryMatchEligible()` is true and preserves edges whose endpoints are both eligible. Every built-in and external matcher therefore receives the same filtered graph, while the full `PackageRegistry` remains shared so enrichment is still deduplicated by PURL. Analysis and auditors continue with the complete original graph.

Published registry releases are eligible, including releases downloaded through custom registries or mirrors. Application and manifest nodes are always ineligible. Project, workspace/link, file, Git, and arbitrary URL occurrences are ineligible. An omitted source and unknown plugin-defined source values remain eligible for protocol-v1 compatibility. Relationship `unknown` does not affect eligibility: a registry package whose parent could not be recovered is still enriched normally. Targeted matching of an ineligible occurrence short-circuits instead of widening to unrelated eligible packages. When eligible and ineligible occurrences share an exact PURL, vulnerability findings reference only the eligible occurrences; if a PURL has no eligible occurrence, detector- or SBOM-supplied vulnerability data remains auditable on its local occurrences.

### Decision: unresolved dependency parents use an explicit unknown relationship

Lockfiles can contain a package component whose parent chain cannot be
Expand Down Expand Up @@ -152,6 +158,12 @@ A **subproject** is an independently discovered nested directory (its own discov

Workspace/reactor detectors (npm and pnpm lockfile, cargo, maven) emit one `GraphEntry{Graph, ManifestMetadata}` per module using the pre-existing multi-entry `sdk.GraphContainer` — no SDK type changes. Each module entry carries the module's application root plus its reachable subtree (`detectors.SubgraphFrom`), with paths subproject-relative so consolidation's existing rebase/dedup layer stays a pure select/dedup/rebase stage. Shared transitives appear in multiple entries by design; the merged graph and PURL registry deduplicate them, and report-level counts (text manifest count, markdown/MCP package totals) deduplicate by PURL/ID rather than summing per-manifest lengths. Module directories come from the best per-ecosystem source: npm packages-map member keys, pnpm importer keys, `cargo metadata` member `manifest_path` (or `[workspace] members` globs + member `Cargo.toml`s on the lock path — which also fixed virtual workspace roots erroring), and a recursive pom `<modules>` walk for maven (TGF output carries no paths; unmatched graph roots fall back into the root entry, and any walk failure degrades to today's single merged manifest). **Deferred, degrade to one merged root manifest**: gradle and sbt (no machine-readable per-module graph in one invocation), mix and pub (low value/tool limits), yarn classic (v1 lockfiles carry no member info; berry is a follow-up), and the node *native* detectors (`npm ls` is root-scoped; per-member subprocess fan-out multiplies runtime while the lockfile detectors are the chain primaries anyway).

### Decision: Bun text lockfiles are native; binary lockfiles degrade explicitly

`bun-detector` parses JSONC `bun.lock` versions 0 and 1 directly in Go. The parser removes comments and trailing commas with a string-aware state machine, inventories package tuples before constructing edges, models workspace roots as application nodes, and runs the shared Node relationship finalizer before per-workspace graph partitioning. Bun workspace entries therefore use the same multi-entry `sdk.GraphContainer` contract described above. The lockfile path never invokes or installs Bun, so committed text lockfiles remain deterministic and offline.

The legacy `bun.lockb` binary representation is not parsed in core. The detector chain next invokes `bun-native-detector` when Bun and a package manifest are available. It runs `bun pm ls --all`, preserves displayed nested edges, resolves workspace paths to application identities, normalizes direct npm aliases, and reconciles top-level installed occurrences with `package.json`. Direct edges are created only when one installed occurrence proves the declaration. Duplicate-name occurrences and hoisted packages without a provable parent are attached beneath the application root with `unknown` relationships; they remain eligible for every later pipeline stage. If Bun is unavailable or the installed inventory is empty, Syft remains the final fallback. Users who need full lockfile graph fidelity can migrate with `bun install --save-text-lockfile --frozen-lockfile --lockfile-only`. Install-first is supported only when explicitly requested and runs `bun install`; ordinary detection does not install packages.

### Decision: Package locations are detector-relative today

`PackageLocation.Position.File` is emitted by detectors in the coordinate space of the detector working directory. For single-root projects that is already repository-relative, which lets `bomly diff` compare SARIF locations with repo-relative changed-line ranges.
Expand Down Expand Up @@ -209,7 +221,9 @@ Some native detector chains intentionally prefer a build-tool command over a com

### Decision: dependency graph benchmarking is hidden and local-only

`bomly benchmark` is a hidden maintainer command backed by `internal/benchmark`. It scans public GitHub repositories with native detectors, compares the filtered dependency graph against GitHub Dependency Graph and external Syft SBOMs, and writes deterministic artifacts under `.benchmark-runs/latest`. Bomly scan and SBOM diff execution run in-process through the engine and output model; only the external `git` and `syft` tools remain subprocesses. The in-process adapter builds a native-only registry directly so local configuration and managed-plugin discovery cannot distort benchmark results. Package and relationship scores are comparative engineering signals, not pass/fail gates and not claims that a baseline is ground truth. The benchmark is intentionally local-only so exploratory scoring does not become a release or merge gate before it is calibrated.
`bomly benchmark` is a hidden maintainer command backed by `internal/benchmark`. It scans public GitHub repositories with native detectors, compares the filtered dependency graph against GitHub Dependency Graph and external Syft SBOMs, and writes deterministic artifacts under `.benchmark-runs/latest`. Bomly scan and SBOM diff execution run in-process through the engine and output model; only the external `git` and `syft` tools remain subprocesses. The in-process adapter builds a native-only registry directly so local configuration and managed-plugin discovery cannot distort benchmark results.

The benchmark reports two distinct signals. Raw agreement is the symmetric overlap with every source. Correctness is computed only for evidence sources and excludes reviewable graph extensions: project/non-registry occurrences classified by the native graph and exact target-manifest edges with mandatory evidence text. Observational sources such as Syft remain visible without being promoted to ground truth. `mismatches.json` retains every source-only, Bomly-only, version-mismatched, and adjudicated item, so an extension can never disappear behind the score. Unadjudicated extra data remains a correctness failure. Package and relationship scores are engineering signals, not claims that a baseline is ground truth. The benchmark is intentionally local-only so exploratory scoring does not become a release or merge gate before it is calibrated.

### Decision: Python graph resolution is lockfile-first, validated, and provenance-backed

Expand Down
3 changes: 2 additions & 1 deletion dev-docs/CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ The URL-backed scan cases are defined in the embedded `internal/benchmark/testda
- `args`: additional Bomly scan arguments
- `tools`: package managers required for the case
- `benchmark_enabled`: whether the same target can participate in the local benchmark
- `adjudicated_relationships`: exact PURL edges independently verified in the pinned repository but omitted by a comparison source; every entry requires a reason

Smoke tests use the pinned `ref`. The local benchmark intentionally does not, because GitHub's SBOM API exports the repository's current default-branch state.

Expand Down Expand Up @@ -134,7 +135,7 @@ make benchmark ARGS="--repo https://github.com/owner/repo --ecosystem npm"

The command reads the embedded scan-target manifest, clones each selected repository default branch, captures its HEAD SHA, and writes artifacts under `.benchmark-runs/latest`. Custom repositories must be public `https://github.com/<owner>/<repo>` URLs and use the default branch.

Each completed comparison preserves raw and ecosystem-filtered SBOMs, a detailed `bomly diff --sbom` artifact, and `benchmark-summary.json` files at the source, case, and run levels. Scores cover package agreement, comparable dependency edges, and their mean. Packages without PURLs are reported separately and excluded from scoring. Scores are informational: GitHub SBOM `404` responses and missing Syft executables are recorded as unavailable so other comparisons can continue.
Each completed comparison preserves raw and ecosystem-filtered SBOMs, a detailed `bomly diff --sbom` artifact, an exact `mismatches.json` classification, and `benchmark-summary.json` files at the source, case, and run levels. Evidence sources contribute to the headline correctness score; observational sources such as Syft contribute raw agreement without being treated as ground truth. Multiple serialization formats from one source family remain visible as separate rows and artifacts but receive one aggregate weight. Correctness excludes only graph extensions backed by explicit evidence: non-registry occurrences identified by Bomly's graph model and target-manifest relationships with mandatory reasons. Raw symmetric agreement remains visible alongside correctness, and every excluded package or edge remains listed in the mismatch artifact. Unadjudicated Bomly-only data and all source-only data continue to reduce evidence-source correctness. Packages without PURLs are reported separately and excluded from scoring. Scores are informational: GitHub SBOM `404` responses and missing Syft executables are recorded as unavailable so other comparisons can continue.

GitHub SBOM requests can be unauthenticated, but local unauthenticated runs quickly hit GitHub's low public API rate limit. The benchmark checks token environment variables in this order:

Expand Down
3 changes: 3 additions & 0 deletions dev-docs/MODELS.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ Key helpers:
- `sdk.NormalizeDependencyIdentity(dep)` — canonical identity for diff matching.
- `sdk.CanonicalPackageURLFromDependency(dep)` — derive the canonical PURL when the detector didn't supply one.
- `sdk.RelationshipForPath(path)` — preserve an explicit relationship or derive direct/transitive from a root-to-target path.
- `dep.RegistryMatchEligible()` — classify whether this occurrence may be sent to external registry enrichment.

An `unknown` relationship means that the package was present in the owning
manifest but its parent could not be recovered. The component root is attached
Expand All @@ -108,6 +109,8 @@ for protocol-v1 plugins and is derived from graph structure by consumers.

Dependencies **do not** carry `Licenses`, `Vulnerabilities`, or `Scorecard` fields. Detection-time licenses ride along in metadata; matching-stage data lives on the registry package.

Registry matching eligibility is occurrence-based. Ordinary registry releases are eligible even when their `ResolvedURL` points at a custom registry or mirror. Application/manifest nodes and occurrences sourced from project, workspace, link/file, Git, or arbitrary URL references are ineligible but remain in the complete graph and package registry for analysis, auditing, diff, SBOM, and output. An omitted source remains eligible for protocol-v1 and legacy detector compatibility. Before any built-in or external matcher runs, the engine passes it a cloned graph containing only eligible occurrences and eligible-to-eligible edges; the original graph and full registry continue to later stages unchanged.

## `sdk.Package` — registry artifact (matching)

```go
Expand Down
2 changes: 2 additions & 0 deletions dev-docs/prompts/bomly-benchmark-report.prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Inspect all available artifacts:
- `.benchmark-runs/latest/cases/*/sources/bomly/*.sbom.json`
- `.benchmark-runs/latest/cases/*/sources/*/benchmark-summary.json`
- `.benchmark-runs/latest/cases/*/sources/*/diff.json`
- `.benchmark-runs/latest/cases/*/sources/*/mismatches.json`
- `.benchmark-runs/latest/cases/*/sources/*/source.sbom.json`

The root summary and per-case summaries are the primary source for status and scores. Per-source summaries contain package, relationship, scope, detector-provenance, and artifact details.
Expand All @@ -30,6 +31,7 @@ Definitions:
## Evidence Model

- Use `used_detectors` to identify the native Bomly detector path. Flag `syft-detector` provenance as a benchmark setup problem.
- Report correctness and raw agreement separately. Inspect every adjudicated extension reason and flag any unadjudicated Bomly-only or source-only mismatch.
- Use `diff.json` for concrete package presence and version differences.
- Use filtered `source.sbom.json` and Bomly `*.sbom.json` artifacts for dependency-edge and scope evidence.
- Treat missing baseline scope metadata as unknown, not automatically as a Bomly issue.
Expand Down
4 changes: 2 additions & 2 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ flowchart TD

1. **Discover** — Bomly inspects the target root and finds every supported package-manager root (a `go.mod`, a `package-lock.json`, a `pom.xml`, and so on). With `--recursive` it also walks nested directories, discovering independent subprojects in a monorepo while workspace-aware managers (npm workspaces, Maven reactors, …) keep expanding their own modules from the root. See [Scan targets](SCAN_TARGETS.md#recursive-discovery----recursive).
2. **Detect** — For each root, a [detector](DETECTORS.md) reads the lockfile, manifest, or SBOM and resolves a dependency graph. Per-subproject graphs are then *consolidated* into one graph and one deduplicated package set for the rest of the run. `--scope` narrows the graph to runtime or development dependencies here.
3. **Match** — When you pass `--enrich`, [matchers](MATCHERS.md) add data to each package: known vulnerabilities, licenses, end-of-life status, and project health scores.
3. **Match** — When you pass `--enrich`, [matchers](MATCHERS.md) add data to published registry packages: known vulnerabilities, licenses, end-of-life status, and project health scores. Project roots, workspace members, and local/file/Git/URL artifacts remain in the graph and reports but are not queried as if they were registry releases.
4. **Analyze** — When you pass `--analyze`, [reachability](REACHABILITY.md) analysis runs on top of the matched data to flag whether a vulnerability is actually reachable from your code.
5. **Audit** — When you pass `--audit`, [auditors](AUDITORS.md) evaluate policy (severity thresholds, license rules, denied packages) against the enriched data and produce findings. Combine `--enrich --audit` to gate on fresh external data in one run.
6. **Render** — Bomly emits the result as text, JSON, SARIF, or an SBOM. See [Output formats](OUTPUT_FORMATS.md) and [SBOM formats](SBOM.md).
Expand Down Expand Up @@ -105,7 +105,7 @@ External plugins run as sandboxed, versioned binaries and are disabled until you
**Bomly is offline-safe by default.** A plain `bomly scan` reads files on disk and makes no network calls of its own.

- **Matchers** only run when you pass `--enrich`. `--audit` evaluates data that is already present and never triggers enrichment on its own.
- **Detectors** vary: lockfile parsers (npm, pnpm, Yarn, Composer, Bundler, NuGet, GitHub Actions, SBOM ingest, …) are pure file readers and make no network calls. Build-tool–backed detectors (Go, Maven, Gradle, SBT) shell out to the build tool, which may fetch packages as part of its own normal resolution — that is the build tool's behavior, not Bomly's. Hybrid detectors prefer the lockfile and pass offline flags to any fallback.
- **Detectors** vary: lockfile parsers (npm, pnpm, Yarn, Bun text lockfiles, Composer, Bundler, NuGet, GitHub Actions, SBOM ingest, …) are pure file readers and make no network calls. Build-tool–backed detectors shell out to the package manager when their deterministic file parser cannot resolve the project. Bun prefers `bun.lock`, then uses `bun pm ls --all` for the installed tree, and finally falls back to Syft; displayed child edges are preserved and unprovable hoisted parent relationships remain explicitly `unknown`. Install-first is never implicit.
- `--install-first` is the explicit opt-in that lets supporting detectors run their install command (`npm install`, `pip install`, …) before resolving; this downloads packages by design.

When enrichment is enabled, the **only** services Bomly's built-in matchers contact are OSV, CISA KEV, deps.dev, ClearlyDefined, endoflife.date, and OpenSSF Scorecard. No telemetry, no credentials sent. External plugin matchers may contact their own documented services once you install and enable them. See [Detectors → Network behavior](DETECTORS.md#network-behavior) and [Matchers](MATCHERS.md).
Expand Down
Loading