Skip to content

test(orb): cover the review_audit→export→ingest→analytics fleet path end-to-end - #1270

Merged
JSONbored merged 3 commits into
mainfrom
test/orb-fleet-e2e
Jun 24, 2026
Merged

test(orb): cover the review_audit→export→ingest→analytics fleet path end-to-end#1270
JSONbored merged 3 commits into
mainfrom
test/orb-fleet-e2e

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

The three Orb fleet stages are each unit-tested in isolation today — selfhost-orb-collector.test.ts (export), orb-ingest.test.ts (ingest), orb-analytics.test.ts (analytics) — but nothing asserts they compose on a consistent target_id / repo_hash / pr_hash key. That seam is exactly where the FLEET_QUERY source='gittensory-native' filter or the gate_decisionpr_outcome join can silently break before anything fleet-derived (e.g. auto-tune) is built on it.

This adds one integration test that drives the whole pipeline on a shared key:

  1. seed a self-host instance's review_audit ledger (native gate_decision + realized pr_outcome + an optional reversal),
  2. run exportOrbBatch with a capturing fetch mock,
  3. feed the captured payload into handleOrbIngest,
  4. assert computeFleetAnalytics: 5 clean merges → a 100%-precision fleet of one, and a reverted merge propagates reversal_flag end-to-end into the fleet's reversal + false-positive rates.

The env is set so the export gate is satisfied for both the current (ORB_ENABLED) and the hardwired (GITHUB_APP_PRIVATE_KEY, #1257) export paths, and ORB_ANONYMIZE=false keeps repo_hash/pr_hash readable for assertions.

Scope

  • Test-only — no src/** change, so no codecov/patch exposure
  • In wantedPaths (test/), narrow (1 file), no secrets

Validation

  • npm run typecheck clean; vitest run test/integration/orb-fleet-e2e.test.ts — 2 passed
  • npm run test:ci green locally

Safety

  • No production/behavior change; no auth/CORS/boundary impact

Advances #1255.

@dosubot dosubot Bot added the size:XS label Jun 24, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.05%. Comparing base (7610d8a) to head (423e0ed).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1270   +/-   ##
=======================================
  Coverage   95.05%   95.05%           
=======================================
  Files         177      177           
  Lines       19981    19981           
  Branches     7189     7189           
=======================================
  Hits        18993    18993           
  Misses        395      395           
  Partials      593      593           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…end-to-end

The three Orb fleet stages are each unit-tested in isolation (selfhost-orb-collector,
orb-ingest, orb-analytics), but nothing asserted they COMPOSE on a consistent
target_id/repo_hash/pr_hash key — the seam where the FLEET_QUERY source filter or the
gate_decision↔pr_outcome join can silently break.

Add an integration test that seeds a self-host instance's review_audit ledger, runs
exportOrbBatch with a capturing fetch mock, feeds the captured payload into
handleOrbIngest, then asserts computeFleetAnalytics: 5 clean merges → a 100%-precision
fleet of one, and a reverted merge propagates reversal_flag end-to-end into the fleet's
reversal + false-positive rates. Test-only; no production change.

Advances #1255.
@JSONbored
JSONbored force-pushed the test/orb-fleet-e2e branch from f58e5bb to d680c09 Compare June 24, 2026 22:40
@JSONbored
JSONbored force-pushed the test/orb-fleet-e2e branch from d680c09 to f1a9266 Compare June 24, 2026 22:41
@JSONbored
JSONbored merged commit bb4d7d8 into main Jun 24, 2026
18 checks passed
@JSONbored
JSONbored deleted the test/orb-fleet-e2e branch June 24, 2026 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant