Skip to content

feat(orb): registration-gate the fleet + cap ingest body (das-github-mirror model) - #1274

Merged
JSONbored merged 1 commit into
mainfrom
feat/orb-ingest-registration
Jun 24, 2026
Merged

feat(orb): registration-gate the fleet + cap ingest body (das-github-mirror model)#1274
JSONbored merged 1 commit into
mainfrom
feat/orb-ingest-registration

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Answers "how should /v1/orb/ingest be protected?" by following das-github-mirror's ingress model instead of #1248's shared ORB_INGEST_SECRET — which would break #1257's hardwired-on, no-shared-key telemetry (every self-hoster would 401 without the secret). das-github-mirror keeps ingress open + verified-by-context and gates trust on a registered=false-by-default flag + dedup + a body limit. Orb's many-instances→one-collector topology has no per-instance key the collector could verify, so the faithful translation is:

  • Open ingest, bounded. readOrbIngestBody reads behind a 1 MiB ceiling (streaming, with a content-length fast-path) → 413 on oversize. Mirrors the body limit das-github-mirror puts in front of its open webhook ingress. Dedup already exists via UNIQUE(instance_id, repo_hash, pr_hash).
  • Registration gate (the trust anchor). New orb_instances table — every instance that ingests is recorded (registered=0 by default, like das-github-mirror's registered flag). Signals are still stored for everyone (so registration is retroactive), but computeFleetAnalytics counts only registered instances toward the fleet median. A stranger — or a coordinated ring — can't move calibration until an operator opts them in (directly fixes the "median gameable by a bad-actor ring" risk from Fleet-derived gate auto-tune: design + data-readiness gate (do not build yet) #1272).
  • Operator endpoints (internal-token gated): GET /v1/internal/orb/instances lists pending + registered instances with their stored-signal counts; POST /v1/internal/orb/instances/register opts one in ({instanceId, registered?}, defaults true).

Supersedes #1248 (keeps its DoS hardening, drops the shared-secret requirement).

Behavioral note

The fleet surfaces shipped in #1268 (operator dashboard fleetMetrics + the gittensory_get_fleet_analytics MCP tool) now read empty until the operator registers at least one instance — the intended trust posture. Register the home instance once and it flows. (The #1270 end-to-end test will get a one-line register once both land — noted there.)

Scope

  • 8 files; migration 0061_orb_instances.sql is the next contiguous number
  • Portable ON CONFLICT … DO UPDATE upserts (no pg-dialect change needed); internal routes (not in OpenAPI)
  • No secrets/wallets/trust-scores; no new env var

Validation

  • npm run test:ci — green (incl. db:migrations:check, typecheck, coverage, workers, ui)
  • 100% branch coverage on every changed src/** line (ingest.ts, analytics.ts, routes.ts) — verified against the v8 JSON report, incl. both ?? [] guards, the json().catch, the body-cap content-length vs streaming paths, and registered/unregistered fold
  • Updated the feat(orb): surface fleet calibration analytics via dashboard + MCP #1268 fleet tests (operator-dashboard, mcp-fleet-analytics) + orb-analytics to register instances; added an explicit "unregistered stranger excluded" test

Safety

  • Ingest stays frictionless (no shared secret) but cannot skew calibration without operator opt-in
  • Best-effort instance bookkeeping never fails an ingest; analytics fail-safes to an empty report on DB error

Advances #1255.

…mirror model)

Answers the "how should /v1/orb/ingest be protected?" question by following
das-github-mirror's ingress model rather than a shared ingest secret (which would
break #1257's hardwired-on, no-shared-key telemetry — every self-hoster would 401):

- Open ingest, bounded. Read the body behind a 1 MiB ceiling (streaming, with a
  content-length fast-path) → 413 on oversize. Mirrors the body limit das-github-mirror
  puts in front of its open webhook ingress. Dedup already exists via
  UNIQUE(instance_id, repo_hash, pr_hash).
- Registration gate (the trust anchor). New orb_instances table — every instance that
  ingests is recorded (registered=0 by default, like das-github-mirror's registered
  flag); signals are still stored for everyone (so registration is retroactive), but
  computeFleetAnalytics counts ONLY registered instances toward the fleet median. A
  stranger — or a ring of them — cannot move calibration until an operator opts them in.
- Operator endpoints (internal-token gated): GET /v1/internal/orb/instances lists
  pending + registered instances with their stored-signal counts;
  POST /v1/internal/orb/instances/register opts one in (or out).

Behavioral note: the fleet surfaces shipped in #1268 (operator dashboard + MCP tool)
now read empty until the operator registers at least one instance — the intended
trust posture. migrations/0061 + portable ON CONFLICT upserts (no pg-dialect change).

Supersedes #1248.
@dosubot dosubot Bot added the size:L label Jun 24, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

JSONbored added a commit that referenced this pull request Jun 24, 2026
@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.05%. Comparing base (480f2c3) to head (6b4d25c).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1274   +/-   ##
=======================================
  Coverage   95.04%   95.05%           
=======================================
  Files         177      177           
  Lines       19944    19981   +37     
  Branches     7179     7189   +10     
=======================================
+ Hits        18956    18993   +37     
  Misses        395      395           
  Partials      593      593           
Files with missing lines Coverage Δ
src/api/routes.ts 94.44% <100.00%> (+0.03%) ⬆️
src/orb/analytics.ts 100.00% <100.00%> (ø)
src/orb/ingest.ts 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

JSONbored added a commit that referenced this pull request Jun 24, 2026
@JSONbored
JSONbored merged commit 7610d8a into main Jun 24, 2026
19 checks passed
@JSONbored
JSONbored deleted the feat/orb-ingest-registration branch June 24, 2026 22:43
JSONbored added a commit that referenced this pull request Jun 24, 2026
…end-to-end (#1270)

Adds an integration seam test for the full Orb fleet pipeline: a self-host instance's review_audit ledger → exportOrbBatch (FLEET_QUERY join + payload) → handleOrbIngest (orb_signals upsert) → computeFleetAnalytics. Forward-compatible with the #1274 registration gate (registers the instance before asserting analytics). Covers clean-merge precision and human-reversal propagation into fleet reversal/FP rates.
JSONbored added a commit that referenced this pull request Jun 25, 2026
Second piece of the central Gittensory Orb GitHub App (#1255). Maintains
orb_github_installations (migration 0064) from the verified /v1/orb/webhook
`installation` lifecycle events — one row per install of the shared Orb App, recording
account + repository_selection and the suspend/unsuspend/deleted lifecycle. This is the
registry onboarding + the token-broker (later PRs) read to know which installations
exist and who owns them.

- registered=0 by default — the Mirror-style manual-onboarding gate (an install is
  RECORDED but not trusted/active until a human opts it in), mirroring #1274.
- The upsert runs synchronously in the receiver, BEFORE recording the webhook event, so a
  failed registry write is flipped to "error" + 500 and GitHub redelivers (the dedup guard
  only suppresses non-error rows). No-op for every non-installation event.

Additive; stacked on #1293 (the webhook receiver). installation_repositories repo-delta
tracking and PR-outcome processing are follow-ups.

Advances #1255.
JSONbored added a commit that referenced this pull request Jun 25, 2026
Second piece of the central Gittensory Orb GitHub App (#1255). Maintains
orb_github_installations (migration 0064) from the verified /v1/orb/webhook
`installation` lifecycle events — one row per install of the shared Orb App, recording
account + repository_selection and the suspend/unsuspend/deleted lifecycle. This is the
registry onboarding + the token-broker (later PRs) read to know which installations
exist and who owns them.

- registered=0 by default — the Mirror-style manual-onboarding gate (an install is
  RECORDED but not trusted/active until a human opts it in), mirroring #1274.
- The upsert runs synchronously in the receiver, BEFORE recording the webhook event, so a
  failed registry write is flipped to "error" + 500 and GitHub redelivers (the dedup guard
  only suppresses non-error rows). No-op for every non-installation event.

Additive; stacked on #1293 (the webhook receiver). installation_repositories repo-delta
tracking and PR-outcome processing are follow-ups.

Advances #1255.
JSONbored added a commit that referenced this pull request Jun 25, 2026
Second piece of the central Gittensory Orb GitHub App (#1255). Maintains
orb_github_installations (migration 0064) from the verified /v1/orb/webhook
`installation` lifecycle events — one row per install of the shared Orb App, recording
account + repository_selection and the suspend/unsuspend/deleted lifecycle. This is the
registry onboarding + the token-broker (later PRs) read to know which installations
exist and who owns them.

- registered=0 by default — the Mirror-style manual-onboarding gate (an install is
  RECORDED but not trusted/active until a human opts it in), mirroring #1274.
- The upsert runs synchronously in the receiver, BEFORE recording the webhook event, so a
  failed registry write is flipped to "error" + 500 and GitHub redelivers (the dedup guard
  only suppresses non-error rows). No-op for every non-installation event.

Additive; stacked on #1293 (the webhook receiver). installation_repositories repo-delta
tracking and PR-outcome processing are follow-ups.

Advances #1255.
JSONbored added a commit that referenced this pull request Jun 25, 2026
Second piece of the central Gittensory Orb GitHub App (#1255). Maintains
orb_github_installations (migration 0064) from the verified /v1/orb/webhook
`installation` lifecycle events — one row per install of the shared Orb App, recording
account + repository_selection and the suspend/unsuspend/deleted lifecycle. This is the
registry onboarding + the token-broker (later PRs) read to know which installations
exist and who owns them.

- registered=0 by default — the Mirror-style manual-onboarding gate (an install is
  RECORDED but not trusted/active until a human opts it in), mirroring #1274.
- The upsert runs synchronously in the receiver, BEFORE recording the webhook event, so a
  failed registry write is flipped to "error" + 500 and GitHub redelivers (the dedup guard
  only suppresses non-error rows). No-op for every non-installation event.

Additive; stacked on #1293 (the webhook receiver). installation_repositories repo-delta
tracking and PR-outcome processing are follow-ups.

Advances #1255.
JSONbored added a commit that referenced this pull request Jun 25, 2026
Second piece of the central Gittensory Orb GitHub App (#1255). Maintains
orb_github_installations (migration 0064) from the verified /v1/orb/webhook
`installation` lifecycle events — one row per install of the shared Orb App, recording
account + repository_selection and the suspend/unsuspend/deleted lifecycle. This is the
registry onboarding + the token-broker (later PRs) read to know which installations
exist and who owns them.

- registered=0 by default — the Mirror-style manual-onboarding gate (an install is
  RECORDED but not trusted/active until a human opts it in), mirroring #1274.
- The upsert runs synchronously in the receiver, BEFORE recording the webhook event, so a
  failed registry write is flipped to "error" + 500 and GitHub redelivers (the dedup guard
  only suppresses non-error rows). No-op for every non-installation event.

Additive; stacked on #1293 (the webhook receiver). installation_repositories repo-delta
tracking and PR-outcome processing are follow-ups.

Advances #1255.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant