Skip to content

feat(orb): add the opt-in federated collector transport client (push/pull) - #6537

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
reyanthony062001-ops:feat/6479-collector-client
Jul 16, 2026
Merged

feat(orb): add the opt-in federated collector transport client (push/pull)#6537
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
reyanthony062001-ops:feat/6479-collector-client

Conversation

@reyanthony062001-ops

Copy link
Copy Markdown
Contributor

Closes #6479. Transport side of federated fleet intelligence (#1970), following the export path merged in #6505 (#6478).

pushFederatedBundle POSTs this instance's own anonymized bundle to the collector an operator configures; pullPeerBundles GETs peer bundles from it. Both are best-effort and off by default.

Scope: transport only — and the import side is deliberately absent

A pulled bundle is fetched, shape-checked, and returned. It is never signature-verified, never trust-gated, never persisted. That is #6480, which is blocked on #6477 (the key-trust/anti-poisoning design).

Verifying here would not merely be out of scope — it would be wrong: there is no trust anchor to verify against yet, and inventing one is exactly what #6477 exists to decide (src/orb/federated-bundle.ts:170-174). So pullPeerBundles returns FederatedSignalBundle[] for a future consumer, mirroring the pure-library shape #6505 already landed.

No default collector, by design

The client only ever talks to an endpoint the operator configured. No hardcoded fallback, no auto-discovery — this codebase's self-host posture assumes no central/managed collector exists. With enabled: true but no collectorUrl, nothing is sent, because there is nowhere to send it. (Contrast the #1255 orb path, which POSTs to a hosted default at orb-collector.ts:168 — a different feature with a different contract.)

Config: extends the existing block, does not add a second one

federatedIntelligence:
  enabled: true
  collectorUrl: https://collector.example.org/v1/federated   # public HTTPS only
  collectorMode: both   # push | pull | both

The issue asks for the inherit | off | enabled convention — the existing {present, enabled} shape already encodes exactly that (present: false = inherit), so this extends federatedIntelligence: rather than duplicating the toggle.

collectorUrl is SSRF-guarded twice:

  • At config-read time — cloned from parseVisualProductionUrl, so an http://, localhost, 127.0.0.1, 10.x, 192.168.x, or *.internal endpoint is dropped with a warning and can never reach the client.
  • At call time — re-checked in resolveCollectorEndpoint (the src/orb/relay.ts:230 idiom), so a snapshot round-tripped through the KV cache cannot smuggle an unsafe URL past a guard it was never parsed by.

Reuse over invention

  • Rate limiting + backoff reuse the engine's existing pure helpersevaluateLocalRateLimit and jitteredBackoffMs (packages/loopover-engine/src/governor/rate-limit.ts, exported at index.ts:156). Both take injected randomness, so retries are deterministic in tests. No new scheme, and the bucket stays caller-owned so this module remains state-free like federated-bundle.ts.
  • FederatedSignalBundle is imported, never redefined — per the issue's "do not invent a second bundle format".
  • fetchFn injection + AbortSignal.timeout follow orb-collector.ts:155/215.
  • 5xx/network retries with jittered backoff; a 4xx is not retried — an operator misconfiguration fails identically next time.

Fail-safe

Every entry point resolves the opt-in before touching the DB or the network, and the whole body is wrapped so nothing here can throw into the gate. The gate never awaits this; it is background best-effort sync. A collector that is unreachable, slow, rate-limited, or returning garbage is indistinguishable from "no peers yet" to every caller.

Verification

  • typecheck, docs:drift-check, engine-parity:drift-check, manifest:drift-check, command-reference:check, selfhost:env-reference:check, ui:openapi:check, db:schema-drift:check — all pass. Example configs byte-identical across both templates (config-templates 24/24).
  • 743 tests pass across the touched suites (21 new in federated-collector.test.ts).
  • Patch coverage measured, not assumed: src/orb/federated-collector.ts 72/72 statements, 64/64 branches (100%); every added line in the engine focus-manifest.ts covered. Covered arms include: opted-out (null/absent/disabled/no-URL/wrong-mode) proving zero network calls via a throwing Proxy fetch; push/pull happy paths; 5xx retried to exhaustion with backoff asserted; 4xx not retried; timeout; non-JSON body; non-array payload; unknown schemaVersion / missing signature dropped; rate-limit exhausted ⇒ zero fetches; every SSRF-guard rejection arm; and a throwing injected dependency still degrading rather than throwing.
  • A privacy regression test asserts the serialized push body contains no repo name, PR id, or internal key.

…pull)

Adds the transport side of federated fleet intelligence (JSONbored#1970): pushFederatedBundle POSTs this instance's
own anonymized bundle to the collector an operator configures, and pullPeerBundles GETs peer bundles from it.

Extends the existing .loopover.yml federatedIntelligence: block (JSONbored#6478) with collectorUrl + collectorMode
rather than adding a second toggle. There is deliberately NO default collector and no auto-discovery: with no
collectorUrl configured, enabled: true still sends nothing, because this codebase's self-host posture assumes
no central service exists. collectorUrl is validated at config-read time against the same isSafeHttpUrl SSRF
guard every other URL-valued manifest field uses, and re-checked at call time (the src/orb/relay.ts idiom) so
a snapshot round-tripped through KV cannot smuggle an unsafe endpoint past it.

Export side only for the payload: pulled bundles are shape-checked and returned, never signature-verified,
never trust-gated and never persisted. That is JSONbored#6480, which is blocked on JSONbored#6477 -- verifying here would not
just be out of scope, it would be wrong, because there is no trust anchor to verify against yet.

Fail-safe throughout: every entry point resolves the opt-in before touching the DB or the network, and the
whole body is wrapped so nothing here can throw into the gate. 5xx/network retries use jittered exponential
backoff; a 4xx is not retried. Rate limiting and backoff reuse the engine's existing pure helpers
(evaluateLocalRateLimit, jitteredBackoffMs) rather than inventing a scheme.

Closes JSONbored#6479
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.57%. Comparing base (53ce7b2) to head (748aea4).
⚠️ Report is 33 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6537   +/-   ##
=======================================
  Coverage   95.57%   95.57%           
=======================================
  Files         590      591    +1     
  Lines       47170    47235   +65     
  Branches    15007    15034   +27     
=======================================
+ Hits        45081    45146   +65     
  Misses       1297     1297           
  Partials      792      792           
Flag Coverage Δ
shard-1 43.98% <5.97%> (-0.06%) ⬇️
shard-2 36.76% <5.97%> (-0.05%) ⬇️
shard-3 32.38% <7.46%> (+0.05%) ⬆️
shard-4 34.69% <89.55%> (+0.11%) ⬆️
shard-5 31.08% <5.97%> (-0.04%) ⬇️
shard-6 45.33% <16.41%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
packages/loopover-engine/src/focus-manifest.ts 99.00% <100.00%> (+<0.01%) ⬆️
src/orb/federated-collector.ts 100.00% <100.00%> (ø)
src/signals/focus-manifest.ts 99.58% <ø> (ø)

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 16, 2026
@loopover-orb

loopover-orb Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Important

🟪🟪🟪🟪🟪🟪🟪🟪🟪🟪🟪🟪

🔍 LoopOver is reviewing…

AI analysis is in progress. This comment will update when the review is complete.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed · 🟪 Reviewing

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoopOver approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 8ab3472 into JSONbored:main Jul 16, 2026
16 checks passed
This was referenced Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Opt-in collector client for federated fleet intelligence (push/pull transport)

1 participant