Skip to content

feat(miner-manage): add optional anonymized Orb telemetry export - #4479

Merged
JSONbored merged 1 commit into
JSONbored:mainfrom
jeffrey701:feat/miner-orb-export
Jul 9, 2026
Merged

feat(miner-manage): add optional anonymized Orb telemetry export#4479
JSONbored merged 1 commit into
JSONbored:mainfrom
jeffrey701:feat/miner-orb-export

Conversation

@jeffrey701

Copy link
Copy Markdown
Contributor

Closes #4277

What

The self-host Orb collector (src/selfhost/orb-collector.ts, #1255) is always-on for a maintainer's own instance. A miner runs on a third-party contributor's laptop with a much lower consent bar, so this export is opt-in (default OFF) — the "optional" in the title. It mirrors the collector's privacy posture, adapted for the miner's local SQLite world.

Privacy posture (mirrors the collector)

  • HMAC-anonymized identifiers. Repo/PR ids are hashed with a per-instance dedicated secret — 256-bit, generated once via node:crypto, persisted in local SQLite, single-purpose (never any App/webhook credential). A repo/PR always hashes the same way (so the receiver can dedup), but the raw name never leaves the machine.
  • Low-cardinality only. Only the decision + a reasonBucket (already one of the miner's REJECTION_REASONS, else "none") + closedAt are exported — never raw repo names or free text.
  • Local data source. Reads the local pr_outcome ledger (pr-outcome.js readPrOutcomes), not a hosted D1. Local SQLite also holds the per-instance secret and the export cursor (the miner has no system_flags/orb_export_cursor table).

Scoped to building the anonymized batch + managing the local secret/cursor — performing the network POST is the caller's job, so the module stays pure over its inputs + local store and needs no network to test.

API (packages/gittensory-miner/lib/orb-export.js)

  • ORB_EXPORT_ENABLED_BY_DEFAULT = false — the opt-in default.
  • openOrbExportStore(dbPath?) — local SQLite store: getOrCreateAnonSecret(), getCursor()/setCursor() (0o700 dir, 0o600 file — the secret stays on-machine).
  • hmacAnonymize(value, key) — sha256 HMAC, first 24 hex (mirrors the collector's hmacField).
  • buildAnonymizedOrbBatch(outcomes, key) — pure; maps a readPrOutcomes map to anonymized rows, deterministic order.
  • collectOrbExportBatch({ store, eventLedger, enabled }) — opt-in-gated; returns null unless enabled.

Files

  • packages/gittensory-miner/lib/orb-export.js (+ hand-written .d.ts sidecar)
  • test/unit/miner-orb-export.test.ts

Testing

npx vitest run test/unit/miner-orb-export.test.ts
npm run typecheck

10/10 tests pass (opt-in gate, secret stability across reopens, anonymization determinism + raw-value hiding, reason bucketing, malformed-record skip — over a real temp-file SQLite store); typecheck clean. packages/gittensory-miner/lib/** carries no coverage wall.

The self-host Orb collector (src/selfhost/orb-collector.ts, JSONbored#1255) is ALWAYS-ON
for a maintainer's own instance; a miner runs on a third-party contributor's
laptop with a much lower consent bar, so this export is OPT-IN (default OFF) —
hence "optional". It mirrors the collector's privacy posture but adapts it for
the miner:

- Repo/PR identifiers are HMAC-anonymized with a per-instance DEDICATED secret
  (256-bit, generated once via node:crypto, persisted in local SQLite, single-
  purpose) — the same key-separation posture as getOrCreateAnonSecret, so a
  repo/PR always hashes the same way but the raw name never leaves the machine.
- Only the decision + a low-cardinality reason bucket (already one of the
  miner's REJECTION_REASONS, else "none") + closedAt are exported — never raw
  repo names or free text.
- The data source is the local pr_outcome ledger (pr-outcome.js readPrOutcomes),
  not a hosted D1. Local SQLite also holds the per-instance secret and the
  export cursor (the miner has no system_flags / orb_export_cursor table).

Scoped to building the anonymized batch + managing the local secret/cursor;
performing the network POST is the caller's job, so the module stays pure over
its inputs + local store and needs no network to test.

- packages/gittensory-miner/lib/orb-export.js (+ hand-written .d.ts): the store
  (openOrbExportStore), hmacAnonymize, buildAnonymizedOrbBatch (pure), and the
  opt-in-gated collectOrbExportBatch.
- test/unit/miner-orb-export.test.ts: opt-in gate, secret stability across
  reopens, anonymization (deterministic + hides raw), reason bucketing, and the
  malformed-record skip, over a real temp-file SQLite store.

Closes JSONbored#4277
@jeffrey701
jeffrey701 requested a review from JSONbored as a code owner July 9, 2026 21:23
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. labels Jul 9, 2026
@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.03%. Comparing base (453c5d4) to head (c4c6a76).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4479   +/-   ##
=======================================
  Coverage   94.03%   94.03%           
=======================================
  Files         420      420           
  Lines       37521    37521           
  Branches    13701    13701           
=======================================
  Hits        35282    35282           
  Misses       1583     1583           
  Partials      656      656           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb

loopover-orb Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-09 21:38:57 UTC

3 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This adds a well-scoped, opt-in-by-default (ORB_EXPORT_ENABLED_BY_DEFAULT = false) local module that mirrors src/selfhost/orb-collector.ts's HMAC-anonymization posture for the miner's local SQLite pr_outcome ledger. The code is pure and self-contained: it builds an anonymized batch and manages a local secret/cursor store, but performs no network call and includes no caller wiring (no CLI flag, no invocation site) in this diff — the PR description states that's deliberately out of scope. Tests cover the happy paths (secret persistence, hashing, batch building, opt-in gating) but never exercise resolveOrbExportDbPath's env-var branches or normalizeDbPath's empty-path throw, and the store's getCursor/setCursor are exposed but never consulted by collectOrbExportBatch, so incremental (non-duplicate) export isn't actually functional yet within this module.

Nits — 5 non-blocking
  • packages/gittensory-miner/lib/orb-export.js:71 — buildAnonymizedOrbBatch trusts outcome.reason as already bucketed to REJECTION_REASONS and only falls back to "none" on falsy values; since pr-outcome.js isn't in this diff, confirm it guarantees bucketed values, otherwise free text could leak through despite the "low-cardinality only" privacy claim.
  • packages/gittensory-miner/lib/orb-export.js:22-40 — resolveOrbExportDbPath's env-var precedence branches (GITTENSORY_MINER_ORB_EXPORT_DB, GITTENSORY_MINER_CONFIG_DIR, XDG_CONFIG_HOME) and normalizeDbPath's empty-path throw have no test coverage in test/unit/miner-orb-export.test.ts.
  • packages/gittensory-miner/lib/orb-export.js:49,83,85,105 — magic numbers (24-char hash truncation, 0o700/0o600 modes, 32-byte secret) lack named constants; consider hoisting them like the collector likely would.
  • packages/gittensory-miner/lib/orb-export.js:58 — buildAnonymizedOrbBatch's cyclomatic complexity (~11) is over the repo's usual threshold of 10; consider extracting the row-validation predicate into a helper.
  • openOrbExportStore exposes getCursor/setCursor but collectOrbExportBatch never reads or advances the cursor, so as written repeated calls would re-collect the full outcome history rather than an incremental delta — confirm this filtering is intentionally deferred to the (not-yet-written) caller.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #4277
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 106 registered-repo PR(s), 45 merged, 4 issue(s).
Contributor context ✅ Confirmed Gittensor contributor jeffrey701; Gittensor profile; 106 PR(s), 4 issue(s).
Gate result ✅ Passing No configured blocker found.
Linked issue satisfaction

Partially addressed
The PR delivers the opt-in gated module, per-instance HMAC secret, pure anonymized batch builder, and a local cursor store as requested, but it does not port or reuse `bucketReasonCode` (it passes the raw `reason` field through, risking a divergent taxonomy from the self-host exporter), does not wire the persisted cursor into `collectOrbExportBatch` to actually filter/advance on re-runs, and has n

Review context
  • Author: jeffrey701
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 106 PR(s), 4 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@JSONbored
JSONbored merged commit 6f8e40a into JSONbored:main Jul 9, 2026
10 checks passed
@loopover-orb loopover-orb Bot removed the gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. label Jul 9, 2026
@JSONbored JSONbored added the gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. label Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

feat(miner-manage): optional anonymized Orb telemetry export for miner outcomes

2 participants