Skip to content

feat(miner-manage): optional anonymized Orb telemetry export for miner outcomes #4277

Description

@JSONbored

src/selfhost/orb-collector.ts (#1255) is the existing Orb telemetry pattern: exportOrbBatch (orb-collector.ts:159-238) reads de-noised, reversal-aware rows from the self-hosted instance's D1 review_audit, HMAC-anonymizes the repo/PR identifiers with a per-instance secret that's generated once and persisted (getOrCreateAnonSecret, orb-collector.ts:74-93 — deliberately a DEDICATED secret, never the App private key or webhook secret), buckets the free-text gate reason into a fixed low-cardinality category (bucketReasonCode, orb-collector.ts:97-107), and POSTs a signed, cursor-advancing batch to https://gittensory-api.aethereal.dev/v1/orb/ingest. Critically, for a self-hosted instance this export is described as "ALWAYS ON once the GitHub App is configured... there is no opt-out flag" (orb-collector.ts:7-9) — that's the right default for a maintainer's own self-hosted instance, but wrong for a miner: a miner runs on a third-party contributor's own laptop with no App key and a much lower consent bar, which is exactly why this roadmap item is titled "optional."

The miner has no D1/review_audit to read from — its equivalent source is local SQLite: the pr_outcome writer from the sibling issue (packages/gittensory-miner/lib/pr-outcome.js, once built) plus event-ledger.js's general history. It also has no system_flags table for a persisted anonymization secret or orb_export_cursor for a watermark — both would need local-SQLite equivalents, most naturally living alongside (or reusing) event-ledger.js's existing since-cursor read pattern (readEvents({ since }), event-ledger.js:167-184) rather than inventing a new cursor mechanism.

Deliverables

  • packages/gittensory-miner/lib/orb-export.jsopt-in only (default OFF; enabled via an explicit config flag or GITTENSORY_MINER_ORB_EXPORT=1, unlike the self-host always-on default), reading miner outcome events (from the sibling pr-outcome.js writer) instead of D1 review_audit
  • A per-miner-instance anonymization secret, generated once and persisted locally (e.g. alongside the other local SQLite stores under the miner's config dir) — mirroring getOrCreateAnonSecret's separation-of-keys principle (never reuse any GitHub token as the anonymization key)
  • Reuse (or port) bucketReasonCode (orb-collector.ts:97-107) so the same fixed reason-code taxonomy is used fleet-wide across self-host AND miner exporters — do not invent a second taxonomy
  • A local watermark/cursor so re-running the export only sends new outcomes, mirroring the since-based pattern already in event-ledger.js:167-184 rather than a new orb_export_cursor-style table
  • Same no-PII contract as the self-host exporter: no diffs, code, comments, logins, or commit SHAs — only verdict + outcome + a bucketed reason + cycle time, with repo/PR identifiers HMAC'd
  • Respect an air-gap style opt-out consistent with ORB_AIR_GAP (orb-collector.ts:161) for symmetry with the self-host flag naming
  • Unit tests with an injected fetchFn (no real network) covering: opt-in gating (export is a no-op when disabled), anonymization (repo/PR never appear in plaintext in the payload when anonymize is on), and cursor advancement

References

  • src/selfhost/orb-collector.ts:1-17,74-93,97-107,159-238 (the full pattern to mirror: header contract, getOrCreateAnonSecret, bucketReasonCode, exportOrbBatch)
  • packages/gittensory-miner/lib/event-ledger.js:167-184 (readEvents({ since }) — the cursor pattern to reuse instead of inventing a new one)
  • Sibling issue: feat(miner-manage): local pr_outcome record writer (the local source table this reads from)

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:featureGittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.gittensor:priorityMaintainer-selected Gittensor priority — scores a 1.5x multiplier.help wantedExtra attention is needed

    Projects

    Status
    Done

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions