feat(miner): add cross-repo evaluation harness (#4788)

[andriypolanski]

Summary

• Adds the Cross-repo evaluation harness #4788 deliverable for Rent-a-Loop Wave 5: a committed benchmark repo set, a repeatable evaluation run, and a per-repo pass/fail report with failures categorized as stack-detection gaps, execution readiness gaps, leaked gittensory assumptions, clone/setup problems, or other.
• The harness is readiness-only — it exercises detectRepoStack, resolveMinerGoalSpec, and buildCodingTaskSpec on each benchmark repo without running the coding agent or calling forge APIs.
• A strict-majority summary reports how many repos pass without a target-repo .loopover-miner.yml.

Change

• packages/loopover-miner/lib/cross-repo-evaluation.js (+ .d.ts) — manifest parser, per-repo readiness evaluator, assumption scanner, report formatter, and summary reducer.
• packages/loopover-miner/benchmarks/cross-repo/manifest.json — six diverse public repos (Node, Python, Rust, Go) with optional requireTestCommand flags.
• packages/loopover-miner/scripts/cross-repo-evaluation.mjs — CLI with --manifest, --repo, --json, and --require-majority.
• packages/loopover-miner/docs/cross-repo-evaluation.md — operator docs for cloning benchmark repos and reading the report.
• test/unit/miner-cross-repo-evaluation.test.ts — 29 unit tests covering manifest parsing, taxonomy, evaluator branches, and the script driver.
• packages/loopover-miner/package.json — cross-repo-eval script + node --check for the new lib module.
• package.json — cross-repo-eval:miner root script.
• packages/loopover-miner/README.md — link to the harness docs.

Closes #4788

Scope

• The PR title follows type(scope): short summary Conventional Commit format.
• This PR is focused (cross-repo evaluation harness only) and does not mix unrelated changes.
• This follows CONTRIBUTING.md and does not touch site/, CNAME, or **/lovable/**.
• Linked issue: Closes Cross-repo evaluation harness #4788.

Validation

• git diff --check
• npm run test:ci
• npm audit --audit-level=moderate
• npm run test -- test/unit/miner-cross-repo-evaluation.test.ts — 29 passing
• npm run build --workspace @loopover/miner — node --check includes lib/cross-repo-evaluation.js

If any required check was skipped, explain why:

• Run the full npm run test:ci + npm audit --audit-level=moderate gate locally before push (required for merge).

Safety

• No secrets, wallet details, hotkeys, coldkeys, PATs, private keys, trust scores, or private rankings exposed.
• Public GitHub text unaffected — miner harness + tests + docs only; no runtime API change.
• Auth/cookie/CORS/session — n/a.
• API/OpenAPI/MCP — n/a.
• No visible UI change.
• No changelog edit.

UI Evidence

No visible UI change. This PR adds a miner evaluation harness, CLI, benchmark manifest, tests, and docs only. No screenshots required.

Notes

• Live evaluation against the committed benchmark set requires cloning the listed repos into LOOPOVER_MINER_REPO_CLONE_DIR first (documented in docs/cross-repo-evaluation.md). CI validates the pure evaluator via fixture paths in unit tests.
• Depends on De-hardcode discovery from gittensory's own conventions #4784–Generalized plan → implement → test → submit loop #4786 (forge config, stack detection, coding-task-spec guidance) already shipped on main.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.08%. Comparing base (e615ec7) to head (2f8ed92).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5790      +/-   ##
==========================================
+ Coverage   95.06%   95.08%   +0.01%     
==========================================
  Files         580      581       +1     
  Lines       46156    46310     +154     
  Branches    14804    14804              
==========================================
+ Hits        43880    44035     +155     
+ Misses       1516     1515       -1     
  Partials      760      760              

Flag	Coverage Δ
shard-1	43.78% <0.00%> (-0.15%)	⬇️
shard-2	36.13% <100.00%> (+0.36%)	⬆️
shard-3	32.16% <0.00%> (-0.10%)	⬇️
shard-4	32.84% <0.00%> (-0.11%)	⬇️
shard-5	31.64% <0.00%> (-0.11%)	⬇️
shard-6	44.48% <0.00%> (-0.15%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...ckages/loopover-miner/lib/cross-repo-evaluation.js	100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[loopover-orb]

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ LoopOver review result - approve/merge recommended

_{Review updated: 2026-07-14 16:41:55 UTC}

10 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

• safe to merge

Review summary
This adds a readiness-only cross-repo evaluation harness (manifest parser, evaluator, CLI, docs, 29 unit tests) that exercises detectRepoStack/resolveMinerGoalSpec/buildCodingTaskSpec against six pinned public benchmark repos without running a live coding agent, matching the stated #4788 scope. The manifest parsing is defensively written (size cap, entry cap, owner/repo path-traversal guards via normalizeCrossRepoFullName) and evaluateRepoReadiness cleanly threads through stack-detection, execution-readiness, and gittensory-assumption-leak checks with injectable dependencies for testing. I don't see a reachable correctness bug in the diff itself — the main soft spot is a semantic quirk in how `usedDefaultGoalSpec: null` (for entries that fail before goal-spec resolution) gets folded into the 'withoutGittensoryConfig' summary bucket.

Nits — 5 non-blocking

• packages/loopover-miner/lib/cross-repo-evaluation.js: `summarizeCrossRepoEvaluation`'s `withoutGittensoryConfig` filter (`r?.usedDefaultGoalSpec !== false`) counts entries that failed before goal-spec was ever checked (e.g. clone_setup or invalid-name failures, where the field is `null`) as 'without gittensory config', which overstates that count in a mostly-failing run — consider excluding `usedDefaultGoalSpec === null` from the numerator.
• packages/loopover-miner/lib/cross-repo-evaluation.js:217 `defaultClaimLedger(repoFullName)` never uses its parameter — drop it or prefix `_repoFullName` to avoid an unused-var lint hit.
• `CROSS_REPO_FAILURE_CATEGORY.GITTENSOR_ASSUMPTION` (key spelled without the trailing 'y') maps to the string value `"gittensory_assumption"` — the key/value spelling mismatch is a bit confusing next to `GITTENSOR_POSITIVE_ASSUMPTION_CHECKS`, which has the same drift; align the identifiers for consistency.
• `evaluateRepoReadiness` and `normalizeRepoList` are both fairly large single functions with several sequential early-return branches — consider extracting the manifest-entry-normalization switch and the readiness-check pipeline into smaller named steps for readability.
• Distinguish 'default goal spec used because none was found' from 'goal spec never checked because the repo failed earlier' in the summary output, e.g. track `usedDefaultGoalSpec` only for repos that reached that check.

Signal	Result	Evidence
Code review	✅ No blockers	1 reviewer
Linked issue	✅ Linked	#4788
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 151 registered-repo PR(s), 98 merged, 26 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	andriypolanski; Gittensor profile; 151 PR(s), 26 issue(s).
Gate result	✅ Passing	No configured blocker found.
Improvement	✅ Minor	risk: clean · value: minor — Code changes are accompanied by test evidence.

Review context

• Author: andriypolanski
• Role context: outside_contributor
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: Python, TypeScript, JavaScript, Rust, Cuda, Kotlin, MDX, Scala
• Official Gittensor activity: 151 PR(s), 26 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Keep the PR focused and include validation evidence before maintainer review.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

• @gittensory ask <question> answers contribution-quality Q&A with source citations and freshness.
• @gittensory chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
• A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

• Re-run LoopOver review

[loopover-orb]

Gittensory approves — the gate is satisfied and CI is green.