feat(miner-hands): tree-sitter-based repo map builder - #4542
Conversation
Add buildRepoMap/renderRepoMap: a compact, structural view of a target repository (function/class/method/interface/type signatures) for a coding-agent driver or the acceptance-criteria/prompt-packet builders upstream of it, without paying the token cost of dumping full file contents into a prompt. Parses with web-tree-sitter (the WASM binding, not a native addon -- this package also ships a Cloudflare Workers deployment target where native Node addons are not an option) using prebuilt grammars from tree-sitter-wasms. Supports JavaScript/TypeScript/TSX today. Graceful degradation: an unsupported file extension or a grammar that fails to load/parse is reported via `skipped` on its RepoMapFileEntry, never thrown -- this module extracts what it safely can rather than blocking the whole driver invocation. renderRepoMap bounds its output to a configurable char budget, truncating (with a marker) rather than blowing out a prompt budget on a large repo. Adds web-tree-sitter and tree-sitter-wasms as new gittensory-engine dependencies (0 npm audit findings); verified npm run ui:build still tree-shakes them out of the Cloudflare Workers bundle since src/ never imports the miner/ subpath. Closes JSONbored#4280
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4542 +/- ##
==========================================
+ Coverage 94.04% 94.05% +0.01%
==========================================
Files 422 423 +1
Lines 37600 37672 +72
Branches 13736 13756 +20
==========================================
+ Hits 35362 35434 +72
Misses 1583 1583
Partials 655 655
🚀 New features to boost your workflow:
|
|
Tip 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 ✅ Gittensory review result - approve/merge recommendedReview updated: 2026-07-10 00:36:36 UTC
✅ Suggested Action - Approve/Merge
Review summary Nits — 5 non-blocking
Linked issue satisfactionAddressed Review context
Contributor next steps
Signal definitions
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
Summary
A repo-map builder gives a coding-agent driver (or the acceptance-criteria/prompt-packet builders upstream of it) a compact, structural view of a target repository — function/class/method/interface/type signatures — without paying the token cost of dumping every file's full contents into the prompt.
buildRepoMap(files, options?): parses each file withweb-tree-sitter(the WASM binding, not a native addon — this package also ships a Cloudflare Workers deployment target where native Node addons are not an option) using prebuilt grammars fromtree-sitter-wasms, and extracts aRepoMapFileEntryper file.export default class {}/export default function() {}expressions (reported with an"<anonymous>"name) rather than silently dropping them.skipped: "unsupported_language" | "grammar_unavailable"on its entry — never thrown. A language's grammar is loaded (and cached) at most once perbuildRepoMapcall even across many files.renderRepoMap(entries, maxOutputChars?): a pure renderer producing a bounded plain-text outline, truncating (with a marker line) once the char budget would be exceeded, so it can't blow out a prompt budget on a large repo.const foo = ...declarator are not walked up to their binding identifier — an anonymous class/function expression reports"<anonymous>"rather than resolving a variable name. Reasonable follow-up, not attempted here.web-tree-sitterandtree-sitter-wasmsas newgittensory-enginedependencies —npm auditreports 0 findings. Verifiednpm run ui:buildstill tree-shakes them out of the Cloudflare Workers bundle (src/never imports theminer/subpath of the engine).Closes #4280
Test plan
test/unit/repo-map.test.ts(25 tests, 100% statement/branch/function/line coverage on the new file): extension→grammar mapping, real parses of JS/TS/TSX fixtures (function/class/method/interface/type-alias extraction, 1-indexed line numbers, anonymous class/function expressions), signature truncation, unsupported-extension skip (without attempting any grammar load),grammar_unavailableon both a rejecting loader and a loader returning a language that fails to parse, per-language grammar-load caching across multiple files, andrenderRepoMap's bounded output (normal render, skipped/empty-file placeholders, multi-symbol files, and truncation triggered at every distinct line-kind: header, skipped, no-symbols, and mid-symbol-list)buildRepoMap+renderRepoMapagainst a real file in this repo (packages/gittensory-engine/src/miner/attempt-metering.ts) and confirmed the extracted outline matches the file's actual types/functionsnpm run typechecknpm audit --audit-level=moderate(0 vulnerabilities)npm run ui:build(Workers bundle unaffected by the new dependencies)npm run test:ci(full local gate, green)