Skip to content

feat(miner-hands): tree-sitter-based repo map builder #4280

Description

@JSONbored

A repo-map builder gives a coding-agent driver (or the acceptance-criteria/prompt-packet builders upstream of it) a compact, structural view of the target repository — function/class/symbol signatures across the codebase — without paying the token cost of dumping every file's full contents into the prompt. This is a well-established convergent pattern (SWE-agent, Aider, and others independently arrived at a tree-sitter-parsed repo map for exactly this reason). Verified via grep -rn "tree-sitter" package.json packages/*/package.json — zero hits, so tree-sitter (in whatever binding form) is not a dependency anywhere in this repo today; this issue adds it.

Deliverables

  • Add a tree-sitter dependency, preferring the WASM binding (web-tree-sitter) over native bindings given this repo also ships a Cloudflare Workers deployment target ("dev": "wrangler dev", "deploy": "wrangler deploy" in package.json) where native Node addons are not an option — confirm whether this module actually needs to run in that context before committing to a binding; if it only ever runs in the local miner/CLI process, a native binding may be acceptable and simpler.
  • A repo-map builder that walks a target repository's source files, parses each with the appropriate language grammar, and extracts a compact symbol/signature outline (not full file bodies), bounded to a fixed output size so it can't blow out a prompt budget on a large repo.
  • Graceful degradation for languages/files without an available grammar (skip, don't crash the whole map) and for tree-sitter itself failing to initialize (fall back to a simpler heuristic, e.g. a plain file listing, rather than blocking the whole driver invocation).
  • Tests against fixture files per supported language, asserting the extracted symbol outline is stable (a golden-fixture style test, similar in spirit to test/contract/engine-parity.test.ts's golden-JSON comparisons, though this is a new, separate suite).
  • Document which languages are supported at launch (likely starting with TypeScript/JavaScript, given that's what this repo and most target repos are written in) and how to add another grammar later.

References

  • Verified via grep -rn "tree-sitter" package.json packages/*/package.json — zero hits, confirming this is a net-new dependency
  • package.json: "dev": "wrangler dev", "deploy": "wrangler deploy" (confirms a Workers runtime target exists in this repo — relevant to the native-vs-WASM binding choice, though verify this module's actual runtime context before assuming the constraint applies)
  • test/contract/engine-parity.test.ts:1-11 (existing golden-fixture test STYLE this issue's fixture tests can follow, not a shared harness to extend)
  • Feeds the acceptance-criteria file (feat(miner-hands): immutable acceptance-criteria file written before the coding agent starts #4271, this batch) and packages/gittensory-engine/src/prompt-packet.ts as an additional context source, though wiring it into those is separable follow-up, not required by this issue

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:featureGittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.gittensor:priorityMaintainer-selected Gittensor priority — scores a 1.5x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions