Skip to content

feat(miner): wire stack detection into the coding-task attempt prompt - #5717

Closed
carlh7777 wants to merge 1 commit into
JSONbored:mainfrom
carlh7777:feat/wire-stack-detection-attempt-prompt-4786
Closed

feat(miner): wire stack detection into the coding-task attempt prompt#5717
carlh7777 wants to merge 1 commit into
JSONbored:mainfrom
carlh7777:feat/wire-stack-detection-attempt-prompt-4786

Conversation

@carlh7777

Copy link
Copy Markdown
Contributor

Summary

Closes #4786.

detectRepoStack / renderStackSummary (#4785) already return a structured target-repo stack (language, package manager, build/test/lint/format commands), but nothing in the attempt path consumed them — buildCodingTaskSpec's instructions were issue text + an acceptance-criteria path only. This wires the real detection result into the coding-agent prompt so the agent validates against THIS repository's tooling rather than assuming LoopOver/gittensory CI, Codecov, or npm run test:ci.

How

  • coding-task-spec.js — after a ready acceptance-criteria write, call detectRepoStack(workingDirectory) (injectable for tests; production default is the real detector) and append buildValidationGuidance to instructions: stack summary, an explicit "do not assume gittensory CI" clause, and only the confidently-inferred commands (null commands stay omitted — fail-closed, no guessing).
  • Fail-closed { detected: false } results still reach the prompt via renderStackSummary, so detection failure is honest rather than a silent npm/Codecov default.
  • README updated to reflect the wire-up (was "Detection only; wiring … is the follow-up (Generalized plan → implement → test → submit loop #4786)").

Acceptance criteria

  • A detected Node/Rust/etc. worktree embeds its real stack summary + validation commands in instructions
  • An undetected worktree embeds the fail-closed reason (no silent gittensory default)
  • Partial stacks list only non-null commands; an all-null command set tells the agent not to guess

Out of scope: replacing self-review's LoopOver predicted-gate with foreign-repo check-run success (larger follow-up on the same epic).

Tests

test/unit/miner-coding-task-spec.test.ts — detected Node scripts, fail-closed empty worktree, no-command detected stack, production default detector (Cargo.toml), and both sides of each command ternary (build/test vs lint/format).

Scope

  • The PR title follows type(scope): short summary Conventional Commit format
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME
  • I linked a currently open issue this PR resolves — Closes #4786

Validation

  • npx vitest run test/unit/miner-coding-task-spec.test.ts
  • git diff --check
  • npm run test:ci / full gate (run before push)

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.

…ck commands (JSONbored#4786)

- Updated README to clarify the integration of detected stack commands into coding-agent instructions.
- Modified `coding-task-spec.d.ts` to include an injectable `detectRepoStack` function for improved stack detection.
- Enhanced `coding-task-spec.js` to utilize the detected stack information, ensuring the agent validates against the target repo's tooling rather than default assumptions.
- Added tests to verify that detected stack commands are correctly embedded in the coding-agent instructions and handle various scenarios, including undetected stacks and partial command detection.

This update improves the accuracy of the coding-agent's guidance by leveraging real repository configurations, addressing issues raised in JSONbored#4786.
@carlh7777
carlh7777 requested a review from JSONbored as a code owner July 14, 2026 07:21
@superagent-security superagent-security Bot added the contributor:flagged Contributor flagged for review by trust analysis. label Jul 14, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

🚨 Contributor flagged. Click here for more info: Superagent Dashboard

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

❌ 7 Tests Failed:

Tests completed Failed Passed Skipped
2765 7 2758 0
View the full list of 7 ❄️ flaky test(s)
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > actually narrows the PRs-tracked count to a selected $repo, and 'All' still includes every repo

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.0358s run time
AssertionError: expected '0' to be '3' // Object.is equality

Expected: "3"
Received: "0"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:478:22
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > adds local, webhook-observed issue-activity stat panels alongside the review_targets PR panels (#3716, switched off the GitHub API 2026-07)

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.0213s run time
AssertionError: expected 'SELECT count(*) AS opened FROM issues…' to contain '(${repo:sqlstring} = \'$__all\' OR re…'

Expected: "(${repo:sqlstring} = '$__all' OR repo = ${repo:sqlstring})"
Received: "SELECT count(*) AS opened FROM issues WHERE (${repo:sqlstring} = '__ALL__' OR repo = ${repo:sqlstring}) AND unixepoch(created_at) >= ${__from:date:seconds} AND unixepoch(created_at) < ${__to:date:seconds}"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:343:33
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > drops a bot-authored release PR from the tracked-PR count and table (#4685-follow-up)

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.0427s run time
AssertionError: expected '0' to be '2' // Object.is equality

Expected: "2"
Received: "0"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:546:26
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > filters the pull request table to the selected time window

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.0249s run time
AssertionError: expected '' to contain 'owner/repo|2|new|commented|comment|ne…'

- Expected
+ Received

- owner/repo|2|new|commented|comment|new row|2026-06-29T21:00:00Z

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:446:18
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > issue-activity panels count real rows correctly by state and window

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.0531s run time
AssertionError: expected '0' to be '2' // Object.is equality

Expected: "2"
Received: "0"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:416:20
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > scopes every review_targets panel query to the selected repo with Grafana SQL-string escaping

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.0072s run time
AssertionError: expected 'SELECT count(*) AS prs FROM review_ta…' to contain '(${repo:sqlstring} = \'$__all\' OR re…'

Expected: "(${repo:sqlstring} = '$__all' OR repo = ${repo:sqlstring})"
Received: "SELECT count(*) AS prs FROM review_targets WHERE (submitter NOT LIKE '%[bot]%' OR submitter IS NULL) AND (${repo:sqlstring} = '__ALL__' OR repo = ${repo:sqlstring}) AND unixepoch(updated_at) >= ${__from:date:seconds} AND unixepoch(updated_at) < ${__to:date:seconds}"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:384:32
test/unit/selfhost-grafana-dashboard.test.ts > maintainer Reviews & PRs Grafana dashboard > scopes the issue-activity panels to the selected $repo, same as the PR panels

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 0.00581s run time
AssertionError: expected 'SELECT count(*) AS opened FROM issues…' to contain '(${repo:sqlstring} = \'$__all\' OR re…'

Expected: "(${repo:sqlstring} = '$__all' OR repo = ${repo:sqlstring})"
Received: "SELECT count(*) AS opened FROM issues WHERE (${repo:sqlstring} = '__ALL__' OR repo = ${repo:sqlstring}) AND unixepoch(created_at) >= ${__from:date:seconds} AND unixepoch(created_at) < ${__to:date:seconds}"

 ❯ test/unit/selfhost-grafana-dashboard.test.ts:361:44

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 14, 2026
@loopover-orb

loopover-orb Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 LoopOver review result - reject/close recommended

Review updated: 2026-07-14 07:27:50 UTC

4 files · 1 AI reviewer · 1 blocker · readiness 100/100 · CI failing · blocked

🛑 Suggested Action - Reject/Close

Review summary
This PR wires detectRepoStack/renderStackSummary into buildCodingTaskSpec's instructions builder, appending a stack summary and only non-null validation commands, with a fail-closed path for undetected stacks. The implementation is straightforward and directly matches the description; the test suite exercises the detected-with-commands, fail-closed, no-commands, real-default-detector, and both ternary-arm partial-command cases. The one thing worth confirming is CI: validate and one validate-tests shard failed on this commit, which needs investigation before merge even though the diff itself looks correct.

Blockers

  • CI shows `validate` and `validate-tests (6)` FAILED on this commit — that must be resolved or explained before merge since it's unclear from the diff alone whether it's related to this change (e.g. a stack-detection.js signature mismatch or an unrelated shard).
Nits — 5 non-blocking
  • packages/gittensory-miner/lib/coding-task-spec.js: the `stack?.detected === true` optional-chaining guard in `buildValidationGuidance` suggests `stack` could be nullish, but `detectRepoStack` (per the referenced test file) never returns undefined/null — confirm this defensive check is actually reachable or simplify to `stack.detected`.
  • packages/gittensory-miner/lib/coding-task-spec.js: the JSDoc/comment blocks are quite long relative to the code they describe; consider trimming for maintainability now that the design rationale is captured in the PR/issue history.
  • The 'magic numbers' flagged by the external brief (Generalized plan → implement → test → submit loop #4786 in comments) are just issue references, not literals needing a constant — not an actionable nit.
  • Confirm whether the failing `validate` / `validate-tests (6)` CI jobs are caused by this change (e.g. a type mismatch in coding-task-spec.d.ts or a stack-detection.js export) before merging.
  • Consider whether `buildValidationGuidance`'s optional chaining on `stack` can be removed if `detectRepoStack` is guaranteed to always return a defined result object.

Why this is blocked

  • CI shows `validate` and `validate-tests (6)` FAILED on this commit — that must be resolved or explained before merge since it's unclear from the diff alone whether it's related to this change (e.g. a stack-detection.js signature mismatch or an unrelated shard).
📋 Copy for AI agents — paste into your coding agent
Fix the following blocker(s) from this PR review:

1. CI shows \`validate\` and \`validate-tests \(6\)\` FAILED on this commit — that must be resolved or explained before merge since it's unclear from the diff alone whether it's related to this change \(e.g. a stack-detection.js signature mismatch or an unrelated shard\).

CI checks failing

  • validate
  • validate-tests (6)
Flagged checks (non-blocking)
  • Contributor trust — Contributor flagged for review
Signal Result Evidence
Code review ❌ 1 blocker 1 reviewer
Linked issue ✅ Linked #4786
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 182 registered-repo PR(s), 115 merged, 17 issue(s).
Contributor context ✅ Confirmed Gittensor contributor carlh7777; Gittensor profile; 182 PR(s), 17 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence. LLM value judgment: moderate — The change closes a real, previously-identified gap (#4786) by connecting existing stack-detection output to the coding-agent prompt, with fail-closed behavior and solid ternary-branch test coverage, making it a well-targeted, non-speculative follow-up rather than scope creep.
Linked issue satisfaction

Not yet addressed
This PR closes #4786 by wiring stack detection into the coding-task prompt, which is prep work referenced by this issue's boundaries — but this issue itself asks for a full plan/implement/test/submit execution loop that actually completes an end-to-end cycle and passes CI against a real non-gittensory test repo, which this diff does not attempt (it only changes prompt instructions, with no loop ex

Review context
  • Author: carlh7777
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, JavaScript, TypeScript, Cuda, HTML, MDX, Rust
  • Official Gittensor activity: 182 PR(s), 17 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb

loopover-orb Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Gittensory is closing this pull request on the maintainer's behalf (CI is failing (validate, validate-tests (6)); AI reviewers agree on a likely critical defect: CI shows `validate` and `validate-tests (6)` FAILED on this commit — that must be resolved or explained before merge since it's unclear from the diff alone whether it's related to this change (e.g. a stack-detection.js signature mismatch or an unrelated shard).). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor:flagged Contributor flagged for review by trust analysis. gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generalized plan → implement → test → submit loop

1 participant