You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Diagnosed capability gap: Agents running gh-aw workflows with a narrow tools:/bash allowlist have no procedural guidance for handling tool-denial events. When denied, they tend to retry variants of the same disallowed command until hitting the harness's hard denial-count threshold (commonly 3), which aborts the entire session. They also sometimes misreport an intentionally scoped-down toolset as a credentials/infra problem via missing_tool.
Training evidence:Code Scanning Fixer run 31919288964 — workflow restricts tools.bash to ["git diff:*", "git restore:*", wc]. Agent tried sed -n ... file.js (denied), read(/tmp/.../file.js) (denied), then git status --short\ngit diff --stat (denied, since only git diff/git restore are allowed) — hit 3/3 denials, guard.tool_denials_exceeded fired, session aborted. The agent's missing_tool call then blamed "token scopes, repository permissions" — misdiagnosing an intentional least-privilege config as an infra bug.
Three other failures reviewed in the same training window (Smoke DeepSeek Harness npm ETARGET version-pinning error, Agentic Commands route job "Argument list too long" argv-size bug, Design Decision Gate missing workflow_dispatchpr_number input) were determined to be infrastructure/code bugs, not skill-addressable, and were excluded from the proposal.
Mutation type: Create (new, non-overlapping skill — no existing skill under .github/skills/ covers restricted-toolset triage for the agent's own behavior).
Validation (held-out, 5 samples spanning Code Scanning Fixer success case, Daily Firewall Logs Collector success, PR Sous Chef unrestricted-toolset success, Contribution Check restricted-toolset failure, Test Quality Sentinel single-denial-then-correct-pivot success):
Baseline score: 55/100
Candidate score: 82/100
Regressions: none
Confidence: high
The candidate generalizes beyond the training example (different command families: git log/show/status/cat, go build/vet) without disrupting successful restricted-toolset runs or unrestricted-toolset runs, and adds explicit guardrails against mislabeling an intentional scope restriction as a permissions/credentials bug.
Note: Only .github/skills/restricted-tool-triage/SKILL.md was added. The underlying model and all other repository files (workflows, source code, root instructions) are unchanged.
Generated by 🧬 EvoSkill Evolver · auto · 144.7 AIC · ⌖ 5.28 AIC · ⊞ 8.4K · ◷
expires on Aug 22, 2026, 6:48 PM UTC-08:00
Note
This was originally intended as a pull request, but the git push operation failed.
Original error: ERR_VALIDATION: pushSignedCommits: refusing unsigned push for branch 'evoskill/restricted-tool-triage-070c1acb5f1fd869': Signed-commit payload violates file-protection policy (request_review): .github/skills/restricted-tool-triage/SKILL.md
The bundle file is available in the agent artifact in the workflow run linked above.
To create a pull request with the changes:
# Download the artifact from the workflow run
gh run download 31922222282 -n agent -D /tmp/agent-31922222282
# Fetch the bundle into a temporary ref, then update the local branch
git fetch /tmp/agent-31922222282/aw-evoskill-restricted-tool-triage.bundle refs/heads/evoskill/restricted-tool-triage:refs/bundles/create-pr-evoskill-restricted-tool-triage-070c1acb5f1fd869-93a124c3
git update-ref refs/heads/evoskill/restricted-tool-triage-070c1acb5f1fd869 refs/bundles/create-pr-evoskill-restricted-tool-triage-070c1acb5f1fd869-93a124c3
git checkout evoskill/restricted-tool-triage-070c1acb5f1fd869
# Ensure the working tree matches the updated branch
git reset --hard
# Remove the temporary bundle ref
git update-ref -d refs/bundles/create-pr-evoskill-restricted-tool-triage-070c1acb5f1fd869-93a124c3
# Push the branch to origin
git push origin evoskill/restricted-tool-triage-070c1acb5f1fd869
# Create the pull request
gh pr create --title '[evoskill] Add restricted-tool-triage skill for gh-aw agent tool-denial handling' --base main --head evoskill/restricted-tool-triage-070c1acb5f1fd869 --repo github/gh-aw
Caution
Protected files were modified in this change.
This pull request is in
request_reviewmode and requires explicit human scrutiny before merge.Protected files:
.github/skills/restricted-tool-triage/SKILL.mdEvoSkill: New skill
restricted-tool-triageDiagnosed capability gap: Agents running gh-aw workflows with a narrow
tools:/bash allowlist have no procedural guidance for handling tool-denial events. When denied, they tend to retry variants of the same disallowed command until hitting the harness's hard denial-count threshold (commonly 3), which aborts the entire session. They also sometimes misreport an intentionally scoped-down toolset as a credentials/infra problem viamissing_tool.Training evidence: Code Scanning Fixer run 31919288964 — workflow restricts
tools.bashto["git diff:*", "git restore:*", wc]. Agent triedsed -n ... file.js(denied),read(/tmp/.../file.js)(denied), thengit status --short\ngit diff --stat(denied, since onlygit diff/git restoreare allowed) — hit 3/3 denials,guard.tool_denials_exceededfired, session aborted. The agent'smissing_toolcall then blamed "token scopes, repository permissions" — misdiagnosing an intentional least-privilege config as an infra bug.Three other failures reviewed in the same training window (
Smoke DeepSeek Harnessnpm ETARGET version-pinning error,Agentic Commandsroute job "Argument list too long" argv-size bug,Design Decision Gatemissingworkflow_dispatchpr_numberinput) were determined to be infrastructure/code bugs, not skill-addressable, and were excluded from the proposal.Mutation type: Create (new, non-overlapping skill — no existing skill under
.github/skills/covers restricted-toolset triage for the agent's own behavior).Validation (held-out, 5 samples spanning Code Scanning Fixer success case, Daily Firewall Logs Collector success, PR Sous Chef unrestricted-toolset success, Contribution Check restricted-toolset failure, Test Quality Sentinel single-denial-then-correct-pivot success):
The candidate generalizes beyond the training example (different command families: git log/show/status/cat, go build/vet) without disrupting successful restricted-toolset runs or unrestricted-toolset runs, and adds explicit guardrails against mislabeling an intentional scope restriction as a permissions/credentials bug.
Note: Only
.github/skills/restricted-tool-triage/SKILL.mdwas added. The underlying model and all other repository files (workflows, source code, root instructions) are unchanged.Note
This was originally intended as a pull request, but the git push operation failed.
Original error: ERR_VALIDATION: pushSignedCommits: refusing unsigned push for branch 'evoskill/restricted-tool-triage-070c1acb5f1fd869': Signed-commit payload violates file-protection policy (request_review): .github/skills/restricted-tool-triage/SKILL.md
Workflow Run: View run details and download bundle artifact
The bundle file is available in the
agentartifact in the workflow run linked above.To create a pull request with the changes: