Repository navigation
Hooks: GitHub Copilot CLI support - #710
Conversation
Signed-off-by: Zarif Hossain <hossainzarif2002@gmail.com>
Signed-off-by: Zarif Hossain <hossainzarif2002@gmail.com>
Signed-off-by: Zarif Hossain <hossainzarif2002@gmail.com>
5342369 to
5c441e2
Compare
LoopGlitch26
left a comment
There was a problem hiding this comment.
Thanks, this is careful work: on the v2 hook an unreadable payload, a missing toolCallId, an odd toolName and a non-string apply_patch are all blocked, the deny is exit 2 plus the JSON reason, names of 64+ characters are denied with a clear message (tested at 63/64/65), and uninstall keeps a person's own hooks. The DCO sign-offs are in now too. The signer change reads right: the - splits add no holds on the shipped packs for Gemini or Crush names, and evaluate() is the right place, since tracekit audit replays it. What's left, most important first:
- The v1 hook lets every call through.
_command(agent_hooks.py:1199) names the event only for v2 and Antigravity, and Copilot's payload doesn't name it, sonormalise(:601) returns None andrunexits 0.tracekit init --dev --agent copilot(offered atdocs/coding-agents.md:18), then anybashcall, runs undecided. The v1 tests passpreToolUseon argv, which the installed command doesn't. Name the event for copilot as for Antigravity (and test the installed command line), or refuse v1 for copilot. - Built-in tools missing from the table (
:101). Copilot's hooks and CLI references also listcreate,edit,str_replace_editor,grep,powershell,write_bash/write_powershellandwrite_agent; with no-in them, each is decided as an unknown tool (flagged, and it runs). For exampleedit {"path":"~/.copilot/hooks/tracekit.json"}andcreate {"path":".git/hooks/pre-commit"}meet no file rule, andbash {"command":"bash","mode":"async"}thenwrite_bash {"input":"…"}reaches the shell unread. Your captures (apply_patch,rg) look like the GPT tool set; with a Claude model Copilot usescreate/edit/grep, so please capture those too. Mapcreate→Write,edit/str_replace_editor→Edit,grep→Grep,powershell→PowerShell, andwrite_*→Bash withcommand= the input, as Codex'swrite_stdinis. - Each tool's own path argument.
rg/globwithpathsas a string (the captured shape,test_agent_hooks.py:48) is decided on no path, since the engine readspathsonly as a list (engine.py:458):rg {"pattern":"x","paths":"~/.gnupg/private-keys-v1.d/a.key"}gets no TK-A020 hold. Andviewwith a decoyfile_pathbesidepathis decided on the decoy. Pin each tool's own argument, as the Crush, goose and Antigravity adapters do. COPILOT_HOME. With it set, Copilot reads$COPILOT_HOME/hooks/, so Tracekit's hook never runs. Handle it inconfig_path(:952) and doctor'sUSER_ENV, asCODEX_HOMEis.- Copilot's own config isn't protected. Add
.copilot/,.github/hooks/and.github/copilot/settings*.jsonto TK-F009/TK-F015 and to the limits row at 174, like every other harness's (disableAllHooks: truein.github/copilot/settings.jsonturns the hook off). Two limits rows: Copilot also runs a repo's.claude/settings*.jsonhooks, so a project-level Tracekit Claude Code hook would decide each call twice and block it (as with Devin); and another hook'smodifiedArgscan change a call after Tracekit decided it (Crush has the same row, 423). --projectwrites.github/hooks/tracekit.jsonwith this machine's Python path..github/is usually committed, and teammates and Copilot's cloud agent would read it: the hook can't start there, so every call is denied, and the local path is published. Warn at init and say how to keep it out of the commit.- Smaller:
:614p["toolResult"]["textResultForLlm"]raises KeyError when there's no result; use.get.- Copilot's docs say hook timeouts fail open: say in the docs that the hook answers within 540 s, under the 600 s it sets.
- Doctor assumes a 60 s default timeout; Copilot's is 30.
- Copilot isn't yet in doctor.md, integrations.md, the
harness_hooksdocstring or the support table (with the minimum version). - Uninstall leaves
{"version":1}in Tracekit's own file, and creates that file when nothing was installed. - The new tests are module-level pytest functions; AGENTS.md asks for unittest classes.
- Missing tests:
toolArgssent as a JSON string, a non-stringtoolName,apply_patchwith object args, a missingtoolResult.
Items 1 and 2 are the ones that let a call run undecided, so they come first. Happy to look again as soon as you push.
Signed-off-by: Zarif Hossain <hossainzarif2002@gmail.com>
LoopGlitch26
left a comment
There was a problem hiding this comment.
Thanks, this covers all seven: the v1 command names its event (and the test runs the installed command line), the tools Copilot gives other models are mapped, each file tool is decided on its own path or paths, COPILOT_HOME is followed in init and doctor, Copilot's config is flagged in TK-F009 and TK-F015 with the limits rows, --project warns, and the smaller items are in, with the tests in a unittest class.
One thing left: the coding pack's rules changed, so its version has to move with them. Your branch has 2026.10-v2-77, and main is at -78 since #690. Please merge main and set version: '2026.10-v2-79'. If another pack change lands first, I'll move the number on your branch so you don't have to.
I'll look again once that's in and CI is green.
Signed-off-by: Zarif Hossain <hossainzarif2002@gmail.com>
Signed-off-by: Zarif Hossain <hossainzarif2002@gmail.com>
Signed-off-by: Zarif Hossain <hossainzarif2002@gmail.com>
| sessions on this machine; nothing is installed and nothing leaves it. | ||
| 2. **With their yes, wire your hooks.** `pip install tracekit-ai`, then `tracekit init --dev --v2`, plus | ||
| `--agent codex`, `cursor`, `gemini` or another of the 12. In Claude Code, the plugin can take init's place (one or | ||
| `--agent codex`, `cursor`, `gemini` or another of the 13. In Claude Code, the plugin can take init's place (one or |
There was a problem hiding this comment.
This line continues list item 2, so it keeps its three-space indent:
| `--agent codex`, `cursor`, `gemini` or another of the 13. In Claude Code, the plugin can take init's place (one or | |
| `--agent codex`, `cursor`, `gemini` or another of the 13. In Claude Code, the plugin can take init's place (one or |
|
Thanks for the README counts. Copilot CLI also belongs in the integration list under the badge, next to the other coding agents, and the badge then counts 35. L156, add L12, the badge and its alt text from 34 to 35: The third change is a one-click suggestion on L241: it keeps the list item's three-space indent. |
|
#605 has merged, so please merge main into this branch. It changed |
Signed-off-by: Zarif Hossain <hossainzarif2002@gmail.com>
Signed-off-by: Zarif Hossain <hossainzarif2002@gmail.com>
|
Thanks for the merge and the README line. CI fails on one test on every runner, Please take main's version of that file and change only the two |
|
I made the last changes myself and carried this in #782, since a push to your branch was refused. Your eleven commits are in it unchanged, with your authorship and sign-offs. On top of them:
This PR closes when #782 merges. |
|
Merged with your commits in #799 (via #782): Tracekit now guards GitHub Copilot CLI, including its typed shell input and its policy folder. It ships in the next release. Thank you. And a follow-up to the earlier request to run Tracekit for a week: if you've been running it on your coding agent, |
Closed #365
What it does
preToolUse,postToolUseandpostToolUseFailure, paired by Copilot's owntoolCallId.~/.copilot/hooks/tracekit.json(--project:.github/hooks/tracekit.json), with a 600 s timeout onpreToolUse. Copilot's default is 30 s.{"permissionDecision": "deny", "permissionDecisionReason": ...}on stdout. Copilot denies on any non-zero exit but shows the reason only from the JSON.bash,apply_patch,view,glob,rg,web_fetch,taskmap toBash,Edit(every file the patch names, throughPATCH_FILE),Read,Glob,Grep,WebFetch,Task.web_searchis decided asmcp:github-mcp-server/web_search.-that either name may contain. Any other name with a-is sent asmcp__<name>. The signer'sevaluatealready tried each_split of such a name for Gemini CLI; it now tries each-split as well, plus the bare tool names and the unknown-tool reading, and the strictest decision stands. One signed decision per call.preMcpToolCall: not read, as agreed on the issue. It runs after the decision, can't block and isn't documented.Tests
mcp__<name>; a hyphenless unknown name stays itself.tracekit-space-get-secret) is still held byTK-A037.toolCallIdis blocked.postToolUseFailurepayload becomes a failed result.initwrites the three events with a 600 s pre-tool timeout, anduninstallkeeps a person's own hooks.Run in a Linux container: the hook, signer-policy, docs, doctor, install and CLI test files passed (590 passed, 10 skipped) before merging
main; after the merge the hook, signer-policy and docs files passed (394 passed, 2 skipped).make lintpasses. I have not run the fullmake test,make buildormake test-ts.Checked in a real run
Copilot CLI 1.0.107, Linux, hooks written by
tracekit init --dev --v2 --agent copilot --project:bashcall and an MCP call ran and were recorded;get_secretcall was held (TK-A037): Copilot showed "Denied by preToolUse hook: [tracekit] HELD, nothing ran …" and the MCP server never received the call;tracekit doctorreported the hooks present, each naming its event, with a 600 s pre-tool timeout.That run was before
mainwas merged in.Limits (added to docs/limits.md)
__is split there alone.Notes for review
commandandtimeoutkeys rather than the documentedbashandtimeoutSec. Copilot 1.0.107 honoured both in a live run (a 40 s hook was not cut at the 30 s default), and it lets the existing installer and doctor code read Copilot's file unchanged. Say if you'd prefer the documented keys.<server>-<tool>; none arrived under a bare name. So table names are not also decided as MCP names.mcp__names. Each extra reading can only make a decision stricter.