Skip to content

Coding agents: a plugin for gptme #574

Closed

Description

@LoopGlitch26

gptme plugins can deny a tool call before it runs (hooks, plugins). A TOOL_CONFIRM hook gets (tool_use, preview, workspace) and returns ConfirmationResult.skip(message) to deny the call (the message is the reason), confirm() to run it without asking, edit(content) to run changed content, or None to leave it to the next hook (hooks docs L177-209). Hooks run by priority, highest first, and a guardrail must out-rank the built-ins, the highest of which is guardrails at 200 (L211-235). --no-confirm/-y only drops gptme's interactive confirmation hooks: "A plugin-registered TOOL_CONFIRM hook is still called" (L236-241). A TOOL_CONFIRM hook that raises skips the call ("On error, skip to be safe", confirm.py), and gptme waits for it with no time limit. Other hooks' exceptions are caught and logged. TOOL_EXECUTE_PRE and TOOL_EXECUTE_POST run before and after every tool; the post hook gets log, workspace, tool_use and the result messages, result_msgs (types.py, tools/base.py). SESSION_START and SESSION_END mark the session. A ToolUse has tool, args, content, kwargs and a call_id that can be None (base.py L919-926). Plugins are packages found in the [plugins] paths of ~/.config/gptme/config.toml or a project's gptme.toml (for example ~/.config/gptme/plugins), narrowed by enabled = [...] when set; gptme calls each hooks/ module's register() at startup (plugins docs L10-60, L143-163). The tools include shell, save, append, patch, read, ipython and browser, plus MCP tools. gptme has 4.5k GitHub stars; v0.36.0 shipped on 2 Nov 2026.

Things to design around:

  • Not every tool asks. TOOL_CONFIRM runs only for tools that call gptme's confirmation: shell, save, append, patch and MCP tools do (execute_with_confirmation), and ipython calls get_confirmation. read doesn't ("Built-in read skips execute_with_confirmation() (it is read_only)", read.py), and browser doesn't either. A plugin can't block those calls; list them in the docs as not decided.
  • A person can still edit an allowed call. Returning None keeps gptme's own prompt (cli_confirm), where a person can edit the call, so what runs may not be what Tracekit decided. Returning confirm() skips that prompt. Pick one and say why. With None, the post-tool event must record whether the content that ran is the content decided, as the Pydantic AI integration does ("the call's arguments changed after the decision").
  • Priority. Register above 200; the docs' own example uses 1000. Hooks sort by priority, then name, both descending.
  • gptme and Tracekit live in different Python environments (gptme is usually installed with pipx or uv tool). So don't import tracekit in the plugin. Like OpenCode's (_opencode_plugin in tracekit/agent_hooks.py), make it a small generated plugin that uses only the standard library. It spawns python -I -m tracekit.integrations.harness_hooks gptme <event> with a Claude-shaped payload, skips the call unless the hook exits 0, and kills a hook still running after 600 s and skips. A plugin whose hook module fails to import is logged, and gptme runs without it (plugins/__init__.py), so doctor checks gptme loads it.
  • A call may have no id. Give each call an id of the plugin's own, kept with its ToolUse between TOOL_CONFIRM and TOOL_EXECUTE_POST, as crewai.py does. If gptme runs the post hook for a skipped call, it must not record that call as run.

Task:

  1. In a real gptme session, with a stub plugin that logs TOOL_CONFIRM, TOOL_EXECUTE_PRE/POST, SESSION_START/END and TURN_PRE, record (redact anything private) the ToolUse of shell, save, append, patch, ipython, read and an MCP call, in the markdown tool format and with native tool calls, as cases in tests/data/harness/gptme.json. Say here: which tools reach TOOL_CONFIRM; when call_id is set; whether the post hook runs for a skipped call; what its tool_use holds after a person edits a call in gptme's prompt; how MCP tools are named; whether ~/.config/gptme/plugins is searched with no [plugins] paths set; whether a project's [plugins] enabled can leave out a user's plugin; and whether TURN_PRE gives the typed prompt.
  2. Add gptme to AGENT_NAMES. tool(): shell → Bash (command from content, or the command kwarg), save → Write, append and patch → Edit (path: the first arg, or the path kwarg), read → Read, MCP tools as mcp__<server>__<tool>. Decide how ipython is decided: Tracekit's shell rules don't read Python, and holding it for a person is the safe default (as Coding agents: an extension for Pi (earendil-works/pi) #564 does for Pi's powershell). normalise() reads the payload the plugin sends, prompts included if step 1 finds them.
  3. Generate ~/.config/gptme/plugins/tracekit/ (__init__.py, and hooks/__init__.py with register()) with a marker line, refusing to overwrite a plugin Tracekit didn't write, as _install_opencode does. If ~/.config/gptme/config.toml has no [plugins] table, append one with that path, after a backup; if it has one without the path, change nothing and print the line to add (lean: until someone needs a merge). Wire tracekit init --dev [--v2] --agent gptme / uninstall / doctor.
  4. Tests: in tests/test_agent_hooks.py (the mapping in both formats; the generated plugin's TOOL_CONFIRM against a stub hook command: exit 0 lets the call through, a block, a crash or a kill skips it; the installer's refusal) and the recorded cases through cases("gptme") in tests/test_harness_hooks_v2.py.
  5. Add ~/.config/gptme and gptme.toml to TK-F009/TK-F015 in tracekit/policy2/packs/coding.yaml, a column in docs/coding-agents.md, and docs/limits.md rows: tools that never reach TOOL_CONFIRM (read, browser, and any step 1 finds) aren't decided; a plugin gptme can't load lets it run without Tracekit; how ipython is decided.

Done when: in a real gptme session (say which version and tool format in the PR) a denied shell call doesn't run and the model sees the reason, with and without -y; a held call waits for tracekit approvals approve; and the session's calls show up in tracekit last; the tests pass and make test passes.

Comment here to claim it. Commits need a DCO sign-off (git commit -s).

Activity

  1. souravC01 commented on Nov 4, 2026

    @souravC01
    Contributor

    Hey @LoopGlitch26, I'd like to take this.

    I looked through the issue and checked the OpenCode pattern in tracekit/agent_hooks.py. Using a small stdlib-only plugin makes total sense here since gptme runs in its own Python env via pipx or uv.

    I'll start with step 1. I can spin up a local session on gptme v0.36.0 with a stub hook, log the exact ToolUse payloads for shell, save, patch, and MCP calls, and record them as test fixtures. After that, I'll wire up the TOOL_CONFIRM hook (setting priority 1000 to out-rank the built-in 200 guardrail), the normalization mappings, and unit tests in tests/test_agent_hooks.py.

    Could you assign #574 to me?

  2. LoopGlitch26 commented on Nov 4, 2026

    @LoopGlitch26
    MemberAuthor

    It's yours, @souravC01. Step 1 first: record the real TOOL_CONFIRM and TOOL_EXECUTE_PRE/POST payloads from a gptme session, and say the gptme version and tool format in the PR. Commits need a DCO sign-off (git commit -s).

  3. LoopGlitch26 commented on Nov 6, 2026

    @LoopGlitch26
    MemberAuthor

    @souravC01 are you still on this? If you're busy, say so and it opens up for someone else.

  4. souravC01 commented on Nov 7, 2026

    @souravC01
    Contributor

    Hey @LoopGlitch26 thanks for checking in. Yes, I’m still working on the gptme integration. If it’s urgent, feel free to take it over or open it up for someone else to pick up. I’m happy to share what I’ve looked into so far.

  5. LoopGlitch26 commented on Nov 7, 2026

    @LoopGlitch26
    MemberAuthor

    Thanks, good to hear. Ask here if anything about the hook format is unclear.

  6. souravC01 commented on Nov 12, 2026

    @souravC01
    Contributor

    I opened #778. I captured the hooks from gptme v0.36.0 with a stub plugin and included the redacted Markdown and native tool calls in gptme.json.

    • I saw shell, save, append, patch, IPython, and MCP reach TOOL_CONFIRM. Read and browser bypass confirmation. Markdown didn't set call_id, and native calls carried the provider's ID. MCP tools came through as server.tool.
    • The post hook fired even for a skipped call. When I edited a save call in gptme's prompt, it wrote the edited text, but the post hook still had the original ToolUse. That's why the bridge returns confirm() on allow.
    • gptme didn't search ~/.config/gptme/plugins until I configured a plugin path. A project allowlist could exclude the user plugin if neither allowlist named it. I got the typed prompt from the session log at TURN_PRE.

    I also ran the generated plugin against a real Tracekit v2 dev signer in WSL2. The model got the TK-D001 denial reason with and without -y. The denied command didn't run, the held command ran once after tracekit approvals approve <id>, and tracekit last verified all three runs.

    On Python 3.12, 5,061 tests passed and 476 v2 bundles verified. E17 stopped 100% of deny cases. E21 found 0 regressions across 840 cases. The regenerated policy results matched the committed results under CI's comparison rules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions