Wire and verify automatic capture hooks on Codex CLI (currently skills-only) #47

Activity

1. jsirish

   added 2 commits that reference this issue on Aug 23, 2026

   fix: drop unverified Codex hook wiring, keep skills-only delivery

   ...

   26eb42b

   feat: Codex CLI plugin as a 4th delivery format (skills only)

   ...

   4ee00f3
2. jsirish commented on Aug 25, 2026

   

   jsirish

   on Aug 25, 2026

   MemberAuthor

   More actions

   Empirical findings (2026-08-25, codex-cli 0.149.0)

   Ran a live, throwaway probe hook (never throughline's real scripts) through real codex exec sessions — both the primary ChatGPT-authenticated account and a separate Qwen/Bailian-backed profile once the primary hit its monthly usage limit mid-investigation. Full methodology: a probe plugin installed via codex plugin marketplace add <local-path> → codex plugin add, with a hooks.json that appends a timestamped, env-dumping line to a file outside the sandbox on every fire — so firing counts and env vars are directly observable, not inferred from trace logs (which turned out to be misleading — see below).

   What actually works

   1. Discovery requires an explicit manifest declaration. A bare root-level hooks.json (convention only, no plugin.json key) is never discovered — confirmed with --enable plugin_hooks --dangerously-bypass-hook-trust and no probe firing. The only working path: "hooks": "./relative/path.json" in .codex-plugin/plugin.json.

   2. validate_plugin.py's rejection of the hooks key does not apply to codex plugin add. Installed cleanly with the key present (v0.0.2 and v0.0.3 of the probe plugin). Whatever validator rejects it (used by gh skill publish --dry-run?) is a different code path from the live installer. This reverses the previous finding.

   3. CLAUDE_PLUGIN_ROOT and PLUGIN_ROOT are both set for hook processes, pointing at the installed plugin cache (~/.codex/plugins/cache/<marketplace>/<plugin>/<version>/). CLAUDE_PLUGIN_DATA/PLUGIN_DATA are also set, pointing at a plugin-scoped data dir Codex provides (unused today). This confirms the best-case outcome flagged in the original issue: hooks/_lib.sh's existing fallback (${CLAUDE_PLUGIN_ROOT:-$DIR/..}) works with zero modification if .codex-plugin/plugin.json declares "hooks": "./hooks/hooks.json" — reusing the exact file Claude Code uses, no duplicate config, no drift risk.

   4. No double-registration in this exact layout. With .codex-plugin/plugin.json pointing "hooks" at ./hooks/hooks.json (mirroring this repo's real structure — a hooks/ directory containing hooks.json, no root-level file), each event fired exactly once. The "loading hooks from both <path> and <path>/" collision diagnostic never appeared. That risk is specific to ambiguous convention-based auto-discovery (which doesn't even work per throughline v0.2.0: robustness, secret redaction, compaction-survival #1), not to an explicit single declaration — so session-capture.sh's missing idempotency guard is not a live risk for this recommended pattern.

   5. Event coverage, confirmed by direct firing (not just wire-schema strings): SessionStart ✓, UserPromptSubmit ✓, PostToolUse ✓ (fired once on an actual Bash-tool-call turn), SessionEnd ✓ (fired once — this reverses my own binary-string search, which only found SessionStart/PostToolUse/UserPromptSubmit embedded; SessionEnd fires despite that). PreCompact remains unconfirmed — no compaction was reachable in a short exec session and no wire-schema string was found for it; treat as unsupported until someone can actually trigger compaction under Codex.

      Correction to the JSON payload claim above: the "hook: X" / "hook: X Completed" trace lines in --print-logs/RUST_LOG=debug output are boilerplate lifecycle logging that fires regardless of whether any hook config exists — confirmed by uninstalling the probe plugin entirely and seeing the same trace lines. Only the probe's own append-to-file side effect is real evidence of firing; don't trust the trace lines alone in any future investigation here.

   The actual blocker: a two-part opt-in gate, not a discovery problem

   1. plugin_hooks is a feature flag, off by default. Without --enable plugin_hooks (or a persisted features.plugin_hooks = true), hooks never fire — not even the boilerplate trace lines.
   2. Even with the feature enabled, hooks require "persisted hook trust." In a non-interactive codex exec session there is no way to grant it — it silently no-ops with zero diagnostic output. The only way to make it fire in exec mode is --dangerously-bypass-hook-trust, which the CLI's own help text scopes to "automation that already vets hook sources" — not something to bake into a plugin's shipped default or ask ordinary users to pass routinely.
   3. Reverse-engineered from strings on the binary: trust is granted through an interactive TUI review flow only (hooks/list is a real app-server JSON-RPC method; trust is persisted as a trusted_hash in a hooks.state file, written via config/batchWrite, and invalidated on account change). There's no codex plugin trust CLI subcommand and no config.toml key to pre-grant it. I did not verify this interactive flow actually works end-to-end — driving the real TUI requires a PTY this investigation didn't have, and the underlying app-server JSON-RPC protocol is explicitly [experimental], undocumented, not something to build a shipped feature's trust story on even if reverse-engineered.

   Decision

   Not wiring Codex hooks. Discovery is solved and double-registration is ruled out for the recommended pattern — the plan's original decision gate is technically satisfied — but the newly-found trust gate means "automatic capture" would not actually be automatic for any realistic usage: headless/CI use needs the dangerous bypass flag, and interactive use needs an unverified manual trust step per install. That undermines the entire point of throughline's hooks (zero-action continuous capture). Shipping the hooks key now would create a feature that appears wired but silently does nothing for the vast majority of real sessions — worse than the current honest "skills only" framing.

   If revisited later

   • Confirm the interactive TUI trust flow actually works (needs a PTY-driving test, not attempted here).
   • If Codex ever exposes a non-interactive trust grant (CLI flag, config key, or a stable non-experimental app-server method), the wiring itself is now known-good: .codex-plugin/plugin.json + "hooks": "./hooks/hooks.json", zero changes to hooks/hooks.json or hooks/_lib.sh needed.
   • PreCompact needs a real long-context test to confirm one way or the other.
   • Re-test after plugin_hooks graduates out of feature-flag status, if it does — 0.149.0 was current as of this investigation, 0.149.1 was already available but untested.

   Closing with these findings; skills-only remains correct and is now backed by verified evidence rather than an inconclusive prior attempt.
3. jsirish commented on Aug 25, 2026

   

   jsirish

   on Aug 25, 2026

   MemberAuthor

   More actions

   Closing per the investigation above — skills-only remains correct, now with empirical evidence. Reopen if Codex ever exposes a non-interactive hook-trust grant.
4. jsirish

   closed this as completedon Aug 25, 2026
5. jsirish

   added 2 commits that reference this issue on Aug 25, 2026

   feat(#47): validate Codex/agents manifests in CI, close hooks investi…

   ...

   c70da10

   docs(#45): redesign promo site, restructure README for all 4 harnesses

   ...

   9b581a7