Repository navigation
Add an offline Vercel AI approval/resume example - #514
Conversation
Refs Cygnux-Labs#392 Signed-off-by: Lubai <1174273035@qq.com>
Refs Cygnux-Labs#392 Signed-off-by: Lubai <1174273035@qq.com>
Refs Cygnux-Labs#392 Signed-off-by: Lubai <1174273035@qq.com>
|
Thanks, the example reads well. I approved the workflow runs, so CI now runs the full suite on Linux; that covers the
The |
|
The read-only-home failure you found is fixed in #516, credited to you. It also turned up the same bug in |
Address the two requested changes in Cygnux-Labs#514: use the harmless TK-A001 pretend call and delete private state only after successful run close. Signed-off-by: Lubai <1174273035@qq.com>
|
Thanks — both changes are in a41d3c1.
I reran the real separate-process hold/approval/resume flow: the same approval led to exactly one pretend execution, and the signed bundle verified. Build, lint and docs checks passed; TypeScript tests were 148 passed / 10 existing skips / 0 failed. Independent automated review found no unresolved findings (not human review). The new head's workflows have now started and DCO has passed; I have left this draft pending the remaining checks, as requested. The earlier local full-suite limitation remains documented; I am not treating an older head's CI as validation of this revision. Thank you for addressing the read-only-home bug in #516. |
|
Merged, thanks. Two small follow-ups from review I'll make myself: the usage line will suggest a state file in your own directory rather than |
…t does (#516) * client: reading the config no longer creates the client folder (#515) client_config() took its path from client_dir(), which runs makedirs, so a read (otlp.Mapper.plan asking for signer_isolation) created the folder under the home directory and failed where the home folder is read-only. Found by @sunxiuguo while validating #514. * doctor, report, view, analyze: reading the dev store creates nothing (#515) doctor, report, view --dev and analyze --v2 took the dev data dir from the signer's dev_data_dir(), which creates it. The plugin's hook blocks the calls it can't record once that folder exists, so running doctor before setup turned the plugin from warning to blocking. They now read the path from autospawn.data_dir(); the signer, tracekit up and init still make it. - A missing data dir says a signer makes it when it first starts (view and report used to say "start the signer once"). - D-DATA-FS reads the file system of the dir's nearest existing parent. - tests/conftest.py fails a test that makes the real home's client folder or dev data dir; it is silent where they already exist. * tests: conftest loads without the v2 extra (#515) The release dry run installs core only, so importing the dev signer's autospawn (which needs rfc8785) broke conftest there. Without the v2 extra no dev data dir can be made, so the guard watches only the client folder. * Readers of the dev store make nothing, and say to run tracekit up; the guard covers Windows (#515) Independent review of #516: - tracekit last, export --v2, signer vkey/trust/reveal --dev (signer_config's dev branch) still made the dev data dir. They, report, view --dev and analyze --v2 now read it with dev_data_dir(create=False): a FileNotFoundError that names tracekit up while no dev signer has made it. last says there are no runs yet, as before. - An explicit --data-dir that isn't there keeps the plain message. - On Windows the dev data dir follows LOCALAPPDATA: the tests that isolate XDG_DATA_HOME now isolate LOCALAPPDATA too (NoHooksChosen made the real one on CI). - The guard names the variable each folder follows, and a folder made outside any test (setUpClass) fails the session at its end. - Regression tests for view --dev, analyze --v2 and last. * tests: the session runs on a runtime dir of its own (#515) #516's Windows CI: HoldLine wrote the hook's session state to the real runtime dir, which on Windows is inside the dev data dir, and the guard caught it. Eight test classes used the per-user runtime dir without setting one: on a developer's machine, their own dev signer's, so a test could record into its store. conftest now sets TRACEKIT_RUNTIME_DIR to a private temp dir for the session (a shell's own value is replaced too), stops a dev signer left running there at the end, and removes it. Tests that set their own keep it. * tests: a dev signer left hung in the session's runtime dir fails the session, not the last test (#515) Re-review of #516: a Hung from the session-end down() surfaced as an error at teardown of whichever test ran last, and left the dir behind. It now fails the session at its end, naming the dir; the dir is removed either way, and so are its side dirs (autospawn._side) and any signer in them.
|
Thanks again for contributing to Tracekit. A request: if you use a coding agent (Claude Code, Codex, Cursor, Gemini CLI or OpenCode), would you run Tracekit on it for a week and tell us how it went? After a week, |
Current status — 2026-11-02 07:39 UTC
Ready for review after the requested revision in
a41d3c185a6981af1722e05c2ccd7c1acf1c0ad5. The example now holds the harmless pretendterraform destroycall underTK-A001and deletes its private state file after successful close. Actual separate-process and failure-path validation is described in the revision report.On this exact head, CI, DCO and Supply chain all completed successfully (33 successful checks, 1 skipped coverage check). This supplies the remote full-suite evidence requested by the maintainer; it does not change the historical local read-only-HOME failure below.
The original submission transcript and checklist below are preserved as historical evidence for
8ccec7fe…, not as the current source/output or draft status.What and why
Fixes #392.
Adds an offline
MockLanguageModelV4example that saves a held run and framework messages to a private JSON file, prints the signer's approval command, and resumes the same call in another process usingRunHandle,tracekitAIandapprovalResponses. It closes the run after handling the decision. The README Exit bullet and limits reattach row link to it.The example contains assertions for the hold and execution counts. No SDK, dependency, policy or evidence-format changes are included.
Implementation was AI-assisted and received independent automated review. Its status-message finding was fixed; final review found no unresolved findings. This was automated review, not human review.
At initial submission this was a draft because full-suite validation was incomplete, as described below; see the current status above.
How I ran it as a user
Linux x64, Python 3.12.14, Node 24.19.0, npm 11.9.0 and
ai7.0.159. Each process used the same scratchTRACEKIT_RUNTIME_DIRandXDG_DATA_HOME; HOME was unchanged. The first and resumed Node processes had distinct PIDs (15542 and 15572), with approval in a third process (15568). All exited 0.The transcript below uses redacted aliases
RUN_A,APPROVAL_Aand[scratch]; these are not literal values to paste. No run tokens or signing material are included.After
npm run build, fromsdk/typescript:A separate process applied the normal local approval for this synthetic fixture:
The resumed invocation:
Verification:
The signed bundle contains exactly one approval request, answer, consumption, tool result, close and finalization, with the same approval ID throughout. Pending retries and completed-state replay executed nothing; verification after replay remained VERIFIED. Rejection separately executed nothing, closed the run and verified.
The state file was created exclusively with mode 0600. No run token appeared in command output. Synthetic private state and signer keys were cleaned up.
Validation on the reviewed tree:
npm run build: passed.make test-ts: 148 passed, 10 existing contract skips, 0 failed; unchanged from baseline.tests/test_docs.py: 7 passed, 12 subtests passed.make lint build: passed, including wheel/sdist and Twine checks.Full-suite limitation: before implementation,
PYTHONPATH=$PWD make checkwith the local venv passed lint but encountered:The existing mapping test reaches legacy
client_dir(), which creates its default directory; the session fixture removesTRACEKIT_CLIENT_HOME. A focused reproduction reported 1 failed and 2 passed.After confirming that failure, the full run was manually interrupted with SIGINT—not stopped by
-xor maxfail:The remaining suite was not validated, and the build stage of that
make checkrun was not reached. Standalone lint/build/Twine later passed. No HOME reassignment, test modification or denied-path retry was used. The observed Python skip was Agent Framework's MCP 1.x requirement against installed MCP 2.x; root/Linux coverage beyond the interrupted run remains unproven.Tests ran on the
aa41d057…-based reviewed tree, which exactly matches this branch's published tree. Upstream subsequently merged #508 and #504. Their immutable delta was inspected: contribution requirements and the example's SDK/approval/parser paths are unchanged, with nonoverlapping limits edits. The integrated new-base result has not been rerun.Checklist
make checkpasses (lint, tests, build);make test-tstoo ifsdk/typescript/changed — TypeScript passes; fullmake checkremains incomplete as disclosed.CHANGELOG.md)docs/threat-model-laptop.md,docs/threat-model-server.md) — example links added; security claims unchanged.git commit -s, seeCONTRIBUTING.md)Security invariants
Evidence format
core.sig_message,ledger.make_record, the v1 schema, how the verifier treats existing bundles) or the published v2 formatdocs/sample/andtests/fixtures/) still verify, and the change is called out above — not applicable.