Repository navigation
Policy: hold CI workflow writes that upload secrets (TK-A061) (carries #663) - #781
Merged
Merged
Conversation
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
…ites are held (TK-A063 to TK-A066) `curl --unix-socket /var/run/docker.sock ...` was only flagged in the dev, coding and server packs, and `echo ... > /sys/fs/cgroup/x/release_agent` was allowed in all four packs; both are container escapes (E21's held-out pool). Every pack extends the coding pack, so four ask rules there reach them all: - TK-A063: curl, wget, socat, nc, ncat or netcat naming docker.sock or podman.sock - TK-A064: DOCKER_HOST=unix://... set for a program other than docker, podman, docker-compose or podman-compose (by a path too) - TK-A065: a shell write (redirection, tee) to a file named release_agent or notify_on_release - TK-A066: the file tools writing a file with either name `docker ps`, `docker run`, `ls -la /var/run/docker.sock`, `cat` or `grep` of those files and `DOCKER_HOST=unix://... docker ps` stay allowed. Coding pack 2026.10-v2-79; E17 and E21 re-run so each policy_hash names it, and the numbers README.md, docs/evaluation.md and docs/limits.md quote follow (strict's E17 holds 47 of 901: `docker run --rm hello-world` is TK-F004's). The ids skip TK-A062, which draft PR #605 uses in dev.yaml. Signed-off-by: Munjikesh N <Munjikesh.N@iiitb.ac.in> (cherry picked from commit 6a7ba5f)
…CKER_HOST
The command start now also accepts `time`, `!`, `env` given by a path and
option words (`env --`, `env -i`, `time -p`), with a deny case for each.
The socket value and assignment values stop at ; & | and at parentheses
or braces outside one $(...) or ${...}, so the regex engine stays linear
on 64 KiB lines. docs/limits.md says test runners that reach the socket
through DOCKER_HOST are held on purpose, with the remedy, and lists the
forms the rule doesn't read.
Signed-off-by: Munjikesh N <Munjikesh.N@iiitb.ac.in>
(cherry picked from commit a5686ca)
…he deadline TK-A064's socket and assignment values read \S* past each ';' before a5686ca, so 64 KiB of ';DOCKER_HOST=unix://' went quadratic on the regex engine and the call was denied as nondeterministic. The Speed fillers never built that input. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com> (cherry picked from commit 904faafea2e73b41c7f1cdbd26f878e4bb83a688)
Carries #731 onto main (5d88184): #732 took v2-81 and the open #761 and #767 claim the next numbers, so this takes v2-84; the final number is settled at merge. E17 is 853/853 with 1 false alarm in 901; E21's held-out pool is 259/420 on default and 286/420 on strict, and strict holds 47 of E17's 901 benign calls. The blind pools don't move. The held-out table in docs/evaluation.md quotes the new rows; the machine, engine-version and latency fields stay main's. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
#761 also took v2-82, and the open #767 and #731 claim the next numbers, so this takes v2-85; the final number is settled at merge. Only the policy hashes move: E17 is 846/846 with 1 false alarm in 904, E21's held-out pool is 255/420 on default and 285/420 on strict, and amandeavor's blind pool is 37 and 40 of 45. The machine, engine-version and latency fields stay main's. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
So this branch's E17 and E21 results are recorded on top of #731's rules. The coding pack keeps v2-85; both corpora's fs.jsonl keep both sides' cases; the coding ask row in docs/policy-v2.md has TK-A061 and TK-A063 to TK-A066. The counts and result files take #731's side here and are re-recorded in the next commit. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
E17 is 859/859 with 1 false alarm in 915; E21's held-out pool is 259/420 on default and 286/420 on strict, amandeavor's blind pool 37 and 40 of 45, and strict holds 47 of E17's 915 benign calls (5.1%). The shell rewrites stay 2,509: this PR's cases are file writes. README.md, docs/evaluation.md and docs/limits.md quote the new counts. The machine, engine-version and latency fields stay main's. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
…starts and pipe sources The #781 review found forms the rule claimed or should read but didn't: - a double-quoted value reads a backslash pair as one, and one quoted part may come before the part holding the secret, so escaped and spliced JSON bodies are held - GitHub expressions in any case, with env.X, github.token and toJSON of secrets, github or env; shell expansions such as ${VAR:-x}; PowerShell's $env:X in any case - CircleCI's command: key, GitLab's before_script and after_script, Jenkins' triple-quoted sh - a command after |, || and &, behind keywords, (, {, !, X=$( and the wrappers sudo, doas, env, time, nohup, command, exec, builtin, nice and timeout with their options - env, set, export and declare dumps and cat of a variable piped to an upload, base64 options with digits - $GITHUB_SHA is trusted only when the written text names GITHUB_SHA no other way The limits row now lists the wrappers it reads and what it misses. No corpus decision moves (E17, E21, all packs, both engines); the pattern stays linear on the fallback engine. Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
Member
Author
|
This lands in #799 (a merge train of the policy PRs), which renumbers rule ids and re-records the evals once; your commits stay as they are and this PR will show as merged. |
tylrcc
pushed a commit
to tylrcc/Tracekit
that referenced
this pull request
Nov 13, 2026
Rule ids: Cygnux-Labs#780's TK-A063 to TK-A066 become TK-A071 to TK-A074, and Cygnux-Labs#788's TK-A067 to TK-A070 become TK-A075 to TK-A078 (Cygnux-Labs#778 keeps TK-A063 to TK-A065 and Cygnux-Labs#787 TK-A067 and TK-A068; TK-A066, TK-A069 and TK-A070 stay unused, so no id names two rules across the members), in the coding pack, its tests and docs. Packs: coding 2026.11-v2-82, strict 2026.11-strict-11 and dev 2026.11-dev-5, one step above main's each. dev maps Claude Code's Workflow to `harness` (TK-A062 held it as an unmapped tool given a script), with a benign test. Harness lists: gptme, OpenHands, Copilot CLI, CodeBuddy Code and Cline in AGENTS.md, README (badge 39, the --agent list, "another of the 17"), docs/coding-agents.md's init lines, docs/doctor.md, docs/integrations.md and docs/support.md; gptme's paths in TK-F009's comment. Docs: a limits row for Cygnux-Labs#788's known gaps; two bare | in Cygnux-Labs#781's limits row escaped. The plugin's SKILL.md is tracekit-skills 4bda57a. Evals re-recorded once: E17 (901 harmful stopped, 1 false alarm in 944), E21 policy robustness (held-out 279 / 420 default, 305 / 420 strict; latency rows kept as main's) and E21 robustness (901 originals and 2,701 rewrites contained). Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #405. Carries #663 by Kausthubh (@kausthubhk): their nine commits are kept exactly as they are (authorship,
sign-offs and SHAs; two of them are merges of main that also hold review fixes, so they aren't cherry-picked apart),
followed by my merges and re-records. Pushing to their branch failed (the repository's Git LFS pre-push hook needs push
access to the fork to verify locks), so it is carried here. The rule, tests and docs are theirs; #663 has the full
write-up.
Stacked on #780 (the carry of #731): this branch merges #780's branch so its E17 and E21 results and quoted counts
are recorded with both PRs' rules. Merge #780 first; this PR's base then moves to main.
What and why
TK-A061(coding pack, so dev, server and strict too) asks before a directWriteorEditof a recognized CI file(GitHub workflows and composite actions, GitLab CI, CircleCI, Azure Pipelines, Jenkinsfile) adds a
curl/wgetcommand that sends a variable or secret expression as request data over HTTP(S), or pipes
echo/printf/printenvinto an upload from stdin. The signer derives
ci_workflowfrom the path and the written text only. Anauthentication header, a file upload, a token in the URL and
$GITHUB_SHAaren't treated as a leak.Where the 11-12 review stood
Kausthubh's two commits after the review (f564093, 68f5493) did asks 1-4, 6 and 7: the stale copy in
docs/policy-v2.mdis gone, the quoted counts match,e21_robustness.jsonis re-run, theTK-A049entry and thelaptop-default and coding ask rows name
TK-A061, the speed test runs on RE2 and regex, and the limits row lists thedeploy notice that posts
$GITHUB_REF_NAME,$GITHUB_RUN_ID,$CI_COMMIT_SHORT_SHAor$CIRCLE_SHA1as a knownfalse hold (the rule isn't widened).
What I changed
2026.11-v2-85(ask 5): Coding agents: hooks for CodeBuddy Code (#757) #761 and Fix #749: Policy: deleting or locking a system folder is denied, as deleting root or home is #762 both claimv2-82, Coding agents: Cursor's StrReplace, ApplyPatch and EditNotebook are decided as the edits they are #767 a later one, and Policy: raw Docker or Podman socket calls and cgroup release_agent writes are held (TK-A063 to TK-A066) (carries #731) #780 takesv2-84. The final number is settled at merge.latency fields kept as main's: E17 is 859/859 with 1 false alarm in 915; E21's held-out pool is 259/420 on default
and 286/420 on strict; amandeavor's blind pool is 37 and 40 of 45; strict holds 47 of E17's 915 benign calls
(5.1%). The shell rewrites stay 2,509, as this PR's cases are file writes. README.md,
docs/evaluation.mdanddocs/limits.mdquote these counts.How I ran it as a user
In a sealed container (no network) on this head,
tracekit policy lintgivesokfor coding, dev, server and strict,and each pack decides (RE2 and regex agree on every row):
Checklist
pyflakes) andtests/test_policy2.py,tests/test_signer_policy.py,tests/test_eval_docs.py,tests/test_docs.py,tests/test_e21_policy.py,tests/test_e21_robustness.py: 545 passed, 2 skipped, in asealed Python 3.11 container on this head. A fresh run of the three evals equals the committed files under CI's
filters. The full suite runs in CI here.
tests/test_signer_policy.py,tests/test_policy2.py, E17 corpus cases)docs/policy-v2.md,docs/limits.md,docs/threat-model-server.md,docs/evaluation.md,README.md
Security invariants
ci_workflowisderived by the signer)
Evidence format