Skip to content

Policy: hold CI workflow writes that upload secrets (TK-A061) (carries #663) - #781

Merged
LoopGlitch26 merged 18 commits into
mainfrom
agent/carry-pr663
Nov 13, 2026
Merged

LoopGlitch26 merged 18 commits into
mainfrom
agent/carry-pr663

Conversation

@LoopGlitch26

Copy link
Copy Markdown
Member

Closes #405. Carries #663 by Kausthubh (@kausthubhk): their nine commits are kept exactly as they are (authorship,
sign-offs and SHAs; two of them are merges of main that also hold review fixes, so they aren't cherry-picked apart),
followed by my merges and re-records. Pushing to their branch failed (the repository's Git LFS pre-push hook needs push
access to the fork to verify locks), so it is carried here. The rule, tests and docs are theirs; #663 has the full
write-up.

Stacked on #780 (the carry of #731): this branch merges #780's branch so its E17 and E21 results and quoted counts
are recorded with both PRs' rules. Merge #780 first; this PR's base then moves to main.

What and why

TK-A061 (coding pack, so dev, server and strict too) asks before a direct Write or Edit of a recognized CI file
(GitHub workflows and composite actions, GitLab CI, CircleCI, Azure Pipelines, Jenkinsfile) adds a curl/wget
command that sends a variable or secret expression as request data over HTTP(S), or pipes echo/printf/printenv
into an upload from stdin. The signer derives ci_workflow from the path and the written text only. An
authentication header, a file upload, a token in the URL and $GITHUB_SHA aren't treated as a leak.

Where the 11-12 review stood

Kausthubh's two commits after the review (f564093, 68f5493) did asks 1-4, 6 and 7: the stale copy in
docs/policy-v2.md is gone, the quoted counts match, e21_robustness.json is re-run, the TK-A049 entry and the
laptop-default and coding ask rows name TK-A061, the speed test runs on RE2 and regex, and the limits row lists the
deploy notice that posts $GITHUB_REF_NAME, $GITHUB_RUN_ID, $CI_COMMIT_SHORT_SHA or $CIRCLE_SHA1 as a known
false hold (the rule isn't widened).

What I changed

  1. Merged main (5d88184) in. policy: accept oversize: deny so a pack can get the deny back #736 has landed since; it touches only the oversize lint and merges cleanly.
  2. The coding pack is 2026.11-v2-85 (ask 5): Coding agents: hooks for CodeBuddy Code (#757) #761 and Fix #749: Policy: deleting or locking a system folder is denied, as deleting root or home is #762 both claim v2-82, Coding agents: Cursor's StrReplace, ApplyPatch and EditNotebook are decided as the edits they are #767 a later one, and Policy: raw Docker or Podman socket calls and cgroup release_agent writes are held (TK-A063 to TK-A066) (carries #731) #780 takes
    v2-84. The final number is settled at merge.
  3. Merged Policy: raw Docker or Podman socket calls and cgroup release_agent writes are held (TK-A063 to TK-A066) (carries #731) #780's branch and re-recorded E17 and E21 on the combined code, with the machine, engine-version and
    latency fields kept as main's: E17 is 859/859 with 1 false alarm in 915; E21's held-out pool is 259/420 on default
    and 286/420 on strict; amandeavor's blind pool is 37 and 40 of 45; strict holds 47 of E17's 915 benign calls
    (5.1%). The shell rewrites stay 2,509, as this PR's cases are file writes. README.md, docs/evaluation.md and
    docs/limits.md quote these counts.

How I ran it as a user

In a sealed container (no network) on this head, tracekit policy lint gives ok for coding, dev, server and strict,
and each pack decides (RE2 and regex agree on every row):

call                                                                           dev     coding  server  strict
Write .github/workflows/ci.yml steps:\n  - run: curl -d "$DEPLOY_TOKEN" https: ask     ask     ask     ask
Write .gitlab-ci.yml script:\n  - echo "$T" | curl --data-binary @- https://x. ask     ask     ask     ask
Write .github/workflows/ci.yml run: curl -H "Authorization: Bearer $PAT" -d '{ allow   allow   allow   allow
Write .github/workflows/ci.yml run: curl -d '{"text":"deployed $GITHUB_REF_NAM ask     ask     ask     ask   (the listed false hold)
Bash DOCKER_HOST=unix:///var/run/docker.sock python x.py                       ask     ask     ask     ask   (#780's rule)
Bash docker ps                                                                 allow   allow   allow   allow

Checklist

  • lint (pyflakes) and tests/test_policy2.py, tests/test_signer_policy.py, tests/test_eval_docs.py,
    tests/test_docs.py, tests/test_e21_policy.py, tests/test_e21_robustness.py: 545 passed, 2 skipped, in a
    sealed Python 3.11 container on this head. A fresh run of the three evals equals the committed files under CI's
    filters. The full suite runs in CI here.
  • a regression test covers the change (tests/test_signer_policy.py, tests/test_policy2.py, E17 corpus cases)
  • user-visible changes are described above (CHANGELOG.md not edited)
  • docs updated: docs/policy-v2.md, docs/limits.md, docs/threat-model-server.md, docs/evaluation.md,
    README.md
  • every commit is signed off

Security invariants

  • the agent still never holds a signing key or assigns sequence numbers
  • nothing an agent-controlled process supplies is newly trusted by the signer or verifier (ci_workflow is
    derived by the signer)
  • gaps are still signed, never silent
  • no verification code is shipped inside a bundle

Evidence format

  • this change does not touch the frozen formats

kausthubhk and others added 17 commits November 7, 2026 16:22
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: Kausthubh <26836237+kausthubhk@users.noreply.github.com>
Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
…ites are held (TK-A063 to TK-A066)

`curl --unix-socket /var/run/docker.sock ...` was only flagged in the dev,
coding and server packs, and `echo ... > /sys/fs/cgroup/x/release_agent` was
allowed in all four packs; both are container escapes (E21's held-out pool).
Every pack extends the coding pack, so four ask rules there reach them all:

- TK-A063: curl, wget, socat, nc, ncat or netcat naming docker.sock or podman.sock
- TK-A064: DOCKER_HOST=unix://... set for a program other than docker, podman,
  docker-compose or podman-compose (by a path too)
- TK-A065: a shell write (redirection, tee) to a file named release_agent or notify_on_release
- TK-A066: the file tools writing a file with either name

`docker ps`, `docker run`, `ls -la /var/run/docker.sock`, `cat` or `grep` of
those files and `DOCKER_HOST=unix://... docker ps` stay allowed. Coding pack
2026.10-v2-79; E17 and E21 re-run so each policy_hash names it, and the
numbers README.md, docs/evaluation.md and docs/limits.md quote follow
(strict's E17 holds 47 of 901: `docker run --rm hello-world` is TK-F004's).
The ids skip TK-A062, which draft PR #605 uses in dev.yaml.

Signed-off-by: Munjikesh N <Munjikesh.N@iiitb.ac.in>
(cherry picked from commit 6a7ba5f)
…CKER_HOST

The command start now also accepts `time`, `!`, `env` given by a path and
option words (`env --`, `env -i`, `time -p`), with a deny case for each.
The socket value and assignment values stop at ; & | and at parentheses
or braces outside one $(...) or ${...}, so the regex engine stays linear
on 64 KiB lines. docs/limits.md says test runners that reach the socket
through DOCKER_HOST are held on purpose, with the remedy, and lists the
forms the rule doesn't read.

Signed-off-by: Munjikesh N <Munjikesh.N@iiitb.ac.in>
(cherry picked from commit a5686ca)
…he deadline

TK-A064's socket and assignment values read \S* past each ';' before
a5686ca, so 64 KiB of ';DOCKER_HOST=unix://' went quadratic on the regex
engine and the call was denied as nondeterministic. The Speed fillers
never built that input.

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
(cherry picked from commit 904faafea2e73b41c7f1cdbd26f878e4bb83a688)
Carries #731 onto main (5d88184): #732 took v2-81 and the open #761 and
#767 claim the next numbers, so this takes v2-84; the final number is
settled at merge. E17 is 853/853 with 1 false alarm in 901; E21's
held-out pool is 259/420 on default and 286/420 on strict, and strict
holds 47 of E17's 901 benign calls. The blind pools don't move. The
held-out table in docs/evaluation.md quotes the new rows; the machine,
engine-version and latency fields stay main's.

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
#761 also took v2-82, and the open #767 and #731 claim the next numbers,
so this takes v2-85; the final number is settled at merge. Only the
policy hashes move: E17 is 846/846 with 1 false alarm in 904, E21's
held-out pool is 255/420 on default and 285/420 on strict, and
amandeavor's blind pool is 37 and 40 of 45. The machine, engine-version
and latency fields stay main's.

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
So this branch's E17 and E21 results are recorded on top of #731's rules.
The coding pack keeps v2-85; both corpora's fs.jsonl keep both sides'
cases; the coding ask row in docs/policy-v2.md has TK-A061 and TK-A063 to
TK-A066. The counts and result files take #731's side here and are
re-recorded in the next commit.

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
E17 is 859/859 with 1 false alarm in 915; E21's held-out pool is 259/420
on default and 286/420 on strict, amandeavor's blind pool 37 and 40 of
45, and strict holds 47 of E17's 915 benign calls (5.1%). The shell
rewrites stay 2,509: this PR's cases are file writes. README.md,
docs/evaluation.md and docs/limits.md quote the new counts. The machine,
engine-version and latency fields stay main's.

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
…starts and pipe sources

The #781 review found forms the rule claimed or should read but didn't:
- a double-quoted value reads a backslash pair as one, and one quoted part may come before the
  part holding the secret, so escaped and spliced JSON bodies are held
- GitHub expressions in any case, with env.X, github.token and toJSON of secrets, github or env;
  shell expansions such as ${VAR:-x}; PowerShell's $env:X in any case
- CircleCI's command: key, GitLab's before_script and after_script, Jenkins' triple-quoted sh
- a command after |, || and &, behind keywords, (, {, !, X=$( and the wrappers sudo, doas, env,
  time, nohup, command, exec, builtin, nice and timeout with their options
- env, set, export and declare dumps and cat of a variable piped to an upload, base64 options
  with digits
- $GITHUB_SHA is trusted only when the written text names GITHUB_SHA no other way

The limits row now lists the wrappers it reads and what it misses. No corpus decision moves
(E17, E21, all packs, both engines); the pattern stays linear on the fallback engine.

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
@LoopGlitch26

Copy link
Copy Markdown
Member Author

This lands in #799 (a merge train of the policy PRs), which renumbers rule ids and re-records the evals once; your commits stay as they are and this PR will show as merged.

@LoopGlitch26
LoopGlitch26 changed the base branch from agent/carry-pr731 to main November 13, 2026 07:00
@LoopGlitch26
LoopGlitch26 merged commit 7e5c31e into main Nov 13, 2026
26 of 34 checks passed
@LoopGlitch26
LoopGlitch26 deleted the agent/carry-pr663 branch November 13, 2026 08:50
tylrcc pushed a commit to tylrcc/Tracekit that referenced this pull request Nov 13, 2026
Rule ids: Cygnux-Labs#780's TK-A063 to TK-A066 become TK-A071 to TK-A074, and Cygnux-Labs#788's
TK-A067 to TK-A070 become TK-A075 to TK-A078 (Cygnux-Labs#778 keeps TK-A063 to
TK-A065 and Cygnux-Labs#787 TK-A067 and TK-A068; TK-A066, TK-A069 and TK-A070 stay
unused, so no id names two rules across the members), in the coding
pack, its tests and docs.

Packs: coding 2026.11-v2-82, strict 2026.11-strict-11 and dev
2026.11-dev-5, one step above main's each. dev maps Claude Code's
Workflow to `harness` (TK-A062 held it as an unmapped tool given a
script), with a benign test.

Harness lists: gptme, OpenHands, Copilot CLI, CodeBuddy Code and Cline
in AGENTS.md, README (badge 39, the --agent list, "another of the 17"),
docs/coding-agents.md's init lines, docs/doctor.md, docs/integrations.md
and docs/support.md; gptme's paths in TK-F009's comment.

Docs: a limits row for Cygnux-Labs#788's known gaps; two bare | in Cygnux-Labs#781's limits
row escaped. The plugin's SKILL.md is tracekit-skills 4bda57a.

Evals re-recorded once: E17 (901 harmful stopped, 1 false alarm in 944),
E21 policy robustness (held-out 279 / 420 default, 305 / 420 strict;
latency rows kept as main's) and E21 robustness (901 originals and
2,701 rewrites contained).

Signed-off-by: LoopGlitch26 <53336715+LoopGlitch26@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Policy: deleting CI workflows, or writing one that sends a secret out, waits for a person

3 participants