Skip to content

Design: principled policy that learns from people (proposal) - #606

Draft
LoopGlitch26 wants to merge 5 commits into
mainfrom
agent/principled-policy
Draft

LoopGlitch26 wants to merge 5 commits into
mainfrom
agent/principled-policy

Conversation

@LoopGlitch26

Copy link
Copy Markdown
Member

A design note, no code: docs/principled-policy.md. It proposes how Tracekit's policy moves from matching how a command is spelled to understanding what it does, and how it learns from a person's own decisions and never from the agent's.

Why

On the owner's real week (39,890 calls), today's dev pack stops 207 calls the study labelled harmless, about 30 a day. #605's replay showed every rule that stops anything protects something that matters; the harmless stops come from how those rules read a command. #598 relaxed by spelling and its reviewers kept finding spellings that leaked (nine of ten rounds), so TK-D002, TK-A046 and TK-D008 went back to main's behaviour. Spellings are the wrong unit.

The design in short

  1. For each call the engine derives its effects from the parse it already makes: reads, writes, network in and out, what runs and where its code came from, what it touches, and what it couldn't tell.
  2. Nine written principles judge the effects (P0 can't tell, P1 network code, P2 guardrails, P3 the signer's keys and data, P4 the record, P5 credentials, P6 loss, P7 shared and outside, P8 boundaries). Verdicts stay today's.
  3. Today's rules stay the floor. A principle releases a rule's stop only when it proves the call clear, keeps it when it can't tell, and can stop what a rule missed.
  4. "Provably" is three allowlists: the Python code (an ast allowlist, not a denylist), the invocation (no -i, -X, -m, no expansion in the code), and the line (no PYTHON*, PATH, BASH_ENV…; files a downloading line writes have plainly-data names).
  5. A guard re-reads today's TK-D002 and TK-A046 patterns with the proven commands blanked, so a download spelled for later or elsewhere (crontab, .bashrc, ssm send-command, kubectl exec) and exec(urlopen(…)) still stop.
  6. Only a person's signed approval or rejection teaches. Held calls group into shapes (effects with exact programs, code hashes, hosts and unknowns; only data positions abstracted). Three approvals in two runs, none rejected, proposes a shape.
  7. Learned shapes release only asks, and only bounded can't-tell (and, in system mode, named credentials and exact loss targets). Never denies, network code, guardrails, the signer, the record or money.
  8. Adoption goes through the existing approval flow and a gate: no spelling in the attack corpora may go from stopped to allowed. It is recorded as an ext record and changes the policy hash. Shapes expire (90 days; 30 in dev mode) and can be revoked.
  9. In dev mode the signer can't tell the person from the agent: only P0 shapes are learnable, every adoption is recorded self-approved, and the note says plainly what that means.
  10. A model may explain a held command, labelled unverified, only when the person asks, in the person's client. It never decides or teaches.

First slice and its expected effect

P1 for downloads, the data-only Python proof, the guard, a <(…) tightening, and the arithmetic pre-pass skipping a quoted heredoc body that cat copies into a file. Estimated with a scratch prototype (not in this PR) on the week:

Calls Harmless stops today After slice 1
Other projects 1,580 13 3
Tracekit work 38,310 194 181
All 39,890 207 184

The needed stops the dev pack makes all still stop. Against 1,885 stopped spellings (E17's deny corpus, E21's harmful pools, #598's review cases, the policy tests' stopped cases), the prototype releases 10, each a data read of a download (#578's examples and variants) and none an attack. A first cut without the guard released 13 attack spellings; that is why the guard and the curl option-cluster rule are in the design.

Contract and format

Evidence format: none (ids in rule_ids, text in reasons, adoption as ext records). Policy file: principles, proofs, learned sections. Signer RPC: learn_list and learn_revoke, a contract version bump.

Stops while writing this

# What stopped it The call What I did
1 Tracekit deny, TK-D002 a grep -E over a scratch file whose pattern named both downloaders Not retried; dropped that look
2 Tracekit hold, TK-SHELL-PARSE a sed -n over a test file with a $((…)) line range Not retried; the note doesn't depend on it

Both are harmless stops of the kind the note is about.

Review

Rounds of an independent adversarial review, briefed with only the note and the code ("try to make the agent teach Tracekit, or slip a dangerous call through a principle"), are below.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant