Repository navigation
Design: principled policy that learns from people (proposal) - #606
Draft
LoopGlitch26 wants to merge 5 commits into
Draft
LoopGlitch26 wants to merge 5 commits into
LoopGlitch26 wants to merge 5 commits into
Conversation
…, what unknown means, scoreboard
…wnloader config, learning limits, model)
…d off the pipeline, whole-line guard, learning narrowed)
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A design note, no code:
docs/principled-policy.md. It proposes how Tracekit's policy moves from matching how a command is spelled to understanding what it does, and how it learns from a person's own decisions and never from the agent's.Why
On the owner's real week (39,890 calls), today's dev pack stops 207 calls the study labelled harmless, about 30 a day. #605's replay showed every rule that stops anything protects something that matters; the harmless stops come from how those rules read a command. #598 relaxed by spelling and its reviewers kept finding spellings that leaked (nine of ten rounds), so
TK-D002,TK-A046andTK-D008went back to main's behaviour. Spellings are the wrong unit.The design in short
P0can't tell,P1network code,P2guardrails,P3the signer's keys and data,P4the record,P5credentials,P6loss,P7shared and outside,P8boundaries). Verdicts stay today's.astallowlist, not a denylist), the invocation (no-i,-X,-m, no expansion in the code), and the line (noPYTHON*,PATH,BASH_ENV…; files a downloading line writes have plainly-data names).TK-D002andTK-A046patterns with the proven commands blanked, so a download spelled for later or elsewhere (crontab,.bashrc,ssm send-command,kubectl exec) andexec(urlopen(…))still stop.extrecord and changes the policy hash. Shapes expire (90 days; 30 in dev mode) and can be revoked.P0shapes are learnable, every adoption is recorded self-approved, and the note says plainly what that means.First slice and its expected effect
P1for downloads, the data-only Python proof, the guard, a<(…)tightening, and the arithmetic pre-pass skipping a quoted heredoc body thatcatcopies into a file. Estimated with a scratch prototype (not in this PR) on the week:The needed stops the dev pack makes all still stop. Against 1,885 stopped spellings (E17's deny corpus, E21's harmful pools, #598's review cases, the policy tests' stopped cases), the prototype releases 10, each a data read of a download (#578's examples and variants) and none an attack. A first cut without the guard released 13 attack spellings; that is why the guard and the curl option-cluster rule are in the design.
Contract and format
Evidence format: none (ids in
rule_ids, text inreasons, adoption asextrecords). Policy file:principles,proofs,learnedsections. Signer RPC:learn_listandlearn_revoke, a contract version bump.Stops while writing this
TK-D002grep -Eover a scratch file whose pattern named both downloadersTK-SHELL-PARSEsed -nover a test file with a$((…))line rangeBoth are harmless stops of the kind the note is about.
Review
Rounds of an independent adversarial review, briefed with only the note and the code ("try to make the agent teach Tracekit, or slip a dangerous call through a principle"), are below.