Skip to content

Add prompt audit and remove duplicate state context - #275

Merged
hffmnnj merged 14 commits into
mainfrom
feat/prompt-baseline-and-composition
Aug 3, 2026
Merged

hffmnnj merged 14 commits into
mainfrom
feat/prompt-baseline-and-composition

Conversation

@hffmnnj

@hffmnnj hffmnnj commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a reproducible audit for prompt size, token estimates, absolute-language usage, and emphasis markers
  • remove duplicate workflow-state content and repeated database tool descriptions from assembled system instructions
  • add regression coverage for one state block, an always-on context budget, and V1/V2 parity
  • refresh the project knowledge base with current repository facts

Verification

  • 603 scoped tests passed
  • typecheck passed
  • build passed
  • prompt report reproduced the documented byte baselines
  • working tree and diff checks are clean

hffmnnj and others added 14 commits August 2, 2026 19:01
The project knowledge base was stale: it reported 30 tools and 10 hook factories, listed goop_create_pr as an unregistered orphan, and predated the current V1/V2 adapter architecture. Refreshed it against the actual registries (38 tools, 13 hook factories) and the conventions in AGENTS.md, added a prompt-bearing surface inventory for prompt-optimization work, and replaced duplicated inventory detail with links to canonical source files.
Pure module under src/features/prompt-audit/ measuring per-file chars,
bytes, tokens (reusing estimateTokens from system-transform.ts),
absolute-language census (must|never|always|critical|only), and
bold-span counts for agents/, commands/, references/. Includes a thin
import.meta.main report runner and a prompt-report package script.

Not imported from src/index.ts, so the production bundle is unchanged
(dist hash verified identical before/after). Establishes the
reproducible baseline that all later waves compare deltas against.
…l descriptions

Workflow state was injected twice into every system message: once as
<goopspec_state> (system-transform.ts) and again as a markdown
'## CURRENT STATE' block prepended inside buildEnforcementContext
(phase-context.ts). buildEnforcementContext now returns phase
enforcement rules only; buildStateBlock is the single canonical state
source, already carrying the full field union (interviewComplete and
autopilot included, which the markdown variant omitted).

buildDbContextBlock no longer restates four MCP tool descriptions that
already live in their own tool schemas — it now emits the workflow
document inventory plus a concise pointer.

Reduces the always-on injected composition (state + phase rules + db,
excluding the untouched memory/Field Notes budgets) by ~31% chars/tokens
in a representative execute-phase scenario. No field or enforcement
semantics lost; buildStateContext remains as a tested, documented
standalone formatter, unused by the live injection path.
Cover single-state composition, populated context budgets, and V1/V2 parity after runtime deduplication.
…rser

Add phase, requires, and nextStep as optional fields on LoadedCommand so
parseCommandMarkdown preserves them from frontmatter. loadCommandConfigs
continues to register only template/description/agent with the host,
keeping the OpenCode Config.command contract unchanged. Add missing
frontmatter keys to five command docs that the locked contract requires.
Parse the shipped prompt directories through production loaders and fail clearly on roster, metadata, count, or tool-grant drift.
Read all real agent and command prose, resolve both supported reference pointer forms, and retain a diagnostic negative control for dangling targets.
Guard the real prompt stack before later consolidation removes duplicated prose. Assert runtime-owned gates and phase rules through their assemblers, while accepting shared response and boot protocol pointers at the prompt layer.
Add a single Prompt Authoring Rules section to core-protocol.md
covering layer precedence, absolutes-for-invariants, outcome-first
framing, one autonomy policy per role, and the one shared tool-call
batching rule. Offset the addition by removing genuinely duplicated
material from dispatch-patterns.md (restated batching rule, Anti-
Patterns bullets that duplicated Common Mistakes and Single-Branch
Parallelism Rule, a Quick-mode exception restating its own table) and
subagent-identity.md (an Anti-Patterns section that restated Never Do
verbatim). Net effect: references/ total -2 chars / -1 byte vs the
locked baseline, 0 net token change.

Update the Wave 1 prompt-audit baseline test to the new, intentionally
changed references byte count and absolute-hits count, since those
values were the direct and expected result of this consolidation.
Restructure the Conductor prompt outcome-first: one-line boot pointer,
one autonomy policy merging deviation rules and lazy-autopilot hard
stops, a Delegation section naming all six executor tiers with a
pointer to dispatch-patterns.md's Agent Selection tables instead of
duplicating them, a Gate Enforcement pointer to phase-gates.md's Gate
Overview instead of restating it, and a compacted Context Compaction
section pointing to core-protocol.md's DB-as-State Durability
Guarantees for mechanism detail already documented there. Auto-
Delegation and the Research-First Gate keep their exact behavioral
content (planner-blocked-until-research, skip heuristic, mid-wave
research, fallback, sequencing) in prose instead of decorative example
tables.

Every Wave 2 semantic invariant is preserved: Conductor/executor
separation sentence, all six executor tiers named and reachable, the
five workflow phase names, memory-first boot pointer, response-
envelope pointer, and plan-phase source-write prohibition are all
still asserted by real-agent-stack tests.

Measured (chars/bytes/tokens/absolute-hits/bold-spans):
goop-orchestrator.md 17,109/17,207/4,278/26/60 ->
9,734/9,807/2,434/20/26 (-43.1%/-43.0%/-43.1%/-6/-34).
agents/ category: 99,822 -> 92,422 bytes (decrease, per MH6).
agents+commands combined: 125,369 -> 117,969 bytes, now under the
118,798 (-5%) bar against the locked 125,051 baseline.
Workflow-wide absolute-language hits: 396 -> 390.

Updated the prompt-audit real-tree sentinel test to the new agents
byte/bold-span totals and the new absolute-hit total, since these are
the direct, spec-directed result of this consolidation. The immutable
Wave 1 baseline stays recorded in RESEARCH.md.
…hestrator

Verifier found that the Core Identity bullet Never write code paraphrased
the execute phase's runtime MUST-NOT-DO rule (Write code directly - ALWAYS
delegate to subagents, phase-context.ts). Replaced it with a bare pointer
to dispatch-patterns.md's Conductor Identity / Prohibited Orchestrator
Actions table, which already owns that prohibition, instead of restating
it in different words. Preserved the non-duplicated content in the same
bullet (scoped test evidence discipline, the image-generation boundary
note). No other section changed.

Measured: goop-orchestrator.md 9,807 -> 9,890 bytes (+83, the pointer
sentence is longer than the phrase it replaced) but drops one more
absolute-language hit (20 -> 19) since the bolded Never write code header
is gone. agents/ category: 92,422 -> 92,505 bytes, still a decrease from
the 99,822 baseline. Workflow-wide absolute-hit total: 390 -> 389.

Updated the prompt-audit current-tree sentinel for the new agents byte
total and absolute-hit total. Immutable Wave 1 baseline unchanged in
RESEARCH.md.
…d pointers

Consolidate the 13 non-orchestrator agent files following the
orchestrator pattern: one-line boot pointer to core-protocol.md
§Agent Boot Sequence replacing repeated boot prose, normalized
identity anchor, duplicated tier/delegation tables replaced by
pointers to dispatch-patterns.md, standalone Memory-First sections
removed (boot pointer covers memory-first semantically per the
semantic invariant test), Long-Running Commands trimmed, decorative
examples and closers removed, bold spans and absolute language
trimmed toward the spec targets.

All role-specific responsibilities, scope, methods, deliverables,
stop/block conditions, tool permissions, and safety boundaries
preserved: planner contract/spec/traceability/research-first duties,
debugger hypothesis/evidence/root-cause discipline, tester value-first
test authoring and scoped execution rules, verifier adversarial
spec/security/wiring review, researcher source quality/evidence/
provenance, explorer read-only mapping and uncertainty reporting,
writer documentation ownership and no invented claims, executor tier
scope distinctions, test/commit obligations, and frontend design/
accessibility distinctions.

Measured (bytes/absolute-hits/bold-spans):
agents/ category: 92,505/118/241 -> 76,992/62/127 (-16.8% bytes,
-56 abs, -114 bold). agents+commands combined: 118,052 -> 102,539
bytes, under the 118,798 (-5%) bar and the 112,546 (-10%) stretch
target. Workflow-wide absolute-language hits: 389 -> 333; the <=300
bar is a workflow-wide target requiring further command/reference
trimming. Agent bold spans: 241 -> 127, under the 200 bar.

Updated the prompt-audit real-tree sentinel test to the new agents
byte/bold-span totals and the new absolute-hit total. The immutable
baseline stays recorded in RESEARCH.md.
@hffmnnj
hffmnnj merged commit 3121fa4 into main Aug 3, 2026
0 of 2 checks passed
@hffmnnj
hffmnnj deleted the feat/prompt-baseline-and-composition branch August 3, 2026 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant