Repository navigation
fix(prompts): build bundled assets into the directory the resolver reads - #796
Conversation
`builtInPromptRoot()` resolves packaged prompts from `dist/assets/prompts/` and `scripts/validate-audit-profile-package.mjs` asserts paths under that same directory, but the build copied them to `dist/prompts/`, so the packaged lookup could never succeed. A workspace checkout hid this: the resolver's source-tree fallback, `../../../.ultrafuzz/prompts/` relative to `dist/assets.js`, resolves to the repository's own prompts directory. Inside a sealed workflow execution snapshot the package is relocated under `modules/@ultrafuzz/`, that relative path lands outside any prompts directory, and prompt rendering fails the run. Point the build at the location the resolver and the package validator already agree on. Closes #795 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| ], | ||
| "scripts": { | ||
| "build": "tsc -p tsconfig.json && rm -rf dist/prompts && cp -R ../../.ultrafuzz/prompts dist/prompts", | ||
| "build": "tsc -p tsconfig.json && rm -rf dist/assets/prompts && mkdir -p dist/assets && cp -R ../../.ultrafuzz/prompts dist/assets/prompts", |
There was a problem hiding this comment.
Output-contract templates remain unresolved
When a clean installed or relocated package calls renderPrompt, this build no longer produces the dist/prompts/_templates/output-contract directory that the unconditional template resolver still reads, causing every prompt render to throw unable to locate the packaged output contract templates.
Knowledge Base Used: Prompt library
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/prompts/package.json
Line: 19
Comment:
**Output-contract templates remain unresolved**
When a clean installed or relocated package calls `renderPrompt`, this build no longer produces the `dist/prompts/_templates/output-contract` directory that the unconditional template resolver still reads, causing every prompt render to throw `unable to locate the packaged output contract templates`.
**Knowledge Base Used:** [Prompt library](https://app.greptile.com/monad-foudnation/-/custom-context/knowledge-base/monad-developers/ultrafuzz/-/docs/prompt-library.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.…et layout Moving the prompt tree to dist/assets/prompts fixed builtInPromptRoot but left outputContractTemplateRoot in the same file still looking at dist/prompts, which this build no longer emits. That left the repository fallback as the only resolvable candidate. It exists in a source checkout, so rendering succeeded at submission time and every local test passed. It does not exist in a sealed execution snapshot, where the package sits at modules/@ultrafuzz/prompts/dist and '../../../' reaches modules/. A run therefore rendered, ran for over an hour, and died at WORKFLOW_RENDER_FAILED the first time dynamic expansion re-rendered the workflow from inside the snapshot: unable to locate the packaged output contract templates from .../execution-snapshots/<id>/modules/@ultrafuzz/prompts/dist Add the packaged layout as the first candidate and keep both existing ones. The new test copies the built dist into a directory deep enough that the repository fallback cannot resolve, which is what the existing installed-layout test does not do -- it stages assets at dist/prompts by hand, a layout this build no longer produces, so it could not have caught this. Verified the test fails without the resolver change and passes with it; 103/103 prompts tests.
|
Pushed a second commit: this PR as it stood was incomplete and takes runs down mid-flight. Moving the prompt tree to That fallback exists in a source checkout, so rendering succeeds at submission time and the test suite passes. It does not exist in a sealed execution snapshot, where the package sits at A campaign run therefore submitted cleanly, ran for 1h25m to 24/91 nodes, and then died the first time dynamic expansion re-rendered the workflow from inside the snapshot: The fix adds the packaged layout as the first candidate and keeps both existing ones. Also worth noting why the existing |
…rphaned by #1131 (#1175) * refactor(prompts): delete the unused prompt rename feature and duplicate template resolver renamePromptId, renameConcreteNodeId and rewriteArtifactPathForPromptId (rename.ts), their render.ts helpers (renamePromptArtifactReferences and three private rewriters) and the frontmatter round-trip support they needed (serializePromptDocument, diffPromptIdentity, the allowUnknownFields option, unknownFrontmatter/rawFrontmatter and the invalid-rename error code) had no caller outside their own tests. getPrompt had no caller at all. TemplateOccurrence.rawName existed only so rename could preserve whitespace. Output-contract templates had their own three-candidate resolver whose middle candidate, dist/prompts, is the layout the build stopped producing in #796. They now resolve through builtInPromptRoot(), the packaged-first lookup the agent preamble templates already use, so both kinds of prompt asset follow one rule. The packaged-layout render test, which copies the real dist below a directory with no repository above it, still covers the sealed-snapshot case; the installed-layout test that rebuilt the old dist/prompts layout is removed and its content assertions are folded into the packaged-layout test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(config): delete the prompt-metadata layer and helpers with no production caller - The prompt-metadata layer: resolveConfig applied createDefaultPromptMetadataLayer(), which always returned {}, and an input.promptMetadata that only one test supplied. PromptMetadataLayer, ResolveConfigInput.promptMetadata and the "prompt-metadata" diagnostic source go with it. - restoreRedactedConfig and assertNoRedactionPlaceholders: nothing in runtime, cli, modal or evals calls them. Runs launch from the unredacted resolved config; the redacted copy and its manifest are written for readers of the run directory, and no command restores values from them. docs/config.md no longer describes a restore step or launch guard that does not exist. The "redaction" diagnostic source goes with them. - applyDefaultProfileOverrides: a one-line wrapper only tests called. Its test now drives applyModelProfileOverrides, which validate.ts and plan-run.ts use. - assertResolvedConfigZod and resolvedConfigSchemaEntry: no callers. - resolvedConfigValidatorsAgree: moved into the schema test, its only user. redactResolvedConfig and serializeRedactedResolvedConfigToml, which plan-run.ts and init.ts use, are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: delete benchmark CI scripts orphaned by #1131 #1131 deleted eval-benchmarks.yml, the only workflow that invoked scripts/ci/modal-benchmark-control-window.mjs (create/deadline) and scripts/ci/validate-threat-model-benchmark-gate.mjs. Since then the control-window script was kept alive only by a knip entry and a block of ci-config.test.ts that exercised it, and the threat-model gate only by its own test, which still ran in every PR's test:ci-scripts step. Delete both scripts, the gate's test, the knip entry, and the control-window block of the EVMBench full-mode manifest test; the manifest assertions around it stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(modal): delete unreferenced exports and move test-only helpers into tests Deleted, no reference anywhere: MAX_MODAL_DOCUMENT_BYTES, MODAL_SMOKE_STOP_PATH (smoke-worker.ts builds its own stop path) and DEFAULT_NODE_TIMEOUT_SECONDS. Deleted with the tests that only exercised them: - createTrackedSourceArchive; production archives the exact candidate with createExactCandidateSourceArchive. - persistentWorkspaceRoot. - locateModalResumeWorkspace, documented as "retained for tests"; worker.ts uses findModalResumeWorkspace, and the removed "locates one exact linked durable run" test duplicated the existing tolerant-lookup test. The two finalize tests unwrap findModalResumeWorkspace through a local helper. Moved into tests, their only users: - DEFAULT_BENCHMARK_MODELS, a stale model list no production code reads, is now test/model-spec-fixtures.ts, typed as a const tuple so indexed fixtures need no non-null assertion. - modalBenchmarkConfigValidatorsAgree, WORKER_RESULT_ALLOWED_KEYS (the test now states the expected persisted key set), and publicEvalFailureDiagnosticLogPayload, which composed the same two calls runCommand already makes inline. - PUBLIC_BENCHMARK_MAX_PARALLEL_EVAL_ROWS and its full-lane twin, aliases of the evals lane limits that only a test imported. Un-exported, used only inside their own module: five public-worker constants and WORKER_STDERR_TAIL_BYTES. The source-constant reader in prepare-eval-history-publication.mjs matches top-level const declarations with or without export, and its test against the real public-worker.ts still passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(dashboard): drop the always-false command capabilities commandCapabilities() returned restartWholeRun, doctor, triage, merge, restartFromNode, rerunSelectedNode and arbitraryShell as the constant false. The frontend only declared them in its CommandCapabilities type: capabilityForCommand never maps a command to any of them, so nothing read them. Remove them from the server, the dashboard HTTP schema's commandCapabilities definition, and the frontend type. The schema edit is required because that definition is closed (required plus additionalProperties: false). The dashboard schema feeds neither the validator build identity, which hashes only the artifacts validator modules and the ajv versions, nor the artifact schema bundle digest that planned outputs and the trusted CLI identity bind to. The flow test's two assertions that doctor and merge are false are removed. Every dashboard HTTP response in the tests is still validated against the schema, so a server that emitted one of these keys again would fail the flow test with "must NOT have additional properties" (checked by re-adding doctor: false to the compiled server). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(runtime): drop the unused run-tests selectors packages/runtime/package.json calls scripts/run-tests.mjs with no selector or with "supporting"; nothing passes "materialize", "clean" or "smithers". A single test file can still be selected by name through the existing fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): record the dead-code deletions Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(prompts): leave getPrompt to the catalog work item getPrompt still has no caller, but #1164 adds projectPromptsDifferingFromBuiltIns directly above it in catalog.ts, so deleting it here made the two PRs conflict textually (git merge-tree reported CONFLICT (content) in catalog.ts). catalog.ts is outside this change's files, so restore it to origin/main and leave the deletion for a follow-up once #1164 lands. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): state the dead-code entry only as strongly as the evidence Drop getPrompt, which this change no longer deletes. Name the actual callers of the two CI scripts (eval-benchmarks.yml and the watch-modal-benchmark.sh it ran, both deleted in #1131) instead of "the workflows #1131 deleted": the other two workflows #1131 removed never invoked either script. Stop describing the config prompt-metadata layer as code no production path calls: resolveConfig ran it on every resolution, only ever with an empty layer. Mention the run-tests.mjs selectors and the docs/config.md change, and add the [runtime] and [docs] tags for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes #795
Problem
@ultrafuzz/promptsbuilt its bundled prompt assets intodist/prompts/, while two other places expectdist/assets/prompts/:packages/prompts/src/assets.ts—new URL("./assets/prompts/", import.meta.url), andassets.jsis emitted atdist/assets.jsscripts/validate-audit-profile-package.mjs— requiresdist/assets/prompts/review/aggregate-test-files.mdanddist/assets/prompts/review/dedupe-findings.mdSo the packaged lookup could never succeed, and the two directories never matched.
A workspace checkout masks it.
builtInPromptRoot()falls back to../../../.ultrafuzz/prompts/relative todist/assets.js, which resolves to the repository's own.ultrafuzz/prompts. Inside a sealed workflow execution snapshot the package is relocated undermodules/@ultrafuzz/…, so that same relative path lands at…/runtime/node_modules/.ultrafuzz/prompts, both lookups fail, and the run dies during prompt rendering:Change
Point the build destination at
dist/assets/prompts, the location the resolver and the package validator already agree on.files: ["dist"]already ships it.Verification
dist/assets/prompts/is populated (6 entries) and both files named byvalidate-audit-profile-package.mjsexist. Before the change the directory did not exist at all.builtInPromptRoot()now resolves, so the source-tree fallback is no longer load-bearing outside a checkout.Note
This was found downstream of #793: with that confinement bug fixed, launches got far enough to render prompts and hit this. The two are independent and this PR stands alone.
Greptile Summary
The PR aligns bundled prompt assets with the packaged paths used by both prompt and output-contract resolution.
dist/assets/prompts.Confidence Score: 5/5
The PR appears safe to merge.
The previously reported packaged output-contract failure no longer remains: the build copies templates into
dist/assets/prompts, the resolver checks that exact location, and the relocated-package test exercises rendering without a source-tree fallback.Important Files Changed
dist/assets/promptslayout, resolving the previously reported failure.Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart LR A[Source prompt assets] -->|package build| B[dist/assets/prompts] B --> C[Built-in prompt resolver] B --> D[Output-contract template resolver] C --> E[Packaged prompt rendering] D --> EReviews (2): Last reviewed commit: "fix(prompts): resolve output-contract te..." | Re-trigger Greptile