Skip to content

fix(prompts): build bundled assets into the directory the resolver reads - #796

Merged
aviggiano merged 2 commits into
mainfrom
fix/prompts-asset-path
Aug 25, 2026
Merged

aviggiano merged 2 commits into
mainfrom
fix/prompts-asset-path

Conversation

@aviggiano

@aviggiano aviggiano commented Aug 24, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #795

Problem

@ultrafuzz/prompts built its bundled prompt assets into dist/prompts/, while two other places expect dist/assets/prompts/:

  • packages/prompts/src/assets.ts — new URL("./assets/prompts/", import.meta.url), and assets.js is emitted at dist/assets.js
  • scripts/validate-audit-profile-package.mjs — requires dist/assets/prompts/review/aggregate-test-files.md and dist/assets/prompts/review/dedupe-findings.md

So the packaged lookup could never succeed, and the two directories never matched.

A workspace checkout masks it. builtInPromptRoot() falls back to ../../../.ultrafuzz/prompts/ relative to dist/assets.js, which resolves to the repository's own .ultrafuzz/prompts. Inside a sealed workflow execution snapshot the package is relocated under modules/@ultrafuzz/…, so that same relative path lands at …/runtime/node_modules/.ultrafuzz/prompts, both lookups fail, and the run dies during prompt rendering:

render frame: built-in prompt assets were not found in the package
(<snapshot>/modules/@ultrafuzz/runtime/node_modules/@ultrafuzz/prompts/dist/assets/prompts/)
or source tree
(<snapshot>/modules/@ultrafuzz/runtime/node_modules/.ultrafuzz/prompts/)

Change

Point the build destination at dist/assets/prompts, the location the resolver and the package validator already agree on. files: ["dist"] already ships it.

Verification

  • After a clean rebuild, dist/assets/prompts/ is populated (6 entries) and both files named by validate-audit-profile-package.mjs exist. Before the change the directory did not exist at all.
  • The packaged branch of builtInPromptRoot() now resolves, so the source-tree fallback is no longer load-bearing outside a checkout.

Note

This was found downstream of #793: with that confinement bug fixed, launches got far enough to render prompts and hit this. The two are independent and this PR stands alone.

Greptile Summary

The PR aligns bundled prompt assets with the packaged paths used by both prompt and output-contract resolution.

  • Copies the prompt tree into dist/assets/prompts.
  • Resolves output-contract templates from the new packaged location while retaining legacy fallbacks.
  • Adds a relocated-package rendering test that excludes the repository fallback.

Confidence Score: 5/5

The PR appears safe to merge.

The previously reported packaged output-contract failure no longer remains: the build copies templates into dist/assets/prompts, the resolver checks that exact location, and the relocated-package test exercises rendering without a source-tree fallback.

Important Files Changed

Filename Overview
packages/prompts/package.json The build now copies bundled prompts into the directory consumed by packaged asset resolution.
packages/prompts/src/render.ts Output-contract template lookup now includes the packaged dist/assets/prompts layout, resolving the previously reported failure.
packages/prompts/test/render.test.ts The added test relocates the built package without a repository fallback and verifies packaged output-contract rendering.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Source prompt assets] -->|package build| B[dist/assets/prompts]
  B --> C[Built-in prompt resolver]
  B --> D[Output-contract template resolver]
  C --> E[Packaged prompt rendering]
  D --> E
Loading

Reviews (2): Last reviewed commit: "fix(prompts): resolve output-contract te..." | Re-trigger Greptile

`builtInPromptRoot()` resolves packaged prompts from `dist/assets/prompts/`
and `scripts/validate-audit-profile-package.mjs` asserts paths under that
same directory, but the build copied them to `dist/prompts/`, so the
packaged lookup could never succeed.

A workspace checkout hid this: the resolver's source-tree fallback,
`../../../.ultrafuzz/prompts/` relative to `dist/assets.js`, resolves to
the repository's own prompts directory. Inside a sealed workflow execution
snapshot the package is relocated under `modules/@ultrafuzz/`, that
relative path lands outside any prompts directory, and prompt rendering
fails the run.

Point the build at the location the resolver and the package validator
already agree on.

Closes #795

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
],
"scripts": {
"build": "tsc -p tsconfig.json && rm -rf dist/prompts && cp -R ../../.ultrafuzz/prompts dist/prompts",
"build": "tsc -p tsconfig.json && rm -rf dist/assets/prompts && mkdir -p dist/assets && cp -R ../../.ultrafuzz/prompts dist/assets/prompts",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Output-contract templates remain unresolved

When a clean installed or relocated package calls renderPrompt, this build no longer produces the dist/prompts/_templates/output-contract directory that the unconditional template resolver still reads, causing every prompt render to throw unable to locate the packaged output contract templates.

Knowledge Base Used: Prompt library

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/prompts/package.json
Line: 19

Comment:
**Output-contract templates remain unresolved**

When a clean installed or relocated package calls `renderPrompt`, this build no longer produces the `dist/prompts/_templates/output-contract` directory that the unconditional template resolver still reads, causing every prompt render to throw `unable to locate the packaged output contract templates`.

**Knowledge Base Used:** [Prompt library](https://app.greptile.com/monad-foudnation/-/custom-context/knowledge-base/monad-developers/ultrafuzz/-/docs/prompt-library.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code

…et layout

Moving the prompt tree to dist/assets/prompts fixed builtInPromptRoot but left
outputContractTemplateRoot in the same file still looking at dist/prompts, which
this build no longer emits.

That left the repository fallback as the only resolvable candidate. It exists in
a source checkout, so rendering succeeded at submission time and every local
test passed. It does not exist in a sealed execution snapshot, where the package
sits at modules/@ultrafuzz/prompts/dist and '../../../' reaches modules/. A run
therefore rendered, ran for over an hour, and died at WORKFLOW_RENDER_FAILED the
first time dynamic expansion re-rendered the workflow from inside the snapshot:

  unable to locate the packaged output contract templates from
  .../execution-snapshots/<id>/modules/@ultrafuzz/prompts/dist

Add the packaged layout as the first candidate and keep both existing ones.

The new test copies the built dist into a directory deep enough that the
repository fallback cannot resolve, which is what the existing installed-layout
test does not do -- it stages assets at dist/prompts by hand, a layout this
build no longer produces, so it could not have caught this. Verified the test
fails without the resolver change and passes with it; 103/103 prompts tests.
@aviggiano

Copy link
Copy Markdown
Collaborator Author

Pushed a second commit: this PR as it stood was incomplete and takes runs down mid-flight.

Moving the prompt tree to dist/assets/prompts fixes builtInPromptRoot(), but outputContractTemplateRoot() in the same file still looks at dist/prompts, which this build no longer emits. The repository fallback (../../../.ultrafuzz/prompts/...) then becomes the only resolvable candidate.

That fallback exists in a source checkout, so rendering succeeds at submission time and the test suite passes. It does not exist in a sealed execution snapshot, where the package sits at modules/@ultrafuzz/prompts/dist and ../../../ reaches modules/.

A campaign run therefore submitted cleanly, ran for 1h25m to 24/91 nodes, and then died the first time dynamic expansion re-rendered the workflow from inside the snapshot:

WORKFLOW_RENDER_FAILED: unable to locate the packaged output contract templates from
  .../execution-snapshots/<id>/modules/@ultrafuzz/prompts/dist

The fix adds the packaged layout as the first candidate and keeps both existing ones.

Also worth noting why the existing installed package layout test could not catch this: it stages assets at dist/prompts by hand, which is the layout this PR removes. The added test copies the built dist into a directory deep enough that the repository fallback cannot resolve, reproducing the snapshot condition. I verified it fails without the resolver change and passes with it. 103/103 prompts tests, format clean.

@aviggiano aviggiano left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aviggiano
aviggiano merged commit b61cd71 into main Aug 25, 2026
7 checks passed
@aviggiano
aviggiano deleted the fix/prompts-asset-path branch August 25, 2026 11:07
aviggiano added a commit that referenced this pull request Sep 29, 2026
…rphaned by #1131 (#1175)

* refactor(prompts): delete the unused prompt rename feature and duplicate template resolver

renamePromptId, renameConcreteNodeId and rewriteArtifactPathForPromptId
(rename.ts), their render.ts helpers (renamePromptArtifactReferences and
three private rewriters) and the frontmatter round-trip support they
needed (serializePromptDocument, diffPromptIdentity, the
allowUnknownFields option, unknownFrontmatter/rawFrontmatter and the
invalid-rename error code) had no caller outside their own tests. getPrompt
had no caller at all. TemplateOccurrence.rawName existed only so rename
could preserve whitespace.

Output-contract templates had their own three-candidate resolver whose
middle candidate, dist/prompts, is the layout the build stopped producing
in #796. They now resolve through builtInPromptRoot(), the packaged-first
lookup the agent preamble templates already use, so both kinds of prompt
asset follow one rule. The packaged-layout render test, which copies the
real dist below a directory with no repository above it, still covers the
sealed-snapshot case; the installed-layout test that rebuilt the old
dist/prompts layout is removed and its content assertions are folded into
the packaged-layout test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(config): delete the prompt-metadata layer and helpers with no production caller

- The prompt-metadata layer: resolveConfig applied
  createDefaultPromptMetadataLayer(), which always returned {}, and an
  input.promptMetadata that only one test supplied. PromptMetadataLayer,
  ResolveConfigInput.promptMetadata and the "prompt-metadata" diagnostic
  source go with it.
- restoreRedactedConfig and assertNoRedactionPlaceholders: nothing in
  runtime, cli, modal or evals calls them. Runs launch from the unredacted
  resolved config; the redacted copy and its manifest are written for
  readers of the run directory, and no command restores values from them.
  docs/config.md no longer describes a restore step or launch guard that
  does not exist. The "redaction" diagnostic source goes with them.
- applyDefaultProfileOverrides: a one-line wrapper only tests called.
  Its test now drives applyModelProfileOverrides, which validate.ts and
  plan-run.ts use.
- assertResolvedConfigZod and resolvedConfigSchemaEntry: no callers.
- resolvedConfigValidatorsAgree: moved into the schema test, its only
  user.

redactResolvedConfig and serializeRedactedResolvedConfigToml, which
plan-run.ts and init.ts use, are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: delete benchmark CI scripts orphaned by #1131

#1131 deleted eval-benchmarks.yml, the only workflow that invoked
scripts/ci/modal-benchmark-control-window.mjs (create/deadline) and
scripts/ci/validate-threat-model-benchmark-gate.mjs. Since then the
control-window script was kept alive only by a knip entry and a block of
ci-config.test.ts that exercised it, and the threat-model gate only by its
own test, which still ran in every PR's test:ci-scripts step.

Delete both scripts, the gate's test, the knip entry, and the
control-window block of the EVMBench full-mode manifest test; the
manifest assertions around it stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(modal): delete unreferenced exports and move test-only helpers into tests

Deleted, no reference anywhere: MAX_MODAL_DOCUMENT_BYTES,
MODAL_SMOKE_STOP_PATH (smoke-worker.ts builds its own stop path) and
DEFAULT_NODE_TIMEOUT_SECONDS.

Deleted with the tests that only exercised them:
- createTrackedSourceArchive; production archives the exact candidate
  with createExactCandidateSourceArchive.
- persistentWorkspaceRoot.
- locateModalResumeWorkspace, documented as "retained for tests"; worker.ts
  uses findModalResumeWorkspace, and the removed "locates one exact linked
  durable run" test duplicated the existing tolerant-lookup test. The two
  finalize tests unwrap findModalResumeWorkspace through a local helper.

Moved into tests, their only users:
- DEFAULT_BENCHMARK_MODELS, a stale model list no production code reads,
  is now test/model-spec-fixtures.ts, typed as a const tuple so indexed
  fixtures need no non-null assertion.
- modalBenchmarkConfigValidatorsAgree, WORKER_RESULT_ALLOWED_KEYS (the
  test now states the expected persisted key set), and
  publicEvalFailureDiagnosticLogPayload, which composed the same two calls
  runCommand already makes inline.
- PUBLIC_BENCHMARK_MAX_PARALLEL_EVAL_ROWS and its full-lane twin, aliases
  of the evals lane limits that only a test imported.

Un-exported, used only inside their own module: five public-worker
constants and WORKER_STDERR_TAIL_BYTES. The source-constant reader in
prepare-eval-history-publication.mjs matches top-level const
declarations with or without export, and its test against the real
public-worker.ts still passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(dashboard): drop the always-false command capabilities

commandCapabilities() returned restartWholeRun, doctor, triage, merge,
restartFromNode, rerunSelectedNode and arbitraryShell as the constant
false. The frontend only declared them in its CommandCapabilities type:
capabilityForCommand never maps a command to any of them, so nothing read
them.

Remove them from the server, the dashboard HTTP schema's
commandCapabilities definition, and the frontend type. The schema edit
is required because that definition is closed (required plus
additionalProperties: false). The dashboard schema feeds neither the
validator build identity, which hashes only the artifacts validator
modules and the ajv versions, nor the artifact schema bundle digest that
planned outputs and the trusted CLI identity bind to.

The flow test's two assertions that doctor and merge are false are
removed. Every dashboard HTTP response in the tests is still validated
against the schema, so a server that emitted one of these keys again
would fail the flow test with "must NOT have additional properties"
(checked by re-adding doctor: false to the compiled server).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(runtime): drop the unused run-tests selectors

packages/runtime/package.json calls scripts/run-tests.mjs with no
selector or with "supporting"; nothing passes "materialize", "clean" or
"smithers". A single test file can still be selected by name through the
existing fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): record the dead-code deletions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(prompts): leave getPrompt to the catalog work item

getPrompt still has no caller, but #1164 adds
projectPromptsDifferingFromBuiltIns directly above it in catalog.ts, so
deleting it here made the two PRs conflict textually
(git merge-tree reported CONFLICT (content) in catalog.ts). catalog.ts is
outside this change's files, so restore it to origin/main and leave the
deletion for a follow-up once #1164 lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): state the dead-code entry only as strongly as the evidence

Drop getPrompt, which this change no longer deletes. Name the actual
callers of the two CI scripts (eval-benchmarks.yml and the
watch-modal-benchmark.sh it ran, both deleted in #1131) instead of "the
workflows #1131 deleted": the other two workflows #1131 removed never
invoked either script. Stop describing the config prompt-metadata layer as
code no production path calls: resolveConfig ran it on every resolution,
only ever with an empty layer. Mention the run-tests.mjs selectors and the
docs/config.md change, and add the [runtime] and [docs] tags for them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prompts package build writes assets to dist/prompts but the resolver and package validator expect dist/assets/prompts

2 participants