feat(agent): surface the run's artist_account_id in the agent system prompt#742
Conversation
…prompt
Headless runs (customer-prompt-task -> /api/chat/runs) receive artistId on every
scheduled run, but it stopped at session provisioning — 0 occurrences past
provisionRunSession — so agents roster-guess and nondeterministically land on the
org-scoped artist list ("Rostrum Pacific" trap): tasks with implicit artists
delivered 1/4 across two benchmark rounds while the one id-embedding task went 2/2
(recoupable/chat#1837).
buildAgentSystemPrompt now emits an IMPORTANT CONTEXT VALUES block (artist_account_id
+ account_id, mirroring the interactive chat's getSystemPrompt) whenever the run has
them; threaded via buildRunAgentInput -> RunAgentWorkflowInput -> runAgentStep.
handleStartChatRun passes the validated artistId; handleChatWorkflowStream passes
session.artist_id.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Review limit reached
Next review available in: 30 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (6)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
2 issues found across 7 files
Confidence score: 4/5
- In
app/lib/workflows/runAgentStep.ts, makingaccount_idoptional at therunAgentStepboundary weakens a useful compile-time guard, so a missing ID could slip through and produce incorrect prompt context at runtime — makeaccount_idrequired at this boundary (or enforce/normalize it before this call) before merging. - In
lib/chat/__tests__/buildAgentSystemPrompt.test.ts, coverage skips the single-ID branches (artistId-only andaccountId-only), which is where prompt assembly can silently drift — add explicit one-ID test cases to de-risk regressions before merge.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="lib/chat/__tests__/buildAgentSystemPrompt.test.ts">
<violation number="1" location="lib/chat/__tests__/buildAgentSystemPrompt.test.ts:34">
P3: The new prompt logic has separate `artistId` and `accountId` branches, but the added tests only verify "both present" and "neither present." That leaves the one-ID paths unprotected, which is where this feature can silently regress in future edits. Adding explicit tests for `artistId`-only and `accountId`-only inputs would make the new behavior contract complete.</violation>
</file>
<file name="app/lib/workflows/runAgentStep.ts">
<violation number="1" location="app/lib/workflows/runAgentStep.ts:42">
P2: The new `account_id` prompt propagation is currently type-optional at the `runAgentStep` boundary, which weakens the compile-time guarantee that this context is always present. Since upstream workflow input already requires `accountId`, making it optional here can let future call sites accidentally omit it and silently remove `account_id` from the system prompt again. Keeping this field required in `RunAgentStepInput` would preserve the end-to-end contract.</violation>
</file>
Architecture diagram
sequenceDiagram
participant Client as External Client/Queue
participant API as API Route (handleStartChatRun / handleChatWorkflowStream)
participant Builder as buildRunAgentInput
participant Workflow as RunAgentWorkflow
participant Step as runAgentStep
participant Prompt as buildAgentSystemPrompt
participant Model as LLM Model (streamText)
Note over Client,Model: NEW: Artist ID flow through agent chain
alt Scheduled Run (Headless)
Client->>API: POST /api/chat/runs (artistId included)
API->>API: Validate artistId
API->>Builder: buildRunAgentInput({artistId, accountId, ...})
Builder-->>API: RunAgentWorkflowInput (includes artistId)
API->>Workflow: runAgentWorkflow({artistId, ...})
Workflow->>Step: runAgentStep({artistId, accountId, ...})
else Interactive Chat
Client->>API: WebSocket / chat message
API->>API: Extract session.artist_id
API->>Builder: buildRunAgentInput({artistId: session.artist_id, accountId, ...})
Builder-->>API: RunAgentWorkflowInput (includes artistId)
API->>Workflow: runAgentWorkflow({artistId, ...})
Workflow->>Step: runAgentStep({artistId, accountId, ...})
end
Note over Step,Prompt: System prompt injection
Step->>Prompt: buildAgentSystemPrompt({artistId, accountId, ...})
alt Both artistId and accountId provided
Prompt->>Prompt: NEW: Generate "IMPORTANT CONTEXT VALUES" block
Note over Prompt: Includes: artist_account_id: <id><br/>account_id: <id>
else Only one provided or none
Prompt->>Prompt: Include only provided IDs or omit block entirely
end
Prompt-->>Step: Complete system prompt with context
Step->>Model: streamText({system: prompt, ...})
Model-->>Step: LLM responses with accurate artist context
Note over Step,Model: Agent uses exact artist_account_id instead of roster search
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| /** Active artist for the run — surfaced in the agent's system prompt (chat#1837). */ | ||
| artistId?: string; | ||
| /** Owning account id — surfaced alongside the artist in the system prompt. */ | ||
| accountId?: string; |
There was a problem hiding this comment.
P2: The new account_id prompt propagation is currently type-optional at the runAgentStep boundary, which weakens the compile-time guarantee that this context is always present. Since upstream workflow input already requires accountId, making it optional here can let future call sites accidentally omit it and silently remove account_id from the system prompt again. Keeping this field required in RunAgentStepInput would preserve the end-to-end contract.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At app/lib/workflows/runAgentStep.ts, line 42:
<comment>The new `account_id` prompt propagation is currently type-optional at the `runAgentStep` boundary, which weakens the compile-time guarantee that this context is always present. Since upstream workflow input already requires `accountId`, making it optional here can let future call sites accidentally omit it and silently remove `account_id` from the system prompt again. Keeping this field required in `RunAgentStepInput` would preserve the end-to-end contract.</comment>
<file context>
@@ -36,6 +36,10 @@ export type RunAgentStepInput = {
+ /** Active artist for the run — surfaced in the agent's system prompt (chat#1837). */
+ artistId?: string;
+ /** Owning account id — surfaced alongside the artist in the system prompt. */
+ accountId?: string;
/**
* Whether a user is present to answer `ask_user_question`. Interactive chat
</file context>
| it("emits IMPORTANT CONTEXT VALUES when artistId/accountId are provided", () => { | ||
| const prompt = buildAgentSystemPrompt({ | ||
| artistId: "ebae4bb9-e38f-4763-b3c8-ff30e99f5d01", | ||
| accountId: "fb678396-a68f-4294-ae50-b8cacf9ce77b", | ||
| }); | ||
| expect(prompt).toMatch(/IMPORTANT CONTEXT VALUES/); | ||
| expect(prompt).toContain("artist_account_id: ebae4bb9-e38f-4763-b3c8-ff30e99f5d01"); | ||
| expect(prompt).toContain("account_id: fb678396-a68f-4294-ae50-b8cacf9ce77b"); | ||
| }); |
There was a problem hiding this comment.
P3: The new prompt logic has separate artistId and accountId branches, but the added tests only verify "both present" and "neither present." That leaves the one-ID paths unprotected, which is where this feature can silently regress in future edits. Adding explicit tests for artistId-only and accountId-only inputs would make the new behavior contract complete.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At lib/chat/__tests__/buildAgentSystemPrompt.test.ts, line 34:
<comment>The new prompt logic has separate `artistId` and `accountId` branches, but the added tests only verify "both present" and "neither present." That leaves the one-ID paths unprotected, which is where this feature can silently regress in future edits. Adding explicit tests for `artistId`-only and `accountId`-only inputs would make the new behavior contract complete.</comment>
<file context>
@@ -31,4 +31,17 @@ describe("buildAgentSystemPrompt", () => {
expect(prompt).toMatch(/never fabricate/i);
expect(prompt).toMatch(/sample|estimate|industry average/i);
});
+ it("emits IMPORTANT CONTEXT VALUES when artistId/accountId are provided", () => {
+ const prompt = buildAgentSystemPrompt({
+ artistId: "ebae4bb9-e38f-4763-b3c8-ff30e99f5d01",
</file context>
| it("emits IMPORTANT CONTEXT VALUES when artistId/accountId are provided", () => { | |
| const prompt = buildAgentSystemPrompt({ | |
| artistId: "ebae4bb9-e38f-4763-b3c8-ff30e99f5d01", | |
| accountId: "fb678396-a68f-4294-ae50-b8cacf9ce77b", | |
| }); | |
| expect(prompt).toMatch(/IMPORTANT CONTEXT VALUES/); | |
| expect(prompt).toContain("artist_account_id: ebae4bb9-e38f-4763-b3c8-ff30e99f5d01"); | |
| expect(prompt).toContain("account_id: fb678396-a68f-4294-ae50-b8cacf9ce77b"); | |
| }); | |
| it.each([ | |
| [{ artistId: "ebae4bb9-e38f-4763-b3c8-ff30e99f5d01" }, ["artist_account_id: ebae4bb9-e38f-4763-b3c8-ff30e99f5d01"], ["account_id:"]], | |
| [{ accountId: "fb678396-a68f-4294-ae50-b8cacf9ce77b" }, ["account_id: fb678396-a68f-4294-ae50-b8cacf9ce77b"], ["artist_account_id:"]], | |
| [ | |
| { | |
| artistId: "ebae4bb9-e38f-4763-b3c8-ff30e99f5d01", | |
| accountId: "fb678396-a68f-4294-ae50-b8cacf9ce77b", | |
| }, | |
| [ | |
| "artist_account_id: ebae4bb9-e38f-4763-b3c8-ff30e99f5d01", | |
| "account_id: fb678396-a68f-4294-ae50-b8cacf9ce77b", | |
| ], | |
| [], | |
| ], | |
| ])("emits IMPORTANT CONTEXT VALUES for %o", (options, expected, unexpected) => { | |
| const prompt = buildAgentSystemPrompt(options); | |
| expect(prompt).toMatch(/IMPORTANT CONTEXT VALUES/); | |
| for (const value of expected) expect(prompt).toContain(value); | |
| for (const value of unexpected) expect(prompt).not.toContain(value); | |
| }); |
Preview verification — the implicit-artist prompt that went 0/2 now resolves its artist instantlyRan the exact failing benchmark on this PR's preview ( Before (main, 2 runs) vs after (this PR, 1 run)
The agent's own words: "I checked the YouTube channel for the artist (Cosculluela El Principe / @cosculluelaelprincipe, 8.28M subscribers) by triggering a fresh scrape of the channel's videos page. Result: No new video in the last 24 hours." No roster-guessing, no org-trap, no fabrication — the injected Notes: CI green (format/lint/test/cubic); the run correctly sent no email because the prompt is conditional and there genuinely was no new video in the window — that's the desired no-spam behavior, not a failure. Env note in the test prompt only pointed the sandbox at the test API base (credential scoping); it said nothing about the artist, so the experiment isolates this PR's injection. Tested on preview, 2026-07-02. Pairs with recoupable/skills#71 (defense-in-depth for no-context runs). |
1 participant
Part of recoupable/chat#1837 (primary fix). Base:
main.Problem
Every scheduled run passes
artistId(customer-prompt-task→POST /api/chat/runs), but the value stops at session provisioning — 0 occurrences inbuildRunAgentInput/runAgentWorkflow/runAgentStep. The interactive chat injectsartist_account_idinto its system prompt (getSystemPrompt.ts:44); the headless prompt (buildAgentSystemPrompt) had only env + custom instructions. Result: agents roster-guess, nondeterministically hit the org-scoped artist list, and conclude the artist doesn't exist.Benchmark evidence (#1837 comment): across 2 rounds of Steph@OneRPM's 3 tasks — id-embedded prompt 2/2 delivered; implicit-artist prompts 1/4 (chats
fd8e8b26,aa4eb91f,4fa700b5all declared the artist missing while it sat in the personal roster).Fix
buildAgentSystemPromptgainsartistId?/accountId?→ emits, when present:artistIdthroughBuildRunAgentInputParams→RunAgentWorkflowInput→RunAgentStepInput(+ declaredaccountIdon the step input).handleStartChatRunpasses the validatedartistId. Interactive:handleChatWorkflowStreampassessession.artist_id(sessions already carry it).Tests (TDD)
buildAgentSystemPrompt: emits the context block with both ids; omits it when neither is provided.lib/chat+app/lib/workflowssuites green (570); tsc adds no new errors (2 pre-existing baseline); eslint clean.Preview verification to follow: re-run the implicit-artist Cosculluela prompt verbatim on the preview — Done-when is the agent using the injected
artist_account_idinstead of roster-guessing.🤖 Generated with Claude Code
Summary by cubic
Injects the run’s
artist_account_id(andaccount_id) into the agent system prompt and threadsartistIdthrough the workflow to stop roster-guessing in headless runs, matching interactive chat behavior.buildAgentSystemPromptadds an “IMPORTANT CONTEXT VALUES” block when ids are present.artistIdviabuildRunAgentInput→RunAgentWorkflowInput→runAgentStep.handleStartChatRunpasses the validatedartistId;handleChatWorkflowStreampassessession.artist_id.Written for commit 57496d7. Summary will update on new commits.