Repository navigation
fix(prompt): suppress the default forward so the LLM receives one question - #1992
rishinaren wants to merge 1 commit into
Conversation
…stion The node collects its inputs and emits a single merged, instructed question from closing(), but it never calls preventDefault(). The framework therefore also forwards the untouched incoming question, so a downstream llm_openai receives two and answers both. Only the merged question carries the configured instructions, so the user sees two replies and the first ignores every instruction the node exists to apply. Removing the node yields exactly one reply, which is what identifies it as the source of the second. examples/rag-pipeline.pipe wires prompt_1 from qdrant_2 on both the documents and questions lanes and feeds llm_openai_1 -- the exact shape that duplicates. Same defect and same fix as the guardrails node in rocketride-org#1849 / rocketride-org#1884. Fixes rocketride-org#1985
🤖 Internal: Discord sync markerAuto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe prompt node now prevents default forwarding after collecting questions, so downstream processing receives only the merged question emitted during ChangesPrompt question forwarding
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to The prompt node will stop forwarding the original question so downstream models receive one merged, instructed question instead of duplicate inputs and replies; no actionable merge-blocking risk remains beyond normal checks and review. Possibly related issues
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@rishinaren, CI is failing, could you fix? |
|
For whoever picks this up: the defect is still live.
|
asclearuc
left a comment
There was a problem hiding this comment.
Thanks for this fix, @rishinaren — the double-forward on the questions lane is a real defect, and the comment you left explains why the line exists rather than what it does, which is the useful kind.
Update (2026-09-08): CI has fresh results. My earlier draft said the logs had expired. The run was re-triggered on this same commit, so I could read them this time. Good news first: the Shell API contract failure that an earlier comment flagged as probably-not-yours is now green, and so are Ruff, gitleaks, check-externals and the rest. The bad news is that the two failures left are caused by this change.
Request changes
1. Two existing tests fail because of this change
The same two fail on Build / Ubuntu 22.04 and on Build / Windows Server 2022. Two different platforms, identical failures, so this is not a flake. develop is green. (Build / macOS (ARM64) was still running while I wrote this. I expect the same two there.)
Both failures are the old double-emit behaviour written into fixtures. In that sense they are evidence that your fix works. But they have to be updated inside this PR.
test/test_dynamic.py::TestDynamicNodes::test_node_cases[prompt:services:default]
$questions.[1].instructions[0].instructions: Could not access property at
'[1].instructions[0].instructions': list index out of range
The prompt node's own test case reads the merged question at index [1] — nodes/src/nodes/prompt/services.json:103. Index [1] existed only because the raw forwarded question sat at [0]. Now the merged question is the only output, so the path should be [0].instructions[0].instructions.
test/test_lifecycle_order.py::test_join_flushes_after_every_branch
AssertionError: the join merged 2 entries, expected 4
assert 2 == 4
That test builds a diamond of prompt nodes. EXPECTED_JOIN_ENTRIES = 4 means "2 branches x (forwarded + flushed)" — nodes/test/test_lifecycle_order.py:97-99. You removed the forwarded half, so each branch now reaches the join once. The constant becomes 2.
Please also rewrite the module docstring at nodes/test/test_lifecycle_order.py:37-39. It states the contract as "prompt forwards what it is written straight away AND accumulates it". After this PR that sentence is false. It is the first thing the next reader of that file will believe.
One thing to confirm while you are in there. That test exists to catch a join that flushes before an upstream branch. With one entry per branch instead of two, a dropped flush shows up as 1 instead of 2. So the test still catches the regression — arguably better than before, because there is no forwarded copy left to mask it. Please say in the PR description that you checked this. Otherwise the next reader sees 4 become 2 and reads it as a weakened test.
2. No test for the new behaviour
nodes/test/test_lane_forward_once_nodes.py is the harness built for exactly this contract. It came from #2063, the lane-forwarding sweep that covered anomaly_detector, ner, search_exa, llamaparse and llm_vision_mistral — but not prompt. Adding the prompt case there finishes that sweep. Details inline.
Fixing the two tests in section 1 is necessary but not enough. Neither of them asserts this node's own delivery count.
3. Stale branch
The merge base is 2026-08-14: 24 days old, and 76 commits behind develop. Please rebase, so that the next green run means something.
Two inline comments below: the missing test, and the three sibling lanes that still default-forward.
Still open from my earlier draft: preventDefault() is called inside writeQuestions, which only runs when a question arrives. If the node can close after receiving documents but no questions, is the default forward still suppressed on that path? If the answer is that it does not matter because there is nothing to forward, that is fine — and worth a word in the comment.
| # instructions, and the merged one emitted from closing(). Both are | ||
| # answered, and the first reply ignores every instruction the node | ||
| # exists to apply. Same defect and same fix as the guardrails node. | ||
| self.preventDefault() |
There was a problem hiding this comment.
MUST fix — add the delivery-count regression test.
nodes/test/test_lane_forward_once_nodes.py already exists for exactly this contract. It was added by #2063 ("deliver each lane payload exactly once"), which swept anomaly_detector, ner, search_exa, llamaparse and llm_vision_mistral. prompt was missed. Your fix is the same fix; the test belongs in the same file, with the same test_<node>_<lane>_delivered_once naming.
Its _simulate_engine_dispatch() models the engine rule this fix depends on: the engine forwards the incoming argument of a lane handler after the handler returns, unless the handler raised Ec.PreventDefault. That lets a test assert how many payloads reach downstream, instead of only asserting that preventDefault() was called.
Shape used by the nodes already covered there:
def test_prompt_question_delivered_once():
inst, fake = _build('prompt', types.SimpleNamespace(config={}))
q = FakeQuestion()
q.addQuestion('hi')
delivered = []
_simulate_engine_dispatch(lambda: inst.writeQuestions(q), lambda: delivered.append(q))
delivered.extend(fake.delivered.get('questions', []))
assert delivered == [], 'the raw incoming question must not reach downstream'One thing to watch: FakeQuestion in that file implements only addContext and addQuestion. If you also want to cover closing(), so that the merged question is emitted exactly once, FakeQuestion needs addInstruction and getPrompt next to them.
Why this matters more than usual here: the failure mode is silent. Two answers instead of one, no error, no traceback. A later refactor can drop this line and CI stays green.
nodes/test/guardrails/test_all.py:580 also stubs preventDefault, but it only proves the call happened. The lane-forward harness proves the delivery count, which is the real contract.
| # exists to apply. Same defect and same fix as the guardrails node. | ||
| self.preventDefault() | ||
|
|
||
| def writeDocuments(self, documents): |
There was a problem hiding this comment.
should fix — the other three lanes have the same defect.
writeDocuments, writeText and writeTable do exactly what writeQuestions did before this PR: they collect the payload into self.question and return normally. The engine then default-forwards the raw documents, text or table downstream, while closing() separately emits the merged question that already contains that same content. A downstream LLM listening on both lanes therefore sees the content twice — the defect this PR fixes on the questions lane.
The guardrails node you mirror suppresses every lane it handles, not only questions: nodes/src/nodes/guardrails/IInstance.py:98 (questions), :149 (answers), :174 (documents).
If leaving documents, text and table as deliberate pass-through is the intent, that is your call — please say so in the PR description so the asymmetry is on the record. If it is not intended, self.preventDefault() belongs at the end of those three handlers too, each with its own case in test_lane_forward_once_nodes.py.
Summary
promptnode collects its inputs and emits one merged, instructed question fromclosing(), but never callspreventDefault(). The framework also forwards the untouched incoming question, so a downstreamllm_openaireceives two and answers both.instructions, so the user sees two replies and the first ignores every instruction the node exists to apply.guardrailsnode in guardrails node duplicates every answer (and question) it forwards — missing preventDefault() #1849 and guardrails node duplicates every answer that passes through it #1884.The node's own README already documents the behaviour this implements: "the fully assembled question is not emitted until the node is closing ... the enriched question is emitted on the
questionslane". A single emission, at close, carrying the instructions. The pass-through is not documented anywhere, so this brings the code in line with its contract rather than changing it.examples/rag-pipeline.pipewiresprompt_1fromqdrant_2on both thedocumentsandquestionslanes and feedsllm_openai_1, which is the exact shape that duplicates.Type
fix
Testing
./builder testpassesruff checkandruff format --checkclean.Observed on
chat -> embedding_openai -> pinecone -> prompt -> llm_openai -> response_answersrunning on RocketRide Cloud. One question, two replies: the first ignored the instruction "refer to each excerpt as 'Excerpt N', give exactly one answer", the second followed it. Removing thepromptnode yields exactly one reply.writeDocuments,writeTextandwriteTablekeep collecting without suppression on purpose: their lanes declare no outputs inservices.json, so there is nothing to forward.Checklist
Linked Issue
Fixes #1985