Skip to content

fix(prompt): suppress the default forward so the LLM receives one question - #1992

Open
rishinaren wants to merge 1 commit into
rocketride-org:developfrom
rishinaren:fix/RR-1985-prompt-preventdefault
Open

rishinaren wants to merge 1 commit into
rocketride-org:developfrom
rishinaren:fix/RR-1985-prompt-preventdefault

Conversation

@rishinaren

@rishinaren rishinaren commented Aug 15, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The node's own README already documents the behaviour this implements: "the fully assembled question is not emitted until the node is closing ... the enriched question is emitted on the questions lane". A single emission, at close, carrying the instructions. The pass-through is not documented anywhere, so this brings the code in line with its contract rather than changing it.

examples/rag-pipeline.pipe wires prompt_1 from qdrant_2 on both the documents and questions lanes and feeds llm_openai_1, which is the exact shape that duplicates.

Type

fix

Testing

  • Tests added or updated
  • Tested locally
  • ./builder test passes

ruff check and ruff format --check clean.

Observed on chat -> embedding_openai -> pinecone -> prompt -> llm_openai -> response_answers running on RocketRide Cloud. One question, two replies: the first ignored the instruction "refer to each excerpt as 'Excerpt N', give exactly one answer", the second followed it. Removing the prompt node yields exactly one reply.

writeDocuments, writeText and writeTable keep collecting without suppression on purpose: their lanes declare no outputs in services.json, so there is nothing to forward.

Checklist

  • Commit messages follow conventional commits
  • No secrets or credentials included
  • Wiki updated (if applicable)
  • Breaking changes documented (if applicable): anything relying on the duplicate was relying on undocumented behaviour

Linked Issue

Fixes #1985

…stion

The node collects its inputs and emits a single merged, instructed question
from closing(), but it never calls preventDefault(). The framework therefore
also forwards the untouched incoming question, so a downstream llm_openai
receives two and answers both.

Only the merged question carries the configured instructions, so the user
sees two replies and the first ignores every instruction the node exists to
apply. Removing the node yields exactly one reply, which is what identifies
it as the source of the second.

examples/rag-pipeline.pipe wires prompt_1 from qdrant_2 on both the documents
and questions lanes and feeds llm_openai_1 -- the exact shape that duplicates.

Same defect and same fix as the guardrails node in rocketride-org#1849 / rocketride-org#1884.

Fixes rocketride-org#1985
@github-actions github-actions Bot added the module:nodes Python pipeline nodes label Aug 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor
🤖 Internal: Discord sync marker

Auto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete.

@coderabbitai

coderabbitai Bot commented Aug 15, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 6fff1007-0963-42d0-92fe-87388c90f5f4

📥 Commits

Reviewing files that changed from the base of the PR and between 4a72f8d and 68f88c8.

📒 Files selected for processing (1)
  • nodes/src/nodes/prompt/IInstance.py

📝 Walkthrough

Walkthrough

The prompt node now prevents default forwarding after collecting questions, so downstream processing receives only the merged question emitted during closing().

Changes

Prompt question forwarding

Layer / File(s) Summary
Suppress original question forwarding
nodes/src/nodes/prompt/IInstance.py
writeQuestions calls preventDefault() after collecting incoming questions.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 68f88

The prompt node will stop forwarding the original question so downstream models receive one merged, instructed question instead of duplicate inputs and replies; no actionable merge-blocking risk remains beyond normal checks and review.

Possibly related issues

  • #1985 — The change fixes the prompt node defect described in this issue by suppressing forwarding of the original question.

Suggested reviewers: jmaionchi, rod-christensen, stepmikhaylov

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: suppressing default forwarding so the LLM receives one merged question.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@dylan-savage

Copy link
Copy Markdown
Collaborator

@rishinaren, CI is failing, could you fix?

@nihalnihalani

Copy link
Copy Markdown
Collaborator

For whoever picks this up: the defect is still live. prompt/IInstance.py on develop collects in writeQuestions at lines 43-48 and the file contains no preventDefault() anywhere, so the incoming question is still forwarded alongside the merged one from closing(). The lane-forwarding sweep that merged as #2063 covered anomaly_detector, ner, llm_vision_mistral, search_exa and llamaparse — not prompt.

@asclearuc asclearuc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this fix, @rishinaren — the double-forward on the questions lane is a real defect, and the comment you left explains why the line exists rather than what it does, which is the useful kind.

Update (2026-09-08): CI has fresh results. My earlier draft said the logs had expired. The run was re-triggered on this same commit, so I could read them this time. Good news first: the Shell API contract failure that an earlier comment flagged as probably-not-yours is now green, and so are Ruff, gitleaks, check-externals and the rest. The bad news is that the two failures left are caused by this change.

Request changes

1. Two existing tests fail because of this change

The same two fail on Build / Ubuntu 22.04 and on Build / Windows Server 2022. Two different platforms, identical failures, so this is not a flake. develop is green. (Build / macOS (ARM64) was still running while I wrote this. I expect the same two there.)

Both failures are the old double-emit behaviour written into fixtures. In that sense they are evidence that your fix works. But they have to be updated inside this PR.

test/test_dynamic.py::TestDynamicNodes::test_node_cases[prompt:services:default]

$questions.[1].instructions[0].instructions: Could not access property at
'[1].instructions[0].instructions': list index out of range

The prompt node's own test case reads the merged question at index [1] — nodes/src/nodes/prompt/services.json:103. Index [1] existed only because the raw forwarded question sat at [0]. Now the merged question is the only output, so the path should be [0].instructions[0].instructions.

test/test_lifecycle_order.py::test_join_flushes_after_every_branch

AssertionError: the join merged 2 entries, expected 4
assert 2 == 4

That test builds a diamond of prompt nodes. EXPECTED_JOIN_ENTRIES = 4 means "2 branches x (forwarded + flushed)" — nodes/test/test_lifecycle_order.py:97-99. You removed the forwarded half, so each branch now reaches the join once. The constant becomes 2.

Please also rewrite the module docstring at nodes/test/test_lifecycle_order.py:37-39. It states the contract as "prompt forwards what it is written straight away AND accumulates it". After this PR that sentence is false. It is the first thing the next reader of that file will believe.

One thing to confirm while you are in there. That test exists to catch a join that flushes before an upstream branch. With one entry per branch instead of two, a dropped flush shows up as 1 instead of 2. So the test still catches the regression — arguably better than before, because there is no forwarded copy left to mask it. Please say in the PR description that you checked this. Otherwise the next reader sees 4 become 2 and reads it as a weakened test.

2. No test for the new behaviour

nodes/test/test_lane_forward_once_nodes.py is the harness built for exactly this contract. It came from #2063, the lane-forwarding sweep that covered anomaly_detector, ner, search_exa, llamaparse and llm_vision_mistral — but not prompt. Adding the prompt case there finishes that sweep. Details inline.

Fixing the two tests in section 1 is necessary but not enough. Neither of them asserts this node's own delivery count.

3. Stale branch

The merge base is 2026-08-14: 24 days old, and 76 commits behind develop. Please rebase, so that the next green run means something.


Two inline comments below: the missing test, and the three sibling lanes that still default-forward.

Still open from my earlier draft: preventDefault() is called inside writeQuestions, which only runs when a question arrives. If the node can close after receiving documents but no questions, is the default forward still suppressed on that path? If the answer is that it does not matter because there is nothing to forward, that is fine — and worth a word in the comment.

# instructions, and the merged one emitted from closing(). Both are
# answered, and the first reply ignores every instruction the node
# exists to apply. Same defect and same fix as the guardrails node.
self.preventDefault()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MUST fix — add the delivery-count regression test.

nodes/test/test_lane_forward_once_nodes.py already exists for exactly this contract. It was added by #2063 ("deliver each lane payload exactly once"), which swept anomaly_detector, ner, search_exa, llamaparse and llm_vision_mistral. prompt was missed. Your fix is the same fix; the test belongs in the same file, with the same test_<node>_<lane>_delivered_once naming.

Its _simulate_engine_dispatch() models the engine rule this fix depends on: the engine forwards the incoming argument of a lane handler after the handler returns, unless the handler raised Ec.PreventDefault. That lets a test assert how many payloads reach downstream, instead of only asserting that preventDefault() was called.

Shape used by the nodes already covered there:

def test_prompt_question_delivered_once():
    inst, fake = _build('prompt', types.SimpleNamespace(config={}))
    q = FakeQuestion()
    q.addQuestion('hi')

    delivered = []
    _simulate_engine_dispatch(lambda: inst.writeQuestions(q), lambda: delivered.append(q))
    delivered.extend(fake.delivered.get('questions', []))

    assert delivered == [], 'the raw incoming question must not reach downstream'

One thing to watch: FakeQuestion in that file implements only addContext and addQuestion. If you also want to cover closing(), so that the merged question is emitted exactly once, FakeQuestion needs addInstruction and getPrompt next to them.

Why this matters more than usual here: the failure mode is silent. Two answers instead of one, no error, no traceback. A later refactor can drop this line and CI stays green.

nodes/test/guardrails/test_all.py:580 also stubs preventDefault, but it only proves the call happened. The lane-forward harness proves the delivery count, which is the real contract.

# exists to apply. Same defect and same fix as the guardrails node.
self.preventDefault()

def writeDocuments(self, documents):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should fix — the other three lanes have the same defect.

writeDocuments, writeText and writeTable do exactly what writeQuestions did before this PR: they collect the payload into self.question and return normally. The engine then default-forwards the raw documents, text or table downstream, while closing() separately emits the merged question that already contains that same content. A downstream LLM listening on both lanes therefore sees the content twice — the defect this PR fixes on the questions lane.

The guardrails node you mirror suppresses every lane it handles, not only questions: nodes/src/nodes/guardrails/IInstance.py:98 (questions), :149 (answers), :174 (documents).

If leaving documents, text and table as deliberate pass-through is the intent, that is your call — please say so in the PR description so the asymmetry is on the record. If it is not intended, self.preventDefault() belongs at the end of those three handlers too, each with its own case in test_lane_forward_once_nodes.py.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:nodes Python pipeline nodes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

prompt node forwards the original question alongside its merged one, so the LLM answers twice — missing preventDefault()

4 participants