Skip to content

perf(orchestration): parallel multi-query fan-out and wave context sliding window - #869

Closed
2001anshkaushik wants to merge 7 commits into
rocketride-org:developfrom
2001anshkaushik:feat/RR-orchestration-parallel-retrieval
Closed

2001anshkaushik wants to merge 7 commits into
rocketride-org:developfrom
2001anshkaushik:feat/RR-orchestration-parallel-retrieval

Conversation

@2001anshkaushik

@2001anshkaushik 2001anshkaushik commented May 13, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR hardens the orchestration layer by addressing two critical bottlenecks in agent workflows: I/O latency and context bloat.

1. Parallel Multi-Query Fan-Out (DocumentStoreBase)

  • The Problem: Multi-question RAG queries previously executed sequentially, blocking the event loop on every vector DB round-trip.
  • The Fix: Replaced the sequential for loop with a ThreadPoolExecutor fan-out. Semantic/Keyword queries now execute concurrently, with a threading.Lock serializing the final dictionary merge. This drops latency from N * p99 to ~p99.
  • Resilience: Includes explicit zero-query tolerance (if not queries: pass) to prevent max_workers=0 crashes, and guarantees in-flight thread cleanup via context managers.

2. Server-Side Sliding Window (RocketRideDriver)

  • The Problem: The waves context previously grew infinitely unless the LLM cooperatively emitted a remove signal, risking context-window overflows.
  • The Fix: Implemented a dual-strategy _prune_wave_context() guard enforcing a sliding window (context_window_waves) and a character budget (wave_context_budget_chars) before every prompt compilation, ensuring context remains mathematically bounded.

Type

performance, enhancement

Testing

  • Tests added or updated (26 new tests in test_document_store_parallel.py, covering concurrency stress tests and zero-query edge cases)
  • Tested locally
  • ./builder test passes cleanly

Checklist

  • Commit messages follow conventional commits
  • No secrets or credentials included
  • 0-blast-radius for single-query pipelines

Summary by CodeRabbit

  • New Features

    • Automatic wave-history context pruning with configurable window size and per-window character budget to keep planning context bounded.
    • Parallelized document-store sub-query execution with a capped concurrency limit to speed multi-query searches and preserve correct aggregation.
  • Tests

    • New test suites covering parallel query fan-out, aggregation, de-duplication, edge cases, concurrency/thread-safety, and context-pruning behavior; includes a test-time stub for an external dependency.

Review Change Stack

…iding window

Gap-2: Parallel multi-query fan-out in DocumentStoreBase._queryDocuments
- Replace sequential for-loop over question.questions with ThreadPoolExecutor
  dispatch for SEMANTIC/PROMPT/QUESTION and KEYWORD question types
- Single-query path bypasses executor entirely (zero overhead for 1 sub-query)
- Concurrent writes to the shared documents dict serialised via threading.Lock
  held only during the merge step, not during the blocking I/O call itself
- Embedding validation runs before any futures are submitted (fail-fast)
- _MAX_PARALLEL_QUERIES = 8 caps thread count for large fan-out pipelines
- Expected latency: N x p99 -> ~p99 (e.g. 3x200ms: 600ms -> ~210ms)

Gap-1: Server-side sliding-window context budget in RocketRideDriver
- _prune_wave_context() with two complementary eviction strategies:
  1. Sliding window (context_window_waves, default 5)
  2. Character budget (wave_context_budget_chars, default 12_000)
- Minimum 1 wave always retained to prevent empty-context LLM calls
- Both configurable via node config fields; budget=0 disables char limit
- Called after every execute_wave append

Tests: 20 new tests in test_document_store_parallel.py (20/20 pass)
- Correctness, parallelism timing, thread safety stress test, edge cases
Defect 1 (Critical): Empty SEMANTIC/PROMPT/QUESTION query list crashed with
  ValueError: max_workers must be greater than 0.
  The 'else' branch (intended for N>1 queries) was entered when len(queries)==0,
  computing min(8, 0)=0 and passing that to ThreadPoolExecutor.
  Fix: add explicit 'if not queries: pass' guard before the == 1 / else split,
  making the three paths: empty → no-op, single → direct call, N>1 → executor.

Defect 2 (Critical): Same max_workers=0 crash for KEYWORD type with empty
  question list. Same fix applied to the KEYWORD branch.

Defect 3 (Logic): The KEYWORD branch was written as an independent 'if' in the
  original code, making KEYWORD and GET true peers. The refactor preserved this
  structure correctly for the KEYWORD 'if', but the relationship to the GET
  'elif' was ambiguous and would have caused GET to silently no-op if a future
  refactor collapsed the two 'if' blocks into an 'elif' chain. Added explicit
  structural comment explaining WHY KEYWORD must remain an independent 'if' and
  must never be made 'elif' off the SEMANTIC block.

  Confirmed: GET is currently reachable because the KEYWORD 'if' is False when
  type==GET, so the elif GET arm fires correctly. But the zero-query KEYWORD
  path fell to else:pass instead of elif GET — this silent no-op was the latent
  bug. After fix: KEYWORD(queries=[]) → returns {} and never touches GET branch.

Tests: 6 regression tests added to TestPreFlightAuditRegressions (26/26 pass)
  - test_semantic_empty_questions_no_crash
  - test_prompt_empty_questions_no_crash
  - test_keyword_empty_questions_no_crash
  - test_keyword_empty_does_not_call_search
  - test_get_type_is_reachable
  - test_get_type_not_affected_by_keyword_zero_queries
@github-actions github-actions Bot added module:nodes Python pipeline nodes module:ai AI/ML modules labels May 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

No description provided.

@coderabbitai

coderabbitai Bot commented May 13, 2026 •

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

Adds configurable wave-history pruning to RocketRideDriver (sliding-window then character-budget eviction, returned as a shallow copy used only for planning prompts) and refactors DocumentStoreBase._queryDocuments to parallelize multi-sub-query SEMANTIC/KEYWORD execution with a max-worker cap and pre-dispatch embedding validation. Adds tests and a pytest rocketlib stub.

Changes

Performance and Context Management

Layer / File(s) Summary
Wave context pruning in RocketRideDriver
nodes/src/nodes/agent_rocketride/rocketride_agent.py
Load context_window_waves and wave_context_budget_chars from config; implement _prune_wave_context(waves) to return a shallow pruned copy applying sliding-window eviction then character-budget eviction; compute context_waves = _prune_wave_context(waves) at each planning iteration and pass it into plan_wave without mutating the master waves/trace.
Parallel sub-query execution in DocumentStore
packages/ai/src/ai/common/store.py
Add ThreadPoolExecutor imports and class attribute _MAX_PARALLEL_QUERIES; refactor _queryDocuments to validate embedding_model before dispatch and to run embedding-based (PROMPT/SEMANTIC/QUESTION) and KEYWORD sub-queries in parallel when multiple sub-queries exist, merging results under a lock; keep single-sub-query and GET direct paths.
Tests and pytest stub
conftest.py, nodes/test/agent_rocketride/test_prune_wave_context.py, packages/ai/tests/ai/common/test_document_store_parallel.py
Install an in-repo rocketlib test stub and add pytest modules: RocketRideDriver pruning tests (window, budget, combined, non-mutating copy) and DocumentStoreBase parallel fan-out tests (aggregation, de-duplication by best score, threshold filtering, embedding prevalidation, exception propagation, concurrency determinism, and stress tests).

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant DocumentStoreBase
  participant ThreadPoolExecutor
  participant SearchBackend as searchSemantic/searchKeyword
  Caller->>DocumentStoreBase: _queryDocuments(question)
  DocumentStoreBase->>DocumentStoreBase: validate embedding_model (pre-flight)
  DocumentStoreBase->>ThreadPoolExecutor: submit sub-query tasks
  ThreadPoolExecutor->>SearchBackend: execute sub-query
  SearchBackend-->>DocumentStoreBase: sub-query results
  DocumentStoreBase->>DocumentStoreBase: merge results under lock
  DocumentStoreBase-->>Caller: aggregated documents
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Suggested reviewers

  • jmaionchi
  • Rod-Christensen
  • stepmikhaylov

Poem

🐰 I pruned the waves to keep thoughts light,
I kept the newest echoes in plain sight,
Threads sprang forth to fetch each matching line,
Merged their finds into a single spine,
Tests hopped in — the server sleeps tight.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 52.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly and clearly describes the two main changes: parallel multi-query fan-out and wave context sliding window, matching the core objectives and substantial code changes in the PR.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@nodes/src/nodes/agent_rocketride/rocketride_agent.py`:
- Around line 281-286: The current code prunes the in-place `waves` list via
`_prune_wave_context(waves)`, which loses data needed for `trace['waves']` and
the fallback `_synthesize(...)`; change the flow to keep an unmodified full
history (e.g., `full_waves = waves` or `history_waves`) and only pass a pruned
copy to planning: call something like `planning_waves =
self._prune_wave_context(list(full_waves))` (or have `_prune_wave_context`
return a pruned copy), then use `planning_waves` for `plan_wave()` while leaving
`full_waves` intact for `trace['waves']` and `_synthesize(...)`; ensure all
references that produce the trace or call `_synthesize` use the unpruned
`full_waves`.

In `@packages/ai/tests/ai/common/test_document_store_parallel.py`:
- Around line 339-345: The test currently validates a local reimplementation
(_DriverStub._prune_wave_context) instead of the real node, so update the tests
to import and exercise the actual RocketRideDriver from
nodes/src/nodes/agent_rocketride/rocketride_agent.py and exercise its
integration path (e.g., call RocketRideDriver._run or public run entry) instead
of using _DriverStub; if the suite cannot import nodes, either move these test
cases to a test target/config that has nodes on PYTHONPATH or make the nodes
package importable from this test suite, then delete the stubbed
_prune_wave_context usage and assert behavior via the real RocketRideDriver.
- Around line 202-220: The test's rigid bound (elapsed < 2 *
SIMULATED_QUERY_LATENCY_S) is too tight and causes CI flakes; change the
assertion to compare against a sequential baseline measured in the same test
run: measure sequential_time by calling store._queryDocuments(question)
n_queries times in a loop (using the same question and store) to get a realistic
sequential_elapsed, then assert that the parallel elapsed measured in
test_parallel_execution_is_faster_than_sequential is significantly less than
that sequential_elapsed (e.g., elapsed < sequential_elapsed * 0.9) instead of
comparing to 2 * SIMULATED_QUERY_LATENCY_S.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c5106df0-8675-45bd-a039-f5c65adcbaec

📥 Commits

Reviewing files that changed from the base of the PR and between c5cde38 and 358232a.

📒 Files selected for processing (3)
  • nodes/src/nodes/agent_rocketride/rocketride_agent.py
  • packages/ai/src/ai/common/store.py
  • packages/ai/tests/ai/common/test_document_store_parallel.py

…ting patterns

Fix 1 — State mutation: _prune_wave_context now returns a pruned copy
  - _prune_wave_context(waves) -> List: returns a shallow copy with window
    and char-budget eviction applied; the master waves list is never touched
  - _run() computes context_waves = _prune_wave_context(waves) at the top of
    each planning iteration and passes context_waves to plan_wave()
  - master waves (= trace['waves']) grows unbounded as the canonical audit
    trail; _synthesize() at max-wave fallback still receives the full trace
  - Removed the post-execute_wave prune call which was the mutation site

Fix 2 — CI flake: replace wall-clock timing assertion with Barrier proof
  - Deleted test_parallel_execution_is_faster_than_sequential (wall-clock
    comparisons are non-deterministic on loaded CI runners / Windows scheduler)
  - Added test_parallel_execution_is_structurally_concurrent: uses
    threading.Barrier(n=3, timeout=5s) to prove all 3 futures execute
    concurrently — a serial executor would deadlock and raise BrokenBarrierError
  - This proof is deterministic and immune to OS scheduler jitter

Fix 3 — Stub anti-pattern: replace _DriverStub with real RocketRideDriver test
  - Deleted TestWaveSlidingWindow from test_document_store_parallel.py (it
    tested a verbatim stub copy of the algorithm, not the production class)
  - Created nodes/test/agent_rocketride/test_prune_wave_context.py following
    the established project pattern (cf. nodes/test/astra_db/test_convert_filter.py)
    — uses RocketRideDriver.__new__ to bypass iGlobal init, sets only
    _context_window_waves and _wave_context_budget_chars, then calls the
    real production method
  - 4 new test classes: TestReturnsCopy (copy semantics), TestSlidingWindow,
    TestCharacterBudget, TestCombinedStrategies

Results:
  packages/ai tests: 16/16 passed (0.67s)
  nodes/test/agent_rocketride: 19/19 passed (2.17s)

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@nodes/test/agent_rocketride/test_prune_wave_context.py`:
- Around line 77-115: The test file contains Unicode multiplication signs (×,
U+00D7) in comments (e.g., in
TestReturnsCopy::test_master_list_never_mutated_by_budget_eviction and other
similar tests) which Ruff flags; replace those characters with the ASCII
asterisk (*) (or "x") in the comment text so the comments read e.g. "3 waves *
100 chars = 300 chars → budget eviction will fire" to avoid encoding/ambiguity
issues while leaving all test logic and identifiers like _prune_wave_context,
TestReturnsCopy, and the named test methods unchanged.
- Line 25: Replace the typing imports and usages with Python 3.10+ native
generics: remove "from typing import Any, Dict, List" and switch all type
annotations that use Dict and List to lowercase forms (e.g., change Dict[str,
Any] to dict[str, Any] and List[Dict[str, Any]] to list[dict[str, Any]]); update
any function signatures, variable annotations or test fixtures in this file that
reference Dict, List, or Any accordingly so only "Any" usage remains either
imported or replaced with concrete types if possible.

In `@packages/ai/tests/ai/common/test_document_store_parallel.py`:
- Line 24: Replace the legacy typing imports and usages: remove "from typing
import Any, Callable, Dict, List, Optional, Tuple" and instead import Any and
Optional from typing only if needed, import Callable from collections.abc, then
update all type annotations to use native generics (e.g., change Dict[str, Any]
→ dict[str, Any], List[Doc] → list[Doc], Tuple[str, int] → tuple[str, int]) and
replace typing.Callable usages with collections.abc.Callable; update any
occurrences of those symbols in the test file (references like Dict, List,
Tuple, Callable) to the new forms so the test uses Python 3.10+ native generics.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 8d9dbca8-66a1-4d53-a078-955737a22c60

📥 Commits

Reviewing files that changed from the base of the PR and between 358232a and 17e2c5a.

📒 Files selected for processing (3)
  • nodes/src/nodes/agent_rocketride/rocketride_agent.py
  • nodes/test/agent_rocketride/test_prune_wave_context.py
  • packages/ai/tests/ai/common/test_document_store_parallel.py

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/ai/tests/ai/common/test_document_store_parallel.py`:
- Around line 38-47: The current stub unconditionally inserts a minimal
_rocketlib module using sys.modules.setdefault which can leave tests with an
incomplete module if another test later expects additional attributes; instead,
fetch any existing module = sys.modules.get('rocketlib') and if none assign the
new ModuleType, otherwise merge missing attributes: ensure _IInstanceBase is
added only when getattr(existing, 'IInstanceBase', None) is None (use setattr
for 'IInstanceBase'), so the stub becomes order-independent (update the code
around _rocketlib, _IInstanceBase, and the sys.modules.setdefault call to
perform this safe-merge logic).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9c4903d0-2d76-40c8-93d7-1b2e9813c138

📥 Commits

Reviewing files that changed from the base of the PR and between 17e2c5a and 2177c4b.

📒 Files selected for processing (2)
  • nodes/test/agent_rocketride/test_prune_wave_context.py
  • packages/ai/tests/ai/common/test_document_store_parallel.py

…st modules

Root cause
----------
Both test files injected rocketlib stubs inline in their module bodies using
sys.modules.setdefault().  The first file collected by pytest wins; the second
file's setdefault() is a no-op.  If test_document_store_parallel.py loaded
first, it registered only IInstanceBase — missing tool_function and
rocketlib.types — causing an ImportError when test_prune_wave_context.py
subsequently imported RocketRideDriver.

The plan's proposed fix (copy the comprehensive stub into both files) was
rejected because it creates two copies of the same module stub.  Any new
symbol added to the rocketlib interface would require updating two places —
exactly the drift that leads to 'works on my machine' CI failures.

Fix
---
Move the stub to conftest.py at the repository root.  pytest guarantees that
conftest.py files are imported before any test module body executes,
regardless of collection order or parallelism.  conftest.py is the canonical
location for session-scoped setup code in pytest projects.

  conftest.py (NEW) — single source of truth for all rocketlib stubs:
    - rocketlib.IInstanceBase  — no-op base class
    - rocketlib.tool_function  — no-op decorator (bare and called forms)
    - rocketlib.types          — sub-module with IInvokeOp/IInvokeTool/
                                 IInvokeMemory stubs

  test_document_store_parallel.py — inline stub removed; replaced with a
    comment pointing to conftest.py explaining the pattern prohibition.

  test_prune_wave_context.py — same removal; same explanatory comment.

Verification
------------
Both orderings confirmed green:
  store first:  35/35 passed (0.85s)
  prune first:  35/35 passed (0.78s)

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/ai/tests/ai/common/test_document_store_parallel.py`:
- Line 36: Remove the unused import of the sys module in the test file: delete
the line importing sys from
packages/ai/tests/ai/common/test_document_store_parallel.py (the standalone
"import sys" at top of the file) since sys is not referenced elsewhere in the
file.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c3a7244d-8c5e-4956-a8cd-7c1fd12c76e5

📥 Commits

Reviewing files that changed from the base of the PR and between 2177c4b and 94717dc.

📒 Files selected for processing (3)
  • conftest.py
  • nodes/test/agent_rocketride/test_prune_wave_context.py
  • packages/ai/tests/ai/common/test_document_store_parallel.py

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/ai/tests/ai/common/test_document_store_parallel.py`:
- Around line 197-201: Test assumes 3 concurrent threads; change n_queries to
honor the configured worker cap by setting n_queries = min(3,
_MAX_PARALLEL_QUERIES) (or read the executor's max workers if available) before
creating threading.Barrier, and use that variable when instantiating barrier so
the rendezvous count never exceeds the executor's worker limit (adjust imports
or access to _MAX_PARALLEL_QUERIES from the module where it's defined if
necessary).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: ebce3529-2bf1-490f-afbf-1a4d20baddc5

📥 Commits

Reviewing files that changed from the base of the PR and between 94717dc and b93d047.

📒 Files selected for processing (1)
  • packages/ai/tests/ai/common/test_document_store_parallel.py

@2001anshkaushik

Copy link
Copy Markdown
Collaborator Author

@coderabbitai resume

@coderabbitai

coderabbitai Bot commented May 13, 2026

Copy link
Copy Markdown
Contributor
✅ Actions performed

Reviews resumed.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
packages/ai/tests/ai/common/test_document_store_parallel.py (1)

197-201: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Guard against vacuous concurrency proof when worker cap is 1.

At Line 197, Barrier(1) can pass immediately, so the test may succeed without proving parallel dispatch. Skip when max workers are below 2.

♻️ Suggested fix
-        n_queries = min(3, DocumentStoreBase._MAX_PARALLEL_QUERIES)
+        max_workers = DocumentStoreBase._MAX_PARALLEL_QUERIES
+        if max_workers < 2:
+            pytest.skip('Structural parallelism is not expected when max workers < 2')
+        n_queries = min(3, max_workers)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/ai/tests/ai/common/test_document_store_parallel.py` around lines 197
- 201, The test creates a threading.Barrier sized by n_queries derived from
DocumentStoreBase._MAX_PARALLEL_QUERIES which is vacuous when that cap is 1;
guard the test by checking DocumentStoreBase._MAX_PARALLEL_QUERIES and skip the
test early if it's less than 2 (e.g. call pytest.skip with a brief message)
before computing n_queries and constructing the barrier so the barrier check
actually validates parallel dispatch.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@packages/ai/tests/ai/common/test_document_store_parallel.py`:
- Around line 197-201: The test creates a threading.Barrier sized by n_queries
derived from DocumentStoreBase._MAX_PARALLEL_QUERIES which is vacuous when that
cap is 1; guard the test by checking DocumentStoreBase._MAX_PARALLEL_QUERIES and
skip the test early if it's less than 2 (e.g. call pytest.skip with a brief
message) before computing n_queries and constructing the barrier so the barrier
check actually validates parallel dispatch.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6feafd7d-48fe-4c19-8182-f5c51475ca34

📥 Commits

Reviewing files that changed from the base of the PR and between 94717dc and 4238999.

📒 Files selected for processing (1)
  • packages/ai/tests/ai/common/test_document_store_parallel.py

@Rod-Christensen

Copy link
Copy Markdown
Collaborator

Wave was intended to operate exactly how you implemented it, running multiple subnodes concurrently. However, a problem came up in testing that needs to be fixed in the c++ engine before we can merge. It is a corner case, but it is pretty serious..

Most python nodes assume the sequential architecture. However, this makes them parallel which has the capability to corrupt "temporary" data that they are holding. The solution is, in the C++ engine, we need to ensure that no node is shared between wave partners, or lock a mutex so a node runs to completion prior to returning. Either way, it requires a backend change. Neither is too terfibly difficuly.

@dsapandora dsapandora self-assigned this Jul 7, 2026

@dsapandora dsapandora left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR should not merge yet. The wave feature is implemented and scoped correctly at the store layer — fan-out via ThreadPoolExecutor in _queryDocuments with a properly-scoped merge lock, and _prune_wave_context() returning a fresh copy per planning iteration instead of mutating waves in place (fixing CodeRabbit's line-286 flag while keeping trace['waves'] and the synthesis fallback intact). The Barrier-based concurrency test is a solid replacement for a flaky timing check, and the invariant tests around 'master list never mutated' are the right shape for this kind of change.

However, testing surfaced a serious corner case that blocks this PR: most Python nodes were written assuming the engine executes sequentially, and several hold temporary/mutable state during execution. Running wave partners concurrently — which is exactly what this PR correctly implements — exposes those nodes to being shared across concurrent wave partners, which can corrupt that temporary state. This is a real regression introduced by turning previously-sequential execution parallel, and it isn't something the current Python-side tests exercise: the concurrency proof and invariant tests here cover the document store's merge safety, not node-level state isolation.

The fix has to land in the C++ engine, not in this PR's Python code: either (a) ensure no node instance is shared between wave partners (give each wave its own node instances), or (b) hold a per-node mutex so a node runs to completion before it can be re-entered by another wave. Either approach looks workable and shouldn't be a heavy lift, but it's a backend change and a prerequisite for merging wave-parallel execution safely — without it, this corner case can produce corrupted results in production under the exact concurrency this PR enables. New tests exercising node-level state isolation (or mutex serialization) should land alongside whichever fix is chosen, since the current suite doesn't cover this failure mode on any platform.

Once that's addressed, the remaining items are minor and non-blocking:

  • The SEMANTIC and KEYWORD fan-out blocks in store.py are near-identical; worth a shared helper if this file is touched again.
  • The 'Defect 3' framing in test_document_store_parallel.py (GET allegedly unreachable via the KEYWORD elif) isn't an actual bug before or after this change — type is a single enum value, so the branch was always correctly reachable regardless of query count. Worth tightening that docstring.
  • _prune_wave_context only sums results[].summary lengths, not the calls payload, so a wave with large tool-call args could still blow past the intended context budget. Worth watching if context-overflow issues resurface.

Given this has apparently been sitting since around May 26, prioritizing the C++ side fix quickly would help — otherwise this is a candidate to close and reopen once the engine change is ready, rather than left open indefinitely.

@charliegillet

Copy link
Copy Markdown
Contributor

Closing per the close-and-reopen suggestion in review rather than leaving this parked. The design was endorsed by both reviewers and the feature is still wanted, but it is hard-blocked on engine-side wave-partner node isolation (per-wave instances or a per-node mutex), which needs to land first and is not yet tracked anywhere. The store refactor in #1663 has also since replaced store.py, so the branch no longer applies. Plan: track the engine prerequisite separately, split off the wave-context pruning half (which does not depend on the concurrency fix) as a small standalone PR, and remake the multi-query fan-out against the new store/ package once the engine work lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:ai AI/ML modules module:nodes Python pipeline nodes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants