Repository navigation
Conversation
|
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds streaming support: ChatBase exposes a chat_stream API with retry semantics; IInstance routes to a streaming path that emits SSE lm.chunk events; several providers declare SUPPORTS_STREAMING and multiple providers implement provider-specific _chat_stream methods. ChangesLLM Streaming Support
Sequence Diagram(s)sequenceDiagram
participant Client
participant IInstance
participant ChatBase
participant Provider
Client->>IInstance: ask(question)
IInstance->>IInstance: check SUPPORTS_STREAMING & expectJson
alt streaming path
IInstance->>ChatBase: chat_stream(question,on_chunk)
ChatBase->>Provider: _chat_stream_with_retries(prompt,on_chunk)
Provider->>ChatBase: on_chunk(text) (multiple)
ChatBase->>IInstance: on_chunk forwarded
IInstance->>Client: sendSSE(llm.chunk,nodeId,seq)
Provider-->>ChatBase: final text
ChatBase-->>IInstance: Answer
IInstance-->>Client: final Answer
else non-streaming path
IInstance->>Provider: chat(question)
Provider-->>IInstance: Answer
IInstance-->>Client: Answer
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
No description provided. |
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
nodes/src/nodes/llm_xai/xai.py (1)
60-62:⚠️ Potential issue | 🟠 Major | ⚡ Quick winRemove duplicate
super().__init__call in constructor.
ChatBaseis initialized twice (Line 48 and Line 61). This can re-run base setup and overwrite state unexpectedly.Proposed fix
# API key validation logic if not apikey or not apikey.startswith('xai-'): raise ValueError('Invalid XAI API key format, please check your API key.') - - # Init the chat base - super().__init__(provider, connConfig, bag)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@nodes/src/nodes/llm_xai/xai.py` around lines 60 - 62, The constructor calls the base class initializer twice which re-runs ChatBase setup and can overwrite state; remove the duplicate super().__init__(provider, connConfig, bag) so ChatBase is only initialized once (keep the original call where the base is first initialized and delete the later redundant call in the XAI constructor that re-invokes super().__init__).nodes/src/nodes/llm_ollama/ollama.py (1)
57-60:⚠️ Potential issue | 🟠 Major | ⚡ Quick winGuard
serverbasebefore calling.rstrip().
config.get('serverbase', None)can returnNone, andNone.rstrip(...)will raiseAttributeError.Proposed fix
- serverbase = config.get('serverbase', None) + serverbase = config.get('serverbase') + if not serverbase: + raise ValueError('Ollama serverbase is required.') if not serverbase.rstrip('/').endswith('/v1'): serverbase = serverbase.rstrip('/') + '/v1'🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@nodes/src/nodes/llm_ollama/ollama.py` around lines 57 - 60, The code calls serverbase.rstrip(...) without guarding against serverbase being None (from config.get('serverbase', None)); update the logic in ollama.py to check that serverbase is truthy before calling rstrip (e.g., use "if serverbase and not serverbase.rstrip('/').endswith('/v1'):"), leaving serverbase unchanged when None or empty, so you avoid AttributeError while preserving the existing normalization behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@nodes/src/nodes/llm_base/IInstance.py`:
- Around line 69-77: The except block in IInstance.chat_stream is redundantly
calling map_exception on an exception that _chat_stream_with_retries already
mapped, causing a self-referential exception chain; update the except handler to
stop remapping (remove the second call to self.IGlobal._chat.map_exception) and
simply emit(done=True, error=str(e)) and re-raise the original exception (raise)
so that you don't set e.__cause__ = e; keep emit usage and references to
self.IGlobal._chat, chat_stream, _chat_stream_with_retries and map_exception
intact elsewhere.
- Line 70: The call to chat_stream wraps emit in an unnecessary lambda; replace
the wrapper by passing the emit function directly to
self.IGlobal._chat.chat_stream (i.e., change the second argument from "lambda
delta: emit(delta)" to "emit") since emit(text, done=False, error=None) already
accepts a single positional text argument and matches the expected callback
signature.
In `@nodes/src/nodes/llm_gemini/gemini.py`:
- Line 24: Replace deprecated typing aliases: change the import line to only
import Any from typing and import Callable from collections.abc (e.g., "from
typing import Any" and "from collections.abc import Callable"), then update all
type annotations that use Dict to use the built-in dict (e.g., dict[str, Any] or
dict[Any, Any] as appropriate); ensure no other uses of
typing.Callable/typing.Dict remain in gemini.py.
In `@nodes/src/nodes/llm_ibm_watson/ibm_watson.py`:
- Line 25: Replace deprecated typing aliases in the module import: stop
importing Callable and Dict from typing; import Callable from collections.abc
and use the built-in dict type for annotations instead of typing.Dict. Update
the import line in ibm_watson.py to keep Any from typing (if needed) and ensure
all function/type annotations that used Dict[...] are changed to dict[...], and
annotations using Callable use collections.abc.Callable.
In `@nodes/src/nodes/llm_mistral/mistral.py`:
- Line 40: Replace deprecated typing aliases on the import line by importing
Callable from collections.abc and using built-in dict and tuple types: update
the import statement that currently brings in Any, Callable, Dict, Tuple from
typing, and change Callable to be imported from collections.abc while replacing
Dict and Tuple usages in type annotations within this module (mistral.py) with
dict and tuple respectively; also ensure any annotations that referenced
Callable (e.g., callback or handler parameters) use the newly imported
collections.abc.Callable and that Any remains from typing if still needed.
In `@packages/ai/src/ai/common/chat.py`:
- Around line 508-515: The except block is re-raising mapped exceptions without
preserving the original cause, losing the original traceback; update both
re-raise sites in the except block (the two occurrences of raise
self.map_exception(e)) to chain the original exception (use "raise
self.map_exception(e) from e") so the original SDK/provider traceback is
preserved; locate this logic within the chat streaming method where variables
emitted, is_retryable, attempt, and max_network_retries are used and ensure
map_exception(e) is raised with "from e" in both branches.
- Around line 499-507: The closure gated defined inside the retry loop captures
the mutable emitted dict by reference which can lead to late-binding bugs across
iterations; modify the gated definition in the loop (where emitted is created
and gated is defined) to bind the current emitted dict as a default parameter
(e.g., def gated(text: str, _emitted=emitted): ...) so gated reads/writes the
iteration-specific container, ensuring _chat_stream/on_chunk use the correct
emitted state across retries.
- Line 16: Replace deprecated typing aliases: remove Callable and Dict from the
typing import and instead import Callable from collections.abc (and use the
built-in dict type). Update all occurrences of Dict[str, Any] to dict[str, Any]
and any typing.Callable uses to collections.abc.Callable (or just Callable after
changing the import) in this file (packages/ai/src/ai/common/chat.py) so
annotations use the modern Python 3.10+ forms; adjust the from ... import line
and all annotations referencing Dict or Callable accordingly.
---
Outside diff comments:
In `@nodes/src/nodes/llm_ollama/ollama.py`:
- Around line 57-60: The code calls serverbase.rstrip(...) without guarding
against serverbase being None (from config.get('serverbase', None)); update the
logic in ollama.py to check that serverbase is truthy before calling rstrip
(e.g., use "if serverbase and not serverbase.rstrip('/').endswith('/v1'):"),
leaving serverbase unchanged when None or empty, so you avoid AttributeError
while preserving the existing normalization behavior.
In `@nodes/src/nodes/llm_xai/xai.py`:
- Around line 60-62: The constructor calls the base class initializer twice
which re-runs ChatBase setup and can overwrite state; remove the duplicate
super().__init__(provider, connConfig, bag) so ChatBase is only initialized once
(keep the original call where the base is first initialized and delete the later
redundant call in the XAI constructor that re-invokes super().__init__).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 64e5c47e-1cab-4e53-ae5c-af5b9ab02885
📒 Files selected for processing (16)
nodes/src/nodes/llm_anthropic/anthropic.pynodes/src/nodes/llm_base/IInstance.pynodes/src/nodes/llm_bedrock/bedrock.pynodes/src/nodes/llm_deepseek/deepseek.pynodes/src/nodes/llm_gemini/gemini.pynodes/src/nodes/llm_gmi_cloud/gmi_cloud.pynodes/src/nodes/llm_ibm_watson/ibm_watson.pynodes/src/nodes/llm_mistral/mistral.pynodes/src/nodes/llm_ollama/ollama.pynodes/src/nodes/llm_openai/openai_client.pynodes/src/nodes/llm_openai_api/openai_client.pynodes/src/nodes/llm_perplexity/perplexity.pynodes/src/nodes/llm_qwen/qwen_client.pynodes/src/nodes/llm_vertex/vertex.pynodes/src/nodes/llm_xai/xai.pypackages/ai/src/ai/common/chat.py
| # ============================================================================= | ||
|
|
||
| from typing import Any, Dict | ||
| from typing import Any, Callable, Dict |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial | ⚡ Quick win
Replace deprecated typing aliases with Python 3.10+ built-ins
Callable (newly added by this PR) should be imported from collections.abc; Dict should use the built-in dict. Both are flagged UP035 by Ruff.
As per coding guidelines, nodes/**/*.py targets Python 3.10+ and enforces Ruff.
♻️ Proposed fix
-from typing import Any, Callable, Dict
+from collections.abc import Callable
+from typing import AnyUpdate annotations:
-def __init__(self, provider: str, connConfig: Dict[str, Any], bag: Dict[str, Any]):
+def __init__(self, provider: str, connConfig: dict[str, Any], bag: dict[str, Any]):
-def _chat_stream(self, prompt: str, on_chunk: Callable[[str], None]) -> str:
+def _chat_stream(self, prompt: str, on_chunk: Callable[[str], None]) -> str:🧰 Tools
🪛 Ruff (0.15.12)
[warning] 24-24: Import from collections.abc instead: Callable
Import from collections.abc
(UP035)
[warning] 24-24: typing.Dict is deprecated, use dict instead
(UP035)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@nodes/src/nodes/llm_gemini/gemini.py` at line 24, Replace deprecated typing
aliases: change the import line to only import Any from typing and import
Callable from collections.abc (e.g., "from typing import Any" and "from
collections.abc import Callable"), then update all type annotations that use
Dict to use the built-in dict (e.g., dict[str, Any] or dict[Any, Any] as
appropriate); ensure no other uses of typing.Callable/typing.Dict remain in
gemini.py.
|
|
||
| import re | ||
| from typing import Any, Dict | ||
| from typing import Any, Callable, Dict |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial | ⚡ Quick win
Replace deprecated typing aliases with Python 3.10+ built-ins
Callable (newly added by this PR) should be imported from collections.abc; Dict should use the built-in dict. Both are flagged UP035 by Ruff.
As per coding guidelines, nodes/**/*.py targets Python 3.10+ and enforces Ruff.
♻️ Proposed fix
-from typing import Any, Callable, Dict
+from collections.abc import Callable
+from typing import AnyUpdate annotations:
-def __init__(self, provider: str, connConfig: Dict[str, Any], bag: Dict[str, Any]):
+def __init__(self, provider: str, connConfig: dict[str, Any], bag: dict[str, Any]):
-def _chat_stream(self, prompt: str, on_chunk: Callable[[str], None]) -> str:
+def _chat_stream(self, prompt: str, on_chunk: Callable[[str], None]) -> str:📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| from typing import Any, Callable, Dict | |
| from collections.abc import Callable | |
| from typing import Any |
🧰 Tools
🪛 Ruff (0.15.12)
[warning] 25-25: Import from collections.abc instead: Callable
Import from collections.abc
(UP035)
[warning] 25-25: typing.Dict is deprecated, use dict instead
(UP035)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@nodes/src/nodes/llm_ibm_watson/ibm_watson.py` at line 25, Replace deprecated
typing aliases in the module import: stop importing Callable and Dict from
typing; import Callable from collections.abc and use the built-in dict type for
annotations instead of typing.Dict. Update the import line in ibm_watson.py to
keep Any from typing (if needed) and ensure all function/type annotations that
used Dict[...] are changed to dict[...], and annotations using Callable use
collections.abc.Callable.
| depends(requirements) | ||
|
|
||
| from typing import Any, Dict, Tuple | ||
| from typing import Any, Callable, Dict, Tuple |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial | ⚡ Quick win
Replace deprecated typing aliases with Python 3.10+ built-ins
Callable should be imported from collections.abc, and Dict/Tuple should use the built-in dict/tuple — all three are flagged UP035 by Ruff on this line. Callable and Tuple are newly added by this PR.
As per coding guidelines, nodes/**/*.py targets Python 3.10+ and enforces Ruff.
♻️ Proposed fix
-from typing import Any, Callable, Dict, Tuple
+from collections.abc import Callable
+from typing import AnyThen update all affected type annotations in this file:
-def __init__(self, provider: str, connConfig: Dict[str, Any], bag: Dict[str, Any]):
+def __init__(self, provider: str, connConfig: dict[str, Any], bag: dict[str, Any]):
-def _getRetryConfig(self, model: str) -> Tuple[int, float]:
+def _getRetryConfig(self, model: str) -> tuple[int, float]:
-def _chat_stream(self, prompt: str, on_chunk: Callable[[str], None]) -> str:
+def _chat_stream(self, prompt: str, on_chunk: Callable[[str], None]) -> str:🧰 Tools
🪛 Ruff (0.15.12)
[warning] 40-40: Import from collections.abc instead: Callable
Import from collections.abc
(UP035)
[warning] 40-40: typing.Dict is deprecated, use dict instead
(UP035)
[warning] 40-40: typing.Tuple is deprecated, use tuple instead
(UP035)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@nodes/src/nodes/llm_mistral/mistral.py` at line 40, Replace deprecated typing
aliases on the import line by importing Callable from collections.abc and using
built-in dict and tuple types: update the import statement that currently brings
in Any, Callable, Dict, Tuple from typing, and change Callable to be imported
from collections.abc while replacing Dict and Tuple usages in type annotations
within this module (mistral.py) with dict and tuple respectively; also ensure
any annotations that referenced Callable (e.g., callback or handler parameters)
use the newly imported collections.abc.Callable and that Any remains from typing
if still needed.
| import json | ||
| import importlib | ||
| from typing import Dict, Any | ||
| from typing import Dict, Any, Callable |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial | ⚡ Quick win
Replace deprecated typing aliases (Ruff UP035)
Callable and Dict are imported from typing. For Python 3.10+ (the project's declared target), Callable should come from collections.abc and Dict should be the built-in dict.
♻️ Proposed fix
-from typing import Dict, Any, Callable
+from collections.abc import Callable
+from typing import AnyUpdate all Dict[str, Any] annotations in this file to dict[str, Any].
🧰 Tools
🪛 Ruff (0.15.12)
[warning] 16-16: Import from collections.abc instead: Callable
Import from collections.abc
(UP035)
[warning] 16-16: typing.Dict is deprecated, use dict instead
(UP035)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai/src/ai/common/chat.py` at line 16, Replace deprecated typing
aliases: remove Callable and Dict from the typing import and instead import
Callable from collections.abc (and use the built-in dict type). Update all
occurrences of Dict[str, Any] to dict[str, Any] and any typing.Callable uses to
collections.abc.Callable (or just Callable after changing the import) in this
file (packages/ai/src/ai/common/chat.py) so annotations use the modern Python
3.10+ forms; adjust the from ... import line and all annotations referencing
Dict or Callable accordingly.
| for attempt in range(max_network_retries): | ||
| emitted = {'value': False} | ||
|
|
||
| def gated(text: str) -> None: | ||
| emitted['value'] = True | ||
| on_chunk(text) | ||
|
|
||
| try: | ||
| return self._chat_stream(prompt, gated) |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial | ⚡ Quick win
gated closure captures emitted by reference — Ruff B023
gated is re-defined each loop iteration and closes over emitted via late binding. If the loop executes a second iteration before any outstanding gated reference is consumed (e.g., if _chat_stream were ever adapted to an async/generator pattern), gated from iteration N would read emitted from iteration N+1, silently defeating the "did we emit?" gate. Ruff B023 flags this directly.
The fix is to bind the current iteration's dict at function-definition time via a default argument:
♻️ Proposed fix
- def gated(text: str) -> None:
- emitted['value'] = True
- on_chunk(text)
+ def gated(text: str, _emitted: dict = emitted) -> None:
+ _emitted['value'] = True
+ on_chunk(text)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| for attempt in range(max_network_retries): | |
| emitted = {'value': False} | |
| def gated(text: str) -> None: | |
| emitted['value'] = True | |
| on_chunk(text) | |
| try: | |
| return self._chat_stream(prompt, gated) | |
| for attempt in range(max_network_retries): | |
| emitted = {'value': False} | |
| def gated(text: str, _emitted: dict = emitted) -> None: | |
| _emitted['value'] = True | |
| on_chunk(text) | |
| try: | |
| return self._chat_stream(prompt, gated) |
🧰 Tools
🪛 Ruff (0.15.12)
[warning] 503-503: Function definition does not bind loop variable emitted
(B023)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai/src/ai/common/chat.py` around lines 499 - 507, The closure gated
defined inside the retry loop captures the mutable emitted dict by reference
which can lead to late-binding bugs across iterations; modify the gated
definition in the loop (where emitted is created and gated is defined) to bind
the current emitted dict as a default parameter (e.g., def gated(text: str,
_emitted=emitted): ...) so gated reads/writes the iteration-specific container,
ensuring _chat_stream/on_chunk use the correct emitted state across retries.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/ai/src/ai/common/chat.py`:
- Around line 550-555: The streaming path currently wraps getTokens(response) in
a silent try/except which hides tokenizer errors and diverges from chat_string;
change that to catch Exception as e and log the failure (e.g. call debug with a
clear message including e) instead of swallowing it, so failures in
getTokens(response) for the streaming branch are visible; keep the existing
logic using prompt_tokens, result_tokens and self._modelTotalTokens but replace
the bare except with a narrowed catch that logs the error (referencing
getTokens, response, result_tokens, prompt_tokens and self._modelTotalTokens).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 37735c0f-b0e6-4306-815f-8454d119c9c1
📒 Files selected for processing (2)
nodes/src/nodes/llm_base/IInstance.pypackages/ai/src/ai/common/chat.py
|
Thanks @AditM99 for pursuing incremental LLM output to the UI. This draft targets the former |
Summary
Streams the LLM output to the user so user does not need to wait to see the output until the entire output is generated
Type
feature
Testing
./builder testpassesChecklist
Linked Issue
Fixes #730
Summary by CodeRabbit
New Features
Bug Fixes / Resilience