Skip to content

Feat/730 llm streaming chunk - #889

Closed
dsapandora wants to merge 8 commits into
developfrom
feat/730-llm-streaming-chunk
Closed

dsapandora wants to merge 8 commits into
developfrom
feat/730-llm-streaming-chunk

Conversation

@dsapandora

@dsapandora dsapandora commented May 15, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Captura de pantalla 2026-06-10 a las 10 16 46

Adds a unified streaming path that delivers both the visible answer and the model's chain-of-thought to the chat UI as Server-Sent Events, for every LLM provider we ship.

  • Per-token streaming for the answer (chunk SSE).
  • Per-token streaming for the reasoning summary (reasoning_chunk SSE), shown in a collapsible "Thinking…" panel that auto-collapses once the answer starts.
  • chunk_end carries the provider's finishReason so the UI can flag truncation or errors.
  • Transparent fallback to non-streaming invoke() if the provider rejects streaming or yields nothing — the user always gets an answer.

See packages/ai/src/ai/common/STREAMING.md for the full architecture and a mermaid diagram.

Type

feature

Testing

  • Tests added or updated
  • Tested locally
  • ./builder test passes

Checklist

  • Commit messages follow conventional commits
  • No secrets or credentials included
  • Wiki updated (if applicable)
  • Breaking changes documented (if applicable)

Linked Issue

Applies discussion #752

Summary by CodeRabbit

  • New Features

    • Collapsible "Thinking" panels show streamed model reasoning above bot replies and auto-collapse when reasoning finishes (unless user-expanded).
    • Real-time incremental streaming of both reasoning and visible response text for supported models; end-of-response truncation/finish notes shown when applicable.
    • Broader model/provider coverage for streamed reasoning, with routing to preserve separate reasoning deltas.
  • Documentation

    • Added a guide describing reasoning/streaming behavior and the chat UI SSE contract.

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 15, 2026 •

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds end-to-end streamed "reasoning" support: provider-native and Responses API streaming emit SSE events; backend routes reasoning deltas and chunk/finish events; frontend types, hooks, and a collapsible ReasoningPanel render incremental reasoning content.

Changes

Streaming Reasoning Feature Implementation

Layer / File(s) Summary
Message / UI types
apps/chat-ui/src/types/chat.types.ts, packages/shared-ui/src/modules/chat/types.ts
Add optional reasoning?: string and reasoningStreaming?: boolean to message shapes.
Streaming documentation
packages/ai/src/ai/common/STREAMING.md
New guide describing dispatch paths, SSE event contract (chunk, reasoning_chunk, reasoning_end, chunk_end), sequencing, and provider strategies.
Native stream registry & handlers
packages/ai/src/ai/common/llm_native_stream.py
New native stream registry and handlers for Anthropic, Mistral, and OpenAI-compatible reasoning streams with model gating and parsing.
Native stream implementation
packages/ai/src/ai/common/llm_native_stream.py
Provider-specific streaming loops, parsing, finish-reason mapping, and retries.
ChatBase streaming & think-splitter
packages/ai/src/ai/common/chat.py
Responses API streaming path, <think> splitter, callback-based streaming (on_chunk/on_finish/on_reasoning_chunk), and native dispatch routing.
LLMBase SSE emission & identity
packages/ai/src/ai/common/llm_base.py
Forward streaming callbacks, compute stream identity, and emit SSE events with per-stream seq counters and structured error handling.
Provider detection & wiring
tools/src/core/merger.py, tools/src/stamp_reasoning.py, tools/src/providers/base.py
Detect reasoning-capable models via OpenRouter cache, stamp capabilities.reasoning, and adapt provider discovery/merger cache shapes.
Provider client reasoning wiring
nodes/src/nodes/*/*
Detect reasoning-capable models; create raw OpenAI clients or provider-specific thinking kwargs; set _native_stream_provider and _reasoning_kwargs.
Provider API updates
nodes/src/nodes/*/*
Add optional streaming callbacks to chat() signatures and call on_finish with finish reason where applicable.
Frontend SSE handlers
apps/chat-ui/src/hooks/useChatMessages.ts, packages/shared-ui/src/modules/chat/hooks/useChatMessages.ts
Rewrite SSE handling for chunk/reasoning_chunk/reasoning_end/chunk_end with per-(runId,nodeId) seq deduplication, incremental in-place updates, and skip of final extraction when streaming used.
Frontend UI components
apps/chat-ui/src/components/Message.tsx, packages/shared-ui/src/modules/chat/components/MessageBubble.tsx
Add ReasoningPanel component with collapsible state, auto-collapse on stream end, markdown rendering, and integration into bot/system messages.
Build config change
packages/shared-ui/rsbuild.config.ts
Change rsbuild entry from ./index.tsx to ./index.ts.
Services & tooling
nodes/.../services.json, tools/*, tools/scripts/tasks.js
Stamp/annotate profiles with capabilities.reasoning, add stamping script and tasks, extend OpenRouter cache/detection logic, patcher cleanup, and adjust provider discovery fallbacks.

Sequence Diagram

sequenceDiagram
  participant UI
  participant Server
  participant ChatBase
  participant Provider
  UI->>Server: sendMessage
  Server->>ChatBase: chat_string(prompt, callbacks)
  ChatBase->>Provider: stream/invoke
  Provider-->>ChatBase: delta.content
  Provider-->>ChatBase: delta.reasoning_content
  ChatBase-->>Server: SSE events (chunk / reasoning_chunk / reasoning_end / chunk_end)
  Server-->>UI: SSE events delivered
  UI->>UI: update message.text / message.reasoning
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

module:ai, module:nodes, module:ui

Suggested reviewers

  • jmaionchi
  • stepmikhaylov
  • Rod-Christensen

Poem

🐰 I am a rabbit who thinks aloud,
Streams of thought hop soft and proud,
Deltas bloom and panels fold,
Reasoning whispers, brave and bold,
Hooray — the thinking hops into view!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 72.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Feat/730 llm streaming chunk' accurately reflects the main change: implementing per-token streaming for LLM responses delivered as Server-Sent Events.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/730-llm-streaming-chunk

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added module:nodes Python pipeline nodes module:ai AI/ML modules module:ui Chat UI and Dropper UI labels May 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

No description provided.

@github-actions github-actions Bot added the docs Documentation label May 15, 2026
@dsapandora
dsapandora force-pushed the feat/730-llm-streaming-chunk branch from a1f5204 to d225473 Compare May 15, 2026 17:25
@dsapandora
dsapandora marked this pull request as ready for review May 15, 2026 17:31

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 17

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/chat-ui/src/components/Message.tsx`:
- Around line 49-55: The disclosure button in Message.tsx (the button that
toggles isOpen and calls setUserToggled/setIsOpen and renders the
thinking-chevron and thinking-label based on streaming/isOpen) should explicitly
set type="button" to avoid implicit form submission and add aria-expanded
reflecting the current isOpen state (e.g., aria-expanded={isOpen}) to follow the
disclosure pattern and match MessageBubble.tsx; update the button element
attributes accordingly so accessibility and behavior are consistent.

In `@apps/chat-ui/src/hooks/useChatMessages.ts`:
- Around line 105-117: The streaming bubble currently uses Date.now() for
streamingId (in ensureBubble) which can collide; replace Date.now() with the
module-level monotonic id generator used elsewhere (call nextId()) and import it
into this hook, update ensureBubble to set streamingId = nextId(), use that id
when calling setMessages and when doing in-place streaming updates (the map that
compares m.id !== id), and also replace other places in this file that mint ids
with Date.now() (including the status fallback logic that currently mints an id)
to use nextId() so all message ids here are monotonic and collision-free.
- Around line 156-168: The chunk_end branch currently appends a truncation/end
note but does not clear the reasoningStreaming flag, leaving the ReasoningPanel
stuck in "Thinking…"; update the 'chunk_end' handler inside useChatMessages so
that after you append the note (or when streamingId !== null / when a finish
reason is present) you also call the setter that flips reasoningStreaming to
false (e.g., invoke setReasoningStreaming(false) or update the
reasoningStreaming state used by ReasoningPanel) alongside the existing
setMessages call, ensuring the UI collapses/clears the thinking state when a
stream ends.
- Around line 105-135: The current ensureBubble helper always returns an id so
the chunk-dedup check (m.text === delta && m.reasoning === undefined) cannot
tell if ensureBubble seeded the message or the delta legitimately matches
existing content; modify ensureBubble to return an object { id, created } where
created is true when a new bubble was appended, update the onSSE 'chunk' handler
to use const { id, created } = ensureBubble({ text: delta }) and only
short-circuit (skip duplicate) when created is true, and apply the same pattern
to the reasoning_chunk handler so it checks created (since reasoning was seeded)
rather than relying on m.reasoning being undefined; this ensures later identical
deltas aren't dropped while preserving the initial-seed guard.

In `@nodes/src/nodes/llm_mistral/mistral.py`:
- Around line 217-219: The map_exception method currently wraps SDK errors in a
bare Exception which breaks the public contract used by callers (e.g.,
is_retryable_error and ChatBase) and loses the original cause; update
nodes/src/nodes/llm_mistral/mistral.py's map_exception to raise a ValueError
containing the formatted user message and chain the original exception (use
"from error") so the original SDK exception is preserved for tracebacks and
upstream error-type checks (mirror the behavior used in
qwen_client.py/openai_client.py/gmi_cloud.py).
- Around line 221-247: The code in _chat accesses
chat_response.choices[0].message.content without checking that
chat_response.choices exists and is non-empty; add a defensive guard: after
obtaining chat_response, check that getattr(chat_response, 'choices', None) is a
non-empty sequence and that the first choice has a message with content; if
choices is empty or missing, return an empty string (or raise a clearly worded
exception) so the SSE/streaming fallback can surface a meaningful finishReason,
and similarly guard access to choice.message before reading .content.

In `@nodes/src/nodes/llm_perplexity/perplexity.py`:
- Around line 232-233: The call on_finish('stop') is hardcoded and loses real
finish info; change it to extract the finish reason from the response metadata
(use the same pattern as earlier where you read results.response_metadata to get
the finish reason) and pass that to on_finish, defaulting to 'stop' only if the
metadata key is missing or None; update the call in the function/method that
invokes on_finish so it mirrors the logic used around line 195.

In `@nodes/src/nodes/llm_qwen/qwen_client.py`:
- Around line 40-59: The prefix list _THINKING_CAPABLE_PREFIXES contains
redundant entries 'qwen3.5' and 'qwen3.6' because the matching logic in
_supports_thinking (m == p or m.startswith(f'{p}-') or m.startswith(f'{p}.'))
already covers the entire qwen3.x family via 'qwen3'; remove 'qwen3.5' and
'qwen3.6' from the _THINKING_CAPABLE_PREFIXES tuple so future qwen3.* versions
are covered automatically by _supports_thinking.

In `@nodes/src/nodes/llm_vision_mistral/mistral_vision.py`:
- Around line 251-252: Replace the hardcoded 'stop' finish signal with Mistral's
actual finish reason: when calling on_finish (the callback referenced as
on_finish), extract the finish_reason from the provider response
(chat_response.choices[0].finish_reason) and pass that through; if finish_reason
is undefined/falsy fallback to a sensible default like 'stop' to preserve
current behavior. Update the call site where on_finish('stop') is invoked so it
reads the finish reason variable derived from chat_response before invoking
on_finish.

In `@nodes/src/nodes/llm_vision_ollama/ollama_vision.py`:
- Around line 154-155: The code currently calls on_finish('stop') which
hardcodes the completion reason; instead, pass the actual termination reason
from the Ollama response/stream (e.g., a variable like reason, stop_reason, or
response.get('reason')) to on_finish so truncation/timeouts are propagated to
the UI; locate the handler where on_finish is invoked in ollama_vision.py and
replace the literal 'stop' with the real completion reason pulled from the
response or event object (falling back to a sensible default only if the reason
is missing).

In `@packages/ai/src/ai/common/chat.py`:
- Around line 684-689: The chat() call currently streams the initial response
via chat_string(...) with on_chunk/on_finish/on_reasoning_chunk callbacks even
when question.expectJson is true, which causes retries that repair JSON to drop
callbacks and leaves the UI showing the first malformed streamed attempt; update
the logic in the chat() path (the code that invokes chat_string) to disable
streaming when question.expectJson is truthy — i.e., call chat_string without
streaming callbacks or with streaming disabled for JSON-expected questions — or
implement a replacement-stream mechanism that re-attaches/forwards callbacks for
retry attempts; ensure you adjust both the chat() invocation around chat_string
and the analogous block referenced earlier (the other chat_string call handling
reasoning chunks) so retries keep a single final streamed output only after a
successful JSON extraction.
- Around line 466-475: The exception handler currently always falls back to
non-streaming by calling self._llm.invoke(prompt) even after streaming callbacks
have already emitted deltas; change the logic in the except blocks (the ones
surrounding the streaming send loop that call self._llm.invoke and set
finish_reason) to only perform the non-streaming fallback when no streamed delta
was emitted yet (use the existing emitted_any boolean or similar flag),
otherwise either emit an explicit reset/replay SSE event or re-raise/log without
invoking the fallback; update both the handler near the try/except around the
streaming loop (the block that logs "Reasoning streaming disabled for
model=...") and the similar handler at the other occurrence (around lines
handling finish_reason) so final non-streaming answers are only sent if
emitted_any is False.

In `@packages/ai/src/ai/common/llm_native_stream.py`:
- Around line 242-250: The current except block unconditionally returns None
after calling _stream_anthropic_messages_api, which is incorrect if any chunks
were already emitted; change the streaming wrappers so on_chunk and
on_reasoning_chunk set a local "emitted" flag (e.g., emitted = False -> set True
inside the handlers passed to _stream_anthropic_messages_api), and in the except
handler check that flag: if emitted is False then warn and return None (safe
fallback), but if emitted is True re-raise the exception (or propagate it) so
callers like ChatBase.chat_string() do not start a duplicate request; apply the
same pattern to the other two try/except sites you called out (the other
_stream_* calls around the other blocks).

In `@packages/ai/src/ai/common/STREAMING.md`:
- Around line 121-124: The doc currently states the generic LangChain loop "no
change needed" for providers that stream reasoning_content, which contradicts
earlier lines; update the guidance to state that the generic loop does NOT
reliably preserve delta.reasoning_content and therefore providers must opt into
the OpenAI-compatible reasoning path (e.g., set SUPPORTS_REASONING_STREAMING =
True, declare _REASONING_PREFIXES, and assign self._raw_client to an
OpenAI()-compatible client in __init__) or implement the native handler path
(add a handler in llm_native_stream.py, register it, and set
self._native_stream_provider = '<key>' in the driver's __init__) so
reasoning_content is preserved.

In `@packages/shared-ui/src/modules/chat/components/MessageBubble.tsx`:
- Around line 124-134: The disclosure button in MessageBubble.tsx should expose
its expanded state to assistive tech: add aria-expanded={open} to the button
(the one using S.reasoningToggle and toggling setOpen) so it reflects the
current open boolean; also ensure the collapsible region that this button
controls has a stable id and add aria-controls referencing that id from the
button (create an id on the collapsible container if missing) so screen readers
can associate the button with the region.

In `@packages/shared-ui/src/modules/chat/hooks/useChatMessages.ts`:
- Around line 166-179: The chunk_end branch currently appends a truncation/end
note but doesn't clear the reasoningStreaming flag, which can leave a
"Thinking…" ReasoningPanel stuck open; in the chunk_end handler (inside the same
conditional that checks type === 'chunk_end') after computing id and note call
updateMessages to also set reasoningStreaming: false for the matching message
(the same map that updates text), ensuring the message object for streamingId
has reasoningStreaming set to false; reference the existing symbols streamingId,
updateMessages, reasoningStreaming, and the chunk_end branch to locate where to
add this change.
- Around line 116-143: The ensureBubble helper currently only returns an id
which causes the chunk-handler to suppress appends when a later delta equals the
current m.text (the m.text === delta && m.reasoning === undefined check); modify
ensureBubble (used in useChatMessages) to return an object like { id, created }
(created=true when it seeded a new message), update callers in the onSSE "chunk"
and "reasoning_chunk" branches to pass initial values (for reasoning_chunk seed
with reasoning: delta), and in the map/update logic short‑circuit (skip
appending) only when created is true so later identical deltas are not silently
dropped.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5a47c2ed-f5d2-40b1-8c4a-e4db9ccf9f96

📥 Commits

Reviewing files that changed from the base of the PR and between efcc344 and d225473.

📒 Files selected for processing (25)
  • apps/chat-ui/src/components/Message.tsx
  • apps/chat-ui/src/hooks/useChatMessages.ts
  • apps/chat-ui/src/types/chat.types.ts
  • nodes/src/nodes/llm_anthropic/anthropic.py
  • nodes/src/nodes/llm_deepseek/deepseek.py
  • nodes/src/nodes/llm_gmi_cloud/gmi_cloud.py
  • nodes/src/nodes/llm_mistral/mistral.py
  • nodes/src/nodes/llm_ollama/ollama.py
  • nodes/src/nodes/llm_openai/IGlobal.py
  • nodes/src/nodes/llm_openai/openai_client.py
  • nodes/src/nodes/llm_perplexity/perplexity.py
  • nodes/src/nodes/llm_qwen/qwen_client.py
  • nodes/src/nodes/llm_vision_mistral/mistral_vision.py
  • nodes/src/nodes/llm_vision_ollama/ollama_vision.py
  • nodes/src/nodes/llm_xai/IGlobal.py
  • nodes/src/nodes/llm_xai/requirements.txt
  • nodes/src/nodes/llm_xai/xai.py
  • packages/ai/src/ai/common/STREAMING.md
  • packages/ai/src/ai/common/chat.py
  • packages/ai/src/ai/common/llm_base.py
  • packages/ai/src/ai/common/llm_native_stream.py
  • packages/shared-ui/rsbuild.config.ts
  • packages/shared-ui/src/modules/chat/components/MessageBubble.tsx
  • packages/shared-ui/src/modules/chat/hooks/useChatMessages.ts
  • packages/shared-ui/src/modules/chat/types.ts

Comment thread apps/chat-ui/src/components/Message.tsx Resolved
Comment thread nodes/src/nodes/llm_mistral/mistral.py Resolved
Comment thread packages/ai/src/ai/common/STREAMING.md Resolved
@Rod-Christensen
Rod-Christensen marked this pull request as draft May 15, 2026 18:00
@dsapandora
dsapandora marked this pull request as ready for review May 15, 2026 18:45

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
nodes/src/nodes/llm_perplexity/perplexity.py (1)

206-235: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

chat() override still bypasses token streaming and never calls on_chunk.

Even with callback parameters, this path is non-streaming (invoke()), so answer tokens are not emitted incrementally. Route callback-enabled calls through ChatBase.chat(...) to preserve the unified SSE streaming contract.

Suggested fix
     def chat(self, question: Question, on_chunk=None, on_finish=None, on_reasoning_chunk=None) -> Answer:
         """Process a question and return an answer with retry logic.
@@
         ``on_reasoning_chunk`` so the UI panel renders it.
         """
+        if on_chunk is not None or on_finish is not None or on_reasoning_chunk is not None:
+            return super().chat(
+                question,
+                on_chunk=on_chunk,
+                on_finish=on_finish,
+                on_reasoning_chunk=on_reasoning_chunk,
+            )
+
         prompt = validate_prompt(question.getPrompt(), self._modelTotalTokens, self.getTokens)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@nodes/src/nodes/llm_perplexity/perplexity.py` around lines 206 - 235, The
current chat method in perplexity.py bypasses streaming by calling
self._llm.invoke(...) and never emitting on_chunk; change chat to detect when
any callback (on_chunk, on_finish, on_reasoning_chunk) is provided and route
those calls to the superclass streaming implementation (ChatBase.chat(...)) so
the SSE/token-streaming contract is preserved; keep the non-streaming invoke()
path only when no callbacks are passed. Update the chat function to call
super().chat(question, on_chunk, on_finish, on_reasoning_chunk) (or
ChatBase.chat(self, ...)) when callbacks are present and return its Answer,
otherwise retain the existing _llm.invoke(...) logic and post-process <think>
tags as before.
♻️ Duplicate comments (4)
apps/chat-ui/src/hooks/useChatMessages.ts (2)

105-117: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

streamingId = Date.now() collision risk is still unresolved.

This was raised on a prior commit and the diff did not adopt the sibling hook's monotonic nextId() approach. Two messages minted within the same millisecond (the streaming bubble at line 107, the user/bot/system/clear messages at lines 171/212/227/238/259/273) will share an id, and the streaming setMessages(prev => prev.map(m => m.id === id ? ... : m)) updaters will then mutate every collision target. Hoist a module-level nextId() counter (mirroring packages/shared-ui/.../useChatMessages.ts) and use it everywhere an id is generated in this file.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/chat-ui/src/hooks/useChatMessages.ts` around lines 105 - 117, The
current ensureBubble implementation assigns ids with streamingId = Date.now(),
which risks collisions; instead hoist a module-level monotonic counter function
nextId() (like the sibling hook in packages/shared-ui) and use it to generate
ids everywhere this file creates messages (replace streamingId = Date.now() in
ensureBubble and all other id assignments used for user/bot/system/clear
messages and any setMessages callers that rely on ids); update ensureBubble to
call nextId() to set streamingId and return that id, and ensure all setMessages
updaters that compare m.id use these monotonic ids so stream updates only target
the intended message.

155-167: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

chunk_end still doesn't clear reasoningStreaming — "Thinking…" gets stuck.

This was raised on a prior commit and the diff did not change behavior here. Any path where reasoning_end does not fire before chunk_end (abrupt stream end, providers that don't emit a dedicated reasoning terminator, finishReason === 'length'/'error', or even a normal stop without a reasoning terminator) leaves the bubble with reasoningStreaming: true. Since ReasoningPanel only auto-collapses on !streaming && hasContent, the user sees a permanent "Thinking…" label after the final answer is rendered.

Clear reasoningStreaming on every chunk_end regardless of finishReason, and only conditionally append the truncation/end note.

🛠️ Proposed fix
 				if (type === 'chunk_end') {
 					const reason = data.finishReason as string | null | undefined;
-					if (reason && reason !== 'stop' && streamingId !== null) {
-						const id = streamingId;
-						const note = reason === 'length'
-							? '\n\n_[response truncated by max_tokens]_'
-							: `\n\n_[stream ended: ${reason}]_`;
-						setMessages(prev => prev.map(m =>
-							m.id === id ? { ...m, text: m.text + note } : m
-						));
-					}
+					if (streamingId !== null) {
+						const id = streamingId;
+						const note =
+							reason && reason !== 'stop'
+								? reason === 'length'
+									? '\n\n_[response truncated by max_tokens]_'
+									: `\n\n_[stream ended: ${reason}]_`
+								: '';
+						setMessages(prev => prev.map(m =>
+							m.id === id
+								? { ...m, text: note ? m.text + note : m.text, reasoningStreaming: false }
+								: m
+						));
+					}
 					return;
 				}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/chat-ui/src/hooks/useChatMessages.ts` around lines 155 - 167, The
chunk_end branch leaves reasoningStreaming set to true causing "Thinking…" to
stick; update the chunk_end handling in useChatMessages so it always clears the
reasoningStreaming flag for the related message (use streamingId/id and
setMessages to update the message's reasoningStreaming to false), and then
separately handle finishReason to optionally append the truncation/end note
(only append the note when reason exists and is not 'stop'); ensure you use the
same id selection logic (streamingId -> id) and update the message object to
both set reasoningStreaming: false and conditionally concatenate the note to
text.
packages/shared-ui/src/modules/chat/hooks/useChatMessages.ts (1)

164-177: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

chunk_end still doesn't clear reasoningStreaming — same stuck "Thinking…" issue as chat-ui.

This was raised on a prior commit and the diff is unchanged. When chunk_end arrives without a preceding reasoning_end (abrupt termination, providers that don't emit a reasoning terminator, or any finishReason including the stop path that skips the if-branch entirely), the bubble retains reasoningStreaming: true. ReasoningPanel's auto-collapse condition (!streaming && hasContent) never trips, so the user sees an expanded "Thinking…" panel after the answer is fully rendered.

Clear reasoningStreaming unconditionally on chunk_end and only conditionally append the finish-reason note.

🛠️ Proposed fix
 					if (type === 'chunk_end') {
 						const reason = data.finishReason as string | null | undefined;
-						if (reason && reason !== 'stop' && streamingId !== null) {
-							const id = streamingId;
-							const note =
-								reason === 'length'
-									? '\n\n_[response truncated by max_tokens]_'
-									: `\n\n_[stream ended: ${reason}]_`;
-							updateMessages((prev) =>
-								prev.map((m) => (m.id === id ? { ...m, text: m.text + note } : m))
-							);
-						}
+						if (streamingId !== null) {
+							const id = streamingId;
+							const note =
+								reason && reason !== 'stop'
+									? reason === 'length'
+										? '\n\n_[response truncated by max_tokens]_'
+										: `\n\n_[stream ended: ${reason}]_`
+									: '';
+							updateMessages((prev) =>
+								prev.map((m) =>
+									m.id === id
+										? { ...m, text: note ? m.text + note : m.text, reasoningStreaming: false }
+										: m
+								)
+							);
+						}
 						return;
 					}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/shared-ui/src/modules/chat/hooks/useChatMessages.ts` around lines
164 - 177, On 'chunk_end' in useChatMessages, you need to clear the
reasoningStreaming flag unconditionally and only append the finishReason note
conditionally; modify the handler for the 'chunk_end' branch so it calls
updateMessages to set reasoningStreaming: false for the affected message(s) (use
streamingId when available, or clear any message with reasoningStreaming ===
true), and only append the note to text when reason is present and not 'stop'
(keep the existing note logic using data.finishReason). Ensure you still return
after handling 'chunk_end'.
nodes/src/nodes/llm_mistral/mistral.py (1)

233-233: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Guard choices[0] before dereference.

Line 233 can throw IndexError/AttributeError when the provider returns empty or malformed choices, masking the real provider failure path.

Suggested fix
-        content = chat_response.choices[0].message.content
+        choices = getattr(chat_response, 'choices', None) or []
+        if not choices or getattr(choices[0], 'message', None) is None:
+            raise ValueError('Mistral AI returned no usable choices in the response.')
+        content = choices[0].message.content
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@nodes/src/nodes/llm_mistral/mistral.py` at line 233, The assignment "content
= chat_response.choices[0].message.content" can raise IndexError/AttributeError
when the provider returns no or malformed choices; update the code that
processes the Mistral response (the chat_response handling block in mistral.py)
to first verify chat_response is not None, that getattr(chat_response,
'choices', None) is a non-empty list, and that choices[0] has a message with a
content attribute before dereferencing; if any check fails, raise or return a
clear, descriptive error (or log and handle gracefully) so downstream code
doesn't crash silently.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@nodes/src/nodes/llm_vision_mistral/mistral_vision.py`:
- Around line 250-252: Validate that chat_response.choices is present and
non-empty before accessing choices[0]; in the block around answer.setAnswer and
on_finish, check that chat_response.choices and chat_response.choices[0] exist,
and if not, set a safe fallback (e.g., answer.setAnswer("") or a descriptive
error message) and invoke on_finish with a fallback reason (e.g., 'no_choices'
or None) instead of indexing into an empty list; update references to
chat_response.choices[0].message.content and getattr(..., 'finish_reason', None)
accordingly so you never dereference into an absent choice.

In `@packages/ai/src/ai/common/chat.py`:
- Around line 429-431: The Responses-path exception fallback builds fallback
visible content in text_parts but only invokes on_reasoning_chunk and never
calls on_chunk, so the UI doesn't receive the visible fallback; update the
exception handling in the Responses stream fallback to call on_chunk(text)
(using the same assembled text_parts content) in addition to
on_reasoning_chunk('_Thinking…_\n\n') when on_chunk is not None, ensuring you
reference the existing callbacks on_chunk and on_reasoning_chunk and the
text_parts assembly used in that fallback branch.

In `@packages/ai/src/ai/common/STREAMING.md`:
- Line 38: The diagram arrow "DS --> LC" is inconsistent with the documented
implementation that registers DeepSeek/xAI/Qwen under the reasoning_content
native-stream path as openai_compat_reasoning; update the diagram so DS points
to openai_compat_reasoning (or otherwise change the diagram labels) to match the
text, and verify the same alignment where
reasoning_content/openai_compat_reasoning is mentioned (references: DS, LC,
openai_compat_reasoning, and reasoning_content) so the onboarding path in the
diagram and the textual lines are identical.

---

Outside diff comments:
In `@nodes/src/nodes/llm_perplexity/perplexity.py`:
- Around line 206-235: The current chat method in perplexity.py bypasses
streaming by calling self._llm.invoke(...) and never emitting on_chunk; change
chat to detect when any callback (on_chunk, on_finish, on_reasoning_chunk) is
provided and route those calls to the superclass streaming implementation
(ChatBase.chat(...)) so the SSE/token-streaming contract is preserved; keep the
non-streaming invoke() path only when no callbacks are passed. Update the chat
function to call super().chat(question, on_chunk, on_finish, on_reasoning_chunk)
(or ChatBase.chat(self, ...)) when callbacks are present and return its Answer,
otherwise retain the existing _llm.invoke(...) logic and post-process <think>
tags as before.

---

Duplicate comments:
In `@apps/chat-ui/src/hooks/useChatMessages.ts`:
- Around line 105-117: The current ensureBubble implementation assigns ids with
streamingId = Date.now(), which risks collisions; instead hoist a module-level
monotonic counter function nextId() (like the sibling hook in
packages/shared-ui) and use it to generate ids everywhere this file creates
messages (replace streamingId = Date.now() in ensureBubble and all other id
assignments used for user/bot/system/clear messages and any setMessages callers
that rely on ids); update ensureBubble to call nextId() to set streamingId and
return that id, and ensure all setMessages updaters that compare m.id use these
monotonic ids so stream updates only target the intended message.
- Around line 155-167: The chunk_end branch leaves reasoningStreaming set to
true causing "Thinking…" to stick; update the chunk_end handling in
useChatMessages so it always clears the reasoningStreaming flag for the related
message (use streamingId/id and setMessages to update the message's
reasoningStreaming to false), and then separately handle finishReason to
optionally append the truncation/end note (only append the note when reason
exists and is not 'stop'); ensure you use the same id selection logic
(streamingId -> id) and update the message object to both set
reasoningStreaming: false and conditionally concatenate the note to text.

In `@nodes/src/nodes/llm_mistral/mistral.py`:
- Line 233: The assignment "content = chat_response.choices[0].message.content"
can raise IndexError/AttributeError when the provider returns no or malformed
choices; update the code that processes the Mistral response (the chat_response
handling block in mistral.py) to first verify chat_response is not None, that
getattr(chat_response, 'choices', None) is a non-empty list, and that choices[0]
has a message with a content attribute before dereferencing; if any check fails,
raise or return a clear, descriptive error (or log and handle gracefully) so
downstream code doesn't crash silently.

In `@packages/shared-ui/src/modules/chat/hooks/useChatMessages.ts`:
- Around line 164-177: On 'chunk_end' in useChatMessages, you need to clear the
reasoningStreaming flag unconditionally and only append the finishReason note
conditionally; modify the handler for the 'chunk_end' branch so it calls
updateMessages to set reasoningStreaming: false for the affected message(s) (use
streamingId when available, or clear any message with reasoningStreaming ===
true), and only append the note to text when reason is present and not 'stop'
(keep the existing note logic using data.finishReason). Ensure you still return
after handling 'chunk_end'.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 0413c32c-a2b3-4812-853d-0c764e275655

📥 Commits

Reviewing files that changed from the base of the PR and between d225473 and 1cb15f0.

📒 Files selected for processing (8)
  • apps/chat-ui/src/hooks/useChatMessages.ts
  • nodes/src/nodes/llm_mistral/mistral.py
  • nodes/src/nodes/llm_perplexity/perplexity.py
  • nodes/src/nodes/llm_vision_mistral/mistral_vision.py
  • nodes/src/nodes/llm_vision_ollama/ollama_vision.py
  • packages/ai/src/ai/common/STREAMING.md
  • packages/ai/src/ai/common/chat.py
  • packages/shared-ui/src/modules/chat/hooks/useChatMessages.ts

Comment thread packages/ai/src/ai/common/chat.py Resolved
Comment thread packages/ai/src/ai/common/STREAMING.md Resolved
@github-actions github-actions Bot added builder Node builder tooling and ./builder workflows and removed module:nodes Python pipeline nodes module:ai AI/ML modules module:ui Chat UI and Dropper UI labels May 16, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
nodes/src/nodes/llm_anthropic/anthropic.py (1)

38-83: 🧹 Nitpick | 🔵 Trivial | 💤 Low value

Consider documenting the anti-SIGBUS patch rationale.

The monkey-patching of transformers.AutoTokenizer.from_pretrained and langchain_core.utils.tokenization.get_token_ids is a significant runtime modification. While the inline comments explain what it does, a brief note about why SIGBUS crashes occur (likely from tokenizer Rust binaries on certain platforms) would help future maintainers understand when this patch might become unnecessary.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@nodes/src/nodes/llm_anthropic/anthropic.py` around lines 38 - 83, Add a brief
rationale comment above the anti-SIGBUS patch explaining why the monkey-patch is
necessary (tokenizer Rust binaries can cause SIGBUS on some
platforms/containers, so we force slow Python tokenizers and stub Claude token
counts), when it can be removed (once tokenizer Rust issues are fixed or
transformers/langchain versions no longer spawn Rust binaries), and note the
specific symbols affected: transformers.AutoTokenizer.from_pretrained (patched
by _patched_from_pretrained) and langchain_core.utils.tokenization.get_token_ids
(patched by _patched_get_token_ids) so future maintainers know what to revert.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@nodes/src/nodes/llm_deepseek/deepseek.py`:
- Around line 65-66: The ValueError message in deepseek.py uses double quotes;
update it to use single-quoted string literals per nodes/**/*.py style guide.
Locate the check that references serverbase and apikey (the if 'api.deepseek' in
serverbase and not apikey.startswith('sk-') block) and replace the double-quoted
error message with a single-quoted one (raising ValueError with a single-quoted
string).

In `@packages/ai/src/ai/common/llm_native_stream.py`:
- Around line 58-61: The budget calculation in llm_native_stream.py can produce
negative values for small model_output_tokens (the variables involved are
model_output_tokens, budget, and out['thinking']); update the logic that
computes budget so it is clamped to a non-negative value and never greater than
model_output_tokens — for example compute a candidate like candidate =
min(max(2048, model_output_tokens // 2), model_output_tokens - 1024), then set
budget = max(0, candidate) (or otherwise ensure budget is between 0 and
model_output_tokens) before assigning out['thinking'] = {'type': 'enabled',
'budget_tokens': budget}.

In `@tools/src/providers/base.py`:
- Around line 358-359: The warning message always claims "falling back to
OpenRouter" even though the chosen fallback is determined by primary_source;
update the logic around report.warning where primary_source is computed
(referencing report.warning, primary_source, model_sources, and
_enrichment_available) to interpolate or construct the message using the actual
selected primary_source (or a sensible default label if primary_source is None)
so the warning accurately reflects which provider (e.g., 'litellm' or
'OpenRouter') will be used.

---

Outside diff comments:
In `@nodes/src/nodes/llm_anthropic/anthropic.py`:
- Around line 38-83: Add a brief rationale comment above the anti-SIGBUS patch
explaining why the monkey-patch is necessary (tokenizer Rust binaries can cause
SIGBUS on some platforms/containers, so we force slow Python tokenizers and stub
Claude token counts), when it can be removed (once tokenizer Rust issues are
fixed or transformers/langchain versions no longer spawn Rust binaries), and
note the specific symbols affected: transformers.AutoTokenizer.from_pretrained
(patched by _patched_from_pretrained) and
langchain_core.utils.tokenization.get_token_ids (patched by
_patched_get_token_ids) so future maintainers know what to revert.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: e28752af-e3be-4c82-ac0f-81cb1f79b55d

📥 Commits

Reviewing files that changed from the base of the PR and between 1cb15f0 and e00692d.

📒 Files selected for processing (25)
  • nodes/src/nodes/llm_anthropic/anthropic.py
  • nodes/src/nodes/llm_anthropic/services.json
  • nodes/src/nodes/llm_deepseek/deepseek.py
  • nodes/src/nodes/llm_deepseek/services.json
  • nodes/src/nodes/llm_gmi_cloud/gmi_cloud.py
  • nodes/src/nodes/llm_gmi_cloud/services.json
  • nodes/src/nodes/llm_mistral/services.json
  • nodes/src/nodes/llm_ollama/ollama.py
  • nodes/src/nodes/llm_ollama/services.json
  • nodes/src/nodes/llm_openai/openai_client.py
  • nodes/src/nodes/llm_openai/services.json
  • nodes/src/nodes/llm_perplexity/services.json
  • nodes/src/nodes/llm_qwen/qwen_client.py
  • nodes/src/nodes/llm_qwen/services.json
  • nodes/src/nodes/llm_xai/services.json
  • nodes/src/nodes/llm_xai/xai.py
  • packages/ai/src/ai/common/STREAMING.md
  • packages/ai/src/ai/common/chat.py
  • packages/ai/src/ai/common/llm_native_stream.py
  • tools/scripts/tasks.js
  • tools/src/core/merger.py
  • tools/src/core/patcher.py
  • tools/src/providers/base.py
  • tools/src/stamp_reasoning.py
  • tools/src/sync_models.py

Comment thread tools/sync_models/src/providers/base.py Resolved
Comment thread tools/sync_models/src/sync_models.py Resolved

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
packages/ai/src/ai/common/chat.py (1)

462-466: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Fallback content not emitted through on_chunk callback.

When the Responses stream fails before any visible delta and the fallback path executes (lines 463-466), the fallback content is collected in text_parts but never emitted via on_chunk. This means the UI won't receive the fallback content through the streaming lane, even though emitted['any'] is still False at this point.

Suggested fix
             if emitted is None or not emitted['any']:
                 results = self._llm.invoke(prompt)
                 content = getattr(results, 'content', '') or ''
-                text_parts = [content if isinstance(content, str) else str(content)]
+                content_text = content if isinstance(content, str) else str(content)
+                text_parts = [content_text]
+                if content_text and on_chunk is not None:
+                    on_chunk(content_text)
                 finish_reason = 'stop'
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/ai/src/ai/common/chat.py` around lines 462 - 466, The fallback
branch after responses stream failure collects fallback content into text_parts
but never forwards it via the on_chunk callback, so the UI won't receive it;
update the fallback branch in the method containing emitted, results =
self._llm.invoke(prompt) to call self.on_chunk (or the same callback used by the
streaming path) with the assembled text_parts, the determined finish_reason, and
any metadata so it mirrors the streaming emission behavior, and mark
emitted['any'] = True after calling the callback; reference the
variables/results (results, content, text_parts, finish_reason, emitted) to
locate and mirror the streaming-path emission logic.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@nodes/src/nodes/llm_perplexity/perplexity.py`:
- Around line 44-47: Move the import for validate_prompt so all imports are
grouped at the top of the module: relocate "from ai.common.validation import
validate_prompt" above the module-level constant _THINK_RE (i.e., with the other
imports) to preserve standard import ordering and keep _THINK_RE defined after
imports.

---

Duplicate comments:
In `@packages/ai/src/ai/common/chat.py`:
- Around line 462-466: The fallback branch after responses stream failure
collects fallback content into text_parts but never forwards it via the on_chunk
callback, so the UI won't receive it; update the fallback branch in the method
containing emitted, results = self._llm.invoke(prompt) to call self.on_chunk (or
the same callback used by the streaming path) with the assembled text_parts, the
determined finish_reason, and any metadata so it mirrors the streaming emission
behavior, and mark emitted['any'] = True after calling the callback; reference
the variables/results (results, content, text_parts, finish_reason, emitted) to
locate and mirror the streaming-path emission logic.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6c10175b-454a-45df-aa8e-730bcb30f58b

📥 Commits

Reviewing files that changed from the base of the PR and between e00692d and 934b3f7.

📒 Files selected for processing (2)
  • nodes/src/nodes/llm_perplexity/perplexity.py
  • packages/ai/src/ai/common/chat.py

@dsapandora
dsapandora force-pushed the feat/730-llm-streaming-chunk branch from 3b847d8 to 6a264aa Compare May 20, 2026 16:23
@dsapandora
dsapandora force-pushed the feat/730-llm-streaming-chunk branch from 6a264aa to 73f260f Compare May 30, 2026 10:13
@dsapandora
dsapandora force-pushed the feat/730-llm-streaming-chunk branch 6 times, most recently from 5101ae6 to 13f2f34 Compare June 9, 2026 13:40
Add native streaming across LLM providers, splitting inline <think> tags
into a separate reasoning stream and pushing fallback answers through
on_chunk so the UI bubble updates live. Includes deepseek regression fixes.
The chat-ui already renders a collapsible Thinking group from
sseType=='thinking' — the same channel agents use. Revert the parallel UI
work (ReasoningPanel, reasoning_chunk/reasoning_end events, reasoning fields)
and have the LLM node emit its captured chain-of-thought via
sendSSE('thinking', message=...) instead. The answer is written normally (no
token streaming), so the chat-ui matches develop. Native reasoning capture
(Anthropic thinking_delta, OpenAI reasoning_content) and the model-stamping
tooling are kept.
@dsapandora
dsapandora force-pushed the feat/730-llm-streaming-chunk branch from 13f2f34 to be9e177 Compare June 10, 2026 06:25
@github-actions github-actions Bot removed the docs Documentation label Jun 10, 2026
Stop reading capabilities.reasoning from services.json (which required running
the model updater to turn reasoning on). Each provider now decides by model name:
Anthropic and OpenAI o-series/gpt-5 gate by name (they send thinking/reasoning
params that 400 on unsupported models; Haiku is guarded out), while OpenAI-compatible
drivers always attempt the reasoning stream — harmless when the model emits no
reasoning_content. No config or model sync needed to enable reasoning.
Drop the unconditional "_Thinking…_" prime. With adaptive thinking the model
may choose not to reason on simpler turns; the prime left an empty Thinking
box. Reasoning is now surfaced only when the model emits real thinking deltas.
@dsapandora
dsapandora force-pushed the feat/730-llm-streaming-chunk branch from 3640dd5 to 6783588 Compare June 10, 2026 09:55
…and stamp tooling

Reasoning is decided per provider by model name — Anthropic extended thinking,
OpenAI o-series/gpt-5 via the Responses API (discriminated by _raw_client) — and
OpenAI-compatible drivers always attempt the generic reasoning_content stream.
Removes the _is_reasoning flag entirely and reverts the capabilities.reasoning
stamping tooling (merger/patcher/sync_models/tasks.js + stamp_reasoning.py),
which is no longer needed.
@github-actions github-actions Bot added module:nodes Python pipeline nodes module:ai AI/ML modules and removed builder Node builder tooling and ./builder workflows labels Jun 10, 2026
@dsapandora

Copy link
Copy Markdown
Collaborator Author

Close in favor of #1217

@dsapandora dsapandora closed this Jun 10, 2026
@stepmikhaylov
stepmikhaylov deleted the feat/730-llm-streaming-chunk branch September 22, 2026 20:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:ai AI/ML modules module:nodes Python pipeline nodes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant