Repository navigation
feat(node): Add documents lane to frame grabber and vision nodes - #459
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a new "documents" lane across frame_grabber, llm_vision_mistral, llm_vision_ollama, and thumbnail nodes; implements Changes
Sequence Diagram(s)sequenceDiagram
participant Producer as Producer (frame_grabber / upstream)
participant Node as Node.IInstance
participant Chat as ChatService
participant Sink as Downstream Instance
Producer->>Node: send documents (List<Doc> with base64 image + metadata)
Node->>Node: filter Image Docs, decode base64 → data URL, build Question w/ prompt
Node->>Chat: chat(question)
Chat-->>Node: answer (text)
Node->>Sink: writeDocuments([Doc(type='Text', page_content=answer, metadata=orig.meta)])
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@nodes/src/nodes/llm_vision_mistral/IInstance.py`:
- Line 24: Remove the deprecated typing.List import and use the built-in list
for type hints: delete the line "from typing import List" and replace any
occurrences of "List[...]" in this module (e.g., in the IInstance class method
signatures) with "list[...]" (or plain "list" where appropriate), ensuring all
function/method annotations such as those on methods in IInstance are updated to
use the built-in list type.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 57c7a5b0-197d-4226-afc6-7feca25223ea
📒 Files selected for processing (5)
nodes/src/nodes/frame_grabber/services.jsonnodes/src/nodes/llm_vision_mistral/IInstance.pynodes/src/nodes/llm_vision_mistral/services.jsonnodes/src/nodes/llm_vision_ollama/IInstance.pynodes/src/nodes/llm_vision_ollama/services.json
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@nodes/src/nodes/thumbnail/IInstance.py`:
- Around line 132-142: Add a PEP 257 docstring to the writeDocuments method
describing its purpose, parameters, behavior, and return (if any); specifically
document that writeDocuments(self, documents: List[Doc]) filters non-Image or
empty docs (uses doc.type and doc.page_content), converts image content via
ImageProcessor.load_image_from_base64, generates a thumbnail with
ImageProcessor.get_thumbnail, encodes it with ImageProcessor.get_base64, and
forwards a new Doc to self.instance.writeDocuments; keep the docstring concise
and follow other class methods' style (one-line summary plus short description
and parameter description).
- Around line 134-136: The warning is misleading because the branch handles two
cases (non-image type vs empty content) but always logs "unexpected type";
update the check in IInstance (the block using doc.type and doc.page_content) to
distinguish the cases: if doc.type != 'Image' log that the document type is
unexpected, else if not doc.page_content log that the Image has empty
page_content, then continue; ensure you still call warning(...) and continue in
both branches so behavior is unchanged.
In `@nodes/src/nodes/thumbnail/services.json`:
- Around line 71-81: Add missing "description" fields for the "image" input lane
and its output entries to match the "documents" lane style: update the "image"
lane object (the one with "lane": "image") to include a descriptive
"description" explaining the expected image input, and add "description" strings
to each output entry in its "output" array (the outputs with "lane": "documents"
and "lane": "image") describing what each output contains; keep wording
consistent with the existing "documents" lane descriptions so auto-generated
docs/UI tooltips display properly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 19b18fbf-56b8-4394-87b8-e873f861039e
📒 Files selected for processing (2)
nodes/src/nodes/thumbnail/IInstance.pynodes/src/nodes/thumbnail/services.json
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@nodes/src/nodes/llm_vision_mistral/IInstance.py`:
- Around line 76-77: The public method writeDocuments is missing a PEP 257
docstring; add a concise triple-quoted docstring directly above the def
writeDocuments(self, documents: list[Doc]) declaration that describes the method
purpose, parameters (documents: list[Doc]) and any return value or side effects
(e.g., raises, mutations, I/O), following PEP 257 style and the project's
existing docstring conventions so the node module remains consistently
documented.
- Around line 94-95: Wrap the call to self.IGlobal._chat.chat(question) inside a
try/except inside the loop in IInstance so a thrown exception for one document
is caught, logged (or appended to results as an error for that item), and the
loop continues to the next document instead of aborting the batch; specifically
locate the line with answer = self.IGlobal._chat.chat(question), add exception
handling around that call, ensure any needed cleanup or partial result insertion
happens for that document, and do not re-raise the exception so remaining
documents are processed.
In `@nodes/src/nodes/llm_vision_ollama/IInstance.py`:
- Around line 84-85: Add a PEP 257 docstring to the public method writeDocuments
(in IInstance) that briefly describes its purpose, documents the parameters
(documents: list[Doc], referencing the Doc type and that it imports Question),
any return value (or None), and any exceptions it may raise; place the
triple-quoted docstring immediately below the def writeDocuments(...) signature
and follow node coding standards for style and tone.
- Around line 102-105: Wrap each per-document call to self.IGlobal._chat.chat in
a try/except inside the loop that produces answer and calls
self.instance.writeDocuments so a failure for one document doesn't stop
processing of remaining documents; on exception catch the error, log it (e.g.,
via process logger or self.logger) including document.metadata and the
exception, optionally emit a fallback/error Doc with metadata noting the
failure, and then continue to the next document. Ensure you reference the same
Doc construction (Doc(type='Text', page_content=..., metadata=doc.metadata)) and
that exceptions from self.IGlobal._chat.chat or answer.getText() are handled.
In `@nodes/src/nodes/thumbnail/IInstance.py`:
- Around line 150-154: The thumbnail pipeline currently lets failures in
ImageProcessor.load_image_from_base64, ImageProcessor.get_thumbnail, or
ImageProcessor.get_base64 abort the entire batch; change the loop that processes
each doc so each document is handled inside a try/except block that catches
conversion/errors, logs or records the failure, and continues to the next doc
instead of throwing—ensure you still call self.instance.writeDocuments only for
successful conversions (use the same Doc construction: Doc(type='Image',
page_content=thumbnail_base64, metadata=doc.metadata)) and reference
ImageProcessor.load_image_from_base64, ImageProcessor.get_thumbnail,
ImageProcessor.get_base64, and self.instance.writeDocuments when making this
change.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 012787f8-c4db-414b-8ac8-dbc745a384ef
📒 Files selected for processing (4)
nodes/src/nodes/llm_vision_mistral/IInstance.pynodes/src/nodes/llm_vision_ollama/IInstance.pynodes/src/nodes/thumbnail/IInstance.pynodes/src/nodes/thumbnail/services.json
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@nodes/src/nodes/llm_vision_mistral/IInstance.py`:
- Around line 77-85: The docstring for the image-processing method incorrectly
says it "emits vision model responses as questions" but the implementation emits
Doc(type='Text') (see the emission at the Doc(...) call around line where
Doc(type='Text') is created); update the method's docstring in IInstance (the
image processing method) to state that valid image documents produce vision
model responses emitted as text documents (matching the Ollama Vision node
wording), and ensure the Args/behavior description reflects that non-image or
empty documents are skipped with warnings while valid images produce Text Docs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 532eda64-ef66-44f9-96cd-374ad58d35f4
📒 Files selected for processing (3)
nodes/src/nodes/llm_vision_mistral/IInstance.pynodes/src/nodes/llm_vision_ollama/IInstance.pynodes/src/nodes/thumbnail/IInstance.py
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@nodes/src/nodes/frame_grabber/IGlobal.py`:
- Around line 40-44: The current logic in IGlobal (where self.config['interval']
is translated to self.config['fps'] for VideoFrameExtractor) only checks
truthiness and allows negative intervals; update the code that reads
self.config.get('interval') to validate that interval is not None and strictly >
0, and if interval <= 0 raise a ValueError with the invalid value, otherwise set
self.config['fps'] = 1.0 / interval so VideoFrameExtractor never receives a
non-positive fps.
In `@nodes/src/nodes/llm_vision_ollama/ollama_vision.py`:
- Around line 132-142: The daemon thread started with
threading.Thread(target=_invoke, daemon=True) can remain running after
t.join(timeout=hard_timeout) if t.is_alive(), so add a log entry when a
TimeoutError is raised to surface recurring timeouts and orphaned threads;
update the block around t.join / t.is_alive to call the module or instance
logger (e.g., logger or self.logger) and log a warning including hard_timeout,
attempt, and any frame/request identifiers before raising TimeoutError so
operators can track patterns from _invoke timeouts.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: a975b13f-2d4e-48bd-b032-c9b5840ff564
📒 Files selected for processing (4)
nodes/src/nodes/frame_grabber/IGlobal.pynodes/src/nodes/llm_vision_ollama/IInstance.pynodes/src/nodes/llm_vision_ollama/ollama_vision.pynodes/src/nodes/thumbnail/IInstance.py
* add documents lane to frame grabber and vision nodes * add documents lane to thumbnail node * address CR comments * Address CR comments * fix frame interval, prevent double-emit, add vision timeout * fix connection pool exhaustion and server TTL timeout * address CR comments
…ketride-org#459) * add documents lane to frame grabber and vision nodes * add documents lane to thumbnail node * address CR comments * Address CR comments * fix frame interval, prevent double-emit, add vision timeout * fix connection pool exhaustion and server TTL timeout * address CR comments
Summary
Type
feature
Testing
./builder testpassesChecklist
Linked Issue
Fixes #
Summary by CodeRabbit
New Features
Bug Fixes / Reliability