Skip to content

feat(nodes): add accessibility_describe node for visually impaired users - #317

Merged
jmaionchi merged 6 commits into
rocketride-org:developfrom
chinesepowered:feat/accessibility-describe-node
Mar 23, 2026
Merged

jmaionchi merged 6 commits into
rocketride-org:developfrom
chinesepowered:feat/accessibility-describe-node

Conversation

@chinesepowered

@chinesepowered chinesepowered commented Mar 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds a new Accessibility Describe node powered by Google Gemini Vision that generates scene descriptions optimized for blind and visually impaired users.

Features:

  • Structured output: hazards, spatial layout (clock positions), text/OCR, people detection, and navigation guidance
  • Safety-first prompting — always leads with potential hazards
  • Configurable spatial format (clock positions vs relative directions)
  • Configurable hazard priority level
  • Supports Gemini 2.5 Flash, 2.5 Pro, and 2.0 Flash models
  • Retry logic with exponential backoff
  • Follows existing vision node patterns (image lane in, text lane out)

Built as part of the AltVision project — AI glasses for the visually impaired using real-time Gemini Live streaming + RocketRide pipelines.

Type

feature

Testing

  • Tests added or updated
  • Tested locally
  • ./builder test passes

Checklist

  • Commit messages follow conventional commits
  • No secrets or credentials included
  • Wiki updated (if applicable)
  • Breaking changes documented (if applicable)

Related Issues

Summary by CodeRabbit

  • New Features

    • Added an Accessibility Describe node that accepts images (including streamed uploads) and returns accessibility-focused scene descriptions with configurable prompts, hazard-prioritization, and spatial formatting; selectable Gemini Vision profiles and model/token settings.
  • Tests

    • Includes image-based test cases validating non-empty text outputs.
  • Chores

    • Added runtime support for Gemini Vision integration and node configuration for profiles, API keys, and token limits.

Adds a new Accessibility Describe node powered by Google Gemini Vision
that generates scene descriptions optimized for blind and visually
impaired users.

Features:
- Structured output: hazards, spatial layout (clock positions), text/OCR,
  people detection, and navigation guidance
- Safety-first prompting — always leads with potential hazards
- Configurable spatial format (clock positions vs relative directions)
- Configurable hazard priority level
- Supports Gemini 2.5 Flash, 2.5 Pro, and 2.0 Flash models
- Retry logic with exponential backoff
- Follows existing vision node patterns (image lane in, text lane out)

Built as part of the AltVision project — AI glasses for the visually
impaired using real-time Gemini Live streaming + RocketRide pipelines.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Mar 21, 2026 •

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds an accessibility-describe image node: a global lifecycle installs local deps and initializes a Gemini-backed Chat; an instance streams image chunks into a data URL, sends a Question to the Chat, and emits text output. Includes package exports, requirements, and node service configuration with profiles and tests.

Changes

Cohort / File(s) Summary
Node lifecycle & instance
nodes/src/nodes/accessibility_describe/IGlobal.py, nodes/src/nodes/accessibility_describe/IInstance.py
Adds IGlobal that installs local requirements and constructs/clears a shared Chat on begin/end lifecycle; adds IInstance to buffer streamed image chunks, encode to data URL, create a Question, call IGlobal.chat.chat(...), and write text output.
Vision Chat & exports
nodes/src/nodes/accessibility_describe/accessibility_vision.py, nodes/src/nodes/accessibility_describe/__init__.py
Implements Chat (Gemini Vision integration with prompt/system templates, token heuristics, retry/backoff, error mapping) and AccessibilityVisionError; package __init__ re-exports IGlobal, IInstance, and provides getChat().
Service config & deps
nodes/src/nodes/accessibility_describe/services.json, nodes/src/nodes/accessibility_describe/requirements.txt
Adds node service registration (accessibility_describe://, image→text lane, profiles for Gemini models, configurable prompts/behavior, tests) and a local requirement google-genai >=1.14.0.

Sequence Diagram

sequenceDiagram
    participant Instance as IInstance
    participant Global as IGlobal / Chat
    participant Gemini as Google Gemini API

    Instance->>Instance: Receive image chunks (BEGIN → WRITE → END)
    Instance->>Instance: Assemble bytes, base64-encode into data URL, build Question
    Instance->>Global: chat(question)
    Global->>Global: Extract image from Question.context, set prompt/system instructions
    Global->>Gemini: generate_content(image bytes + prompt + system_instruction)
    alt Success
        Gemini-->>Global: Analysis result (text)
    else Retryable error
        Global->>Gemini: Retry (up to 3x, exponential backoff)
        Gemini-->>Global: Success or final error
    else Non-retryable error
        Global-->>Instance: Raise formatted AccessibilityVisionError
    end
    Global-->>Instance: Return Answer with text
    Instance->>Instance: Write text output, clear buffer
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~30 minutes

Poem

🐰 I nibble bytes and watch scenes unfold,
I stitch bright pixels into stories told,
Gemini guides where hazards hide and place,
I hop and whisper context, prompt, and grace,
A rabbit’s note to make vision a kinder space.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main addition: a new accessibility_describe node leveraging Google Gemini Vision for visually impaired users, matching the changeset's primary purpose.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@nodes/src/nodes/accessibility_describe/accessibility_vision.py`:
- Around line 152-155: The loop that checks context items currently uses two
separate startswith calls; update the check inside the loop over
question.context (the context_item variable in the same block that assigns
image_data and breaks) to use a single tuple argument to startswith (e.g.,
context_item.startswith(('data:image/', 'data:application/'))) so the test is
more efficient and readable while preserving the image_data assignment and break
behavior.
- Around line 102-105: The except block in accessibility_vision.py that
initializes self._client catches Exception as e and re-raises a ValueError
without chaining; change the re-raise to use exception chaining (raise
ValueError(f'Failed to initialize Google AI client: {str(e)}') from e) so the
original traceback is preserved when genai.Client(api_key=api_key) fails.
- Line 28: Replace the deprecated typing imports and type annotations: change
the import line "from typing import Any, Dict, Tuple" to only "from typing
import Any" (remove Tuple and Dict imports), then update all type hints in this
module that use typing.Dict[...] to use built-in dict[...] and any
typing.Tuple[...] usages to built-in tuple[...] (or remove/adjust if Tuple is
unused); ensure references in functions/classes like
accessibility_vision-related functions are updated accordingly and remove the
unused Tuple import.
- Around line 163-203: Move the deterministic data-URL parsing out of the retry
loop: parse image_data once before the for attempt in range(max_retries + 1)
loop (split header, extract mime_type, base64.b64decode into image_bytes) and
handle/report any parsing errors immediately; then inside the loop build the
request using the already-parsed image_bytes and mime_type (e.g., reuse
Part.from_bytes(data=image_bytes, mime_type=mime_type) and
Part.from_text(text=prompt_text)) and only retry the call to
self._client.models.generate_content (model=self._model, contents=contents,
...). Ensure variable names image_bytes, mime_type, and contents/parts are
referenced inside the loop and preserve existing retry/backoff logic
(last_error, _shouldRetry, max_retries).

In `@nodes/src/nodes/accessibility_describe/IGlobal.py`:
- Around line 33-45: Add a PEP 257 docstring to the IGlobal class describing its
purpose and runtime role in the pipeline; update the class declaration for
IGlobal to include a top-of-class triple-quoted docstring that succinctly
explains what the node does (e.g., manages global accessibility describe
initialization and creates Chat via beginGlobal), lists important attributes
(like glb, IEndpoint, _chat) and any expected configuration/side effects, and
follows PEP 257 style (one-line summary, optional short description paragraph,
and attribute annotations if useful).
- Around line 30-31: The class attribute chat on IGlobal is unused while the
instance uses self._chat; either remove the unused declaration or make the
property consistent: remove the line "chat: ChatBase | None = None" from class
IGlobal, or rename the internal field _chat to chat across the class (update all
references in IGlobal methods such as where self._chat is assigned/used) and
update IInstance.py to use self.IGlobal.chat instead of self.IGlobal._chat so
all callers refer to the same attribute.

In `@nodes/src/nodes/accessibility_describe/IInstance.py`:
- Around line 43-77: The writeImage method currently falls through with an
implicit None for unknown AVI_ACTION values; add an explicit return for the
unhandled case (e.g., append a final `return None` or `return
self.preventDefault()` at the end of writeImage) to satisfy RET503. While
editing, you can simplify the control flow by changing the `elif` blocks after
early `return` statements into separate `if` checks (keep references:
writeImage, AVI_ACTION, preventDefault) so the function is clearer and always
returns explicitly.
- Around line 40-41: The class-level attribute image_data: bytearray = None
should be moved into instance initialization to avoid suggesting a mutable
default at class scope; remove the class-level assignment and instead set
self.image_data = None (or initialize a new bytearray when needed) inside the
IInstance.__init__ (or the constructor method used) so each instance gets its
own buffer and the type annotation remains via the instance attribute or by
using a dataclass/typing comment if needed.

In `@nodes/src/nodes/accessibility_describe/services.json`:
- Around line 85-105: The config fields accessibility.prioritizeHazards and
accessibility.spatialFormat are defined but unused; update Chat.__init__ in
accessibility_vision.py to read and store these settings (e.g.,
self._prioritize_hazards = config.get('accessibility.prioritizeHazards', 'high')
and self._spatial_format = config.get('accessibility.spatialFormat', 'clock')),
then incorporate them when building the model prompt or messages (modify how
accessibility.systemPrompt/accessibility.prompt are composed or pass them as
parameters to the message generation function) so the Chat class actually honors
the enum values ("high"/"medium"/"low" and "clock"/"relative"/"both");
alternatively remove the two config entries from services.json if you prefer not
to support them.
- Around line 176-199: The test cases in the "test" block reference missing
image files "images/street.jpg" and "images/indoor.jpg" which will break tests;
either add those two image assets to the repo at the referenced paths or update
the "cases" image fields to use existing test images (for example replace
"images/street.jpg" / "images/indoor.jpg" with an existing file like
"testdata/images/ocr.jpg") so the "Street scene description" and "Indoor scene
with text" cases point to valid images; edit the image entries in the cases
inside services.json accordingly and ensure consistency with other vision nodes
(ocr, embedding_image, thumbnail).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 3089b823-a835-4ad1-9ccb-3a38075289fd

📥 Commits

Reviewing files that changed from the base of the PR and between 5bdcd5b and 2b60b2d.

📒 Files selected for processing (6)
  • nodes/src/nodes/accessibility_describe/IGlobal.py
  • nodes/src/nodes/accessibility_describe/IInstance.py
  • nodes/src/nodes/accessibility_describe/__init__.py
  • nodes/src/nodes/accessibility_describe/accessibility_vision.py
  • nodes/src/nodes/accessibility_describe/requirements.txt
  • nodes/src/nodes/accessibility_describe/services.json

@Rod-Christensen Rod-Christensen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome... let's get coderabbit to quit complaining about it and get it merged!

- Use built-in dict/tuple instead of deprecated typing.Dict/Tuple
- Add exception chaining (raise ... from e)
- Merge startswith calls into single tuple argument
- Move deterministic data URL parsing outside retry loop
- Rename _chat to chat to match class attribute declaration
- Add class docstrings to IGlobal
- Move image_data init into __init__ to avoid mutable class-level default
- Add explicit return None for unmatched actions in writeImage
- Implement prioritizeHazards and spatialFormat config consumption
- Fix test image paths to use existing testdata images

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@nodes/src/nodes/accessibility_describe/accessibility_vision.py`:
- Around line 226-227: The code currently raises a generic Exception in
accessibility_vision.py (see the raise Exception(user_friendly_error) inside the
error path that uses self._format_user_error), which prevents callers from
distinguishing accessibility vision failures; define a custom exception class
(e.g., class AccessibilityVisionError(Exception): pass) near the top of the file
and replace the raise Exception(...) with raise
AccessibilityVisionError(user_friendly_error) so callers can catch
AccessibilityVisionError specifically; ensure import/exports or module-level
documentation is updated if needed.
- Around line 219-224: Remove the redundant else: break following the continue
in the retry loop: inside the block that checks "if attempt < max_retries and
self._shouldRetry(e):" keep the delay calculation and time.sleep(base_delay * (2
** attempt)) followed by continue, and remove the else branch so execution
naturally falls through to the break after the if; update the loop around
variables attempt, max_retries, base_delay and the call to self._shouldRetry(e)
accordingly to maintain the same behavior without the unnecessary else.

In `@nodes/src/nodes/accessibility_describe/IInstance.py`:
- Around line 69-70: IInstance.py is accessing the private Chat attribute
self.IGlobal.chat._prompt which breaks encapsulation; add a public accessor on
the Chat class (in accessibility_vision.py) such as a prompt property or
get_prompt() that returns self._prompt, then update IInstance.py to call the new
public member (e.g., self.IGlobal.chat.prompt or self.IGlobal.chat.get_prompt())
instead of accessing _prompt directly so all references use the public API;
ensure the new property/method has a brief docstring and preserves existing
behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: a8fabcd1-d64b-4fe2-8188-13ba430a55cc

📥 Commits

Reviewing files that changed from the base of the PR and between 2b60b2d and c41dc4b.

📒 Files selected for processing (4)
  • nodes/src/nodes/accessibility_describe/IGlobal.py
  • nodes/src/nodes/accessibility_describe/IInstance.py
  • nodes/src/nodes/accessibility_describe/accessibility_vision.py
  • nodes/src/nodes/accessibility_describe/services.json

- Remove redundant else:break after continue in retry loop
- Define AccessibilityVisionError custom exception instead of generic Exception
- Add public prompt property to Chat, stop accessing private _prompt externally
- Chain last_error in final raise for full traceback preservation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@nodes/src/nodes/accessibility_describe/accessibility_vision.py`:
- Around line 187-188: Replace the redundant check "if question.questions and
len(question.questions) > 0:" with a single truthiness check on the questions
attribute (e.g., "if question.questions:") so the code that sets prompt_text =
question.questions[0].text uses the existing truthiness of question.questions to
guard against empty or None lists; update the condition near the prompt_text
assignment to remove the len(...) call and rely on "question.questions" only.
- Around line 193-197: The parsing of the data URL (using image_data → header,
b64_data; deriving mime_type; base64.b64decode to image_bytes) can raise
ValueError/IndexError/binascii.Error for malformed input, so wrap that parsing
block in a try/except that catches ValueError, IndexError, and
(base64.)binascii.Error and raise or log a new clear ValueError that includes a
concise message like "malformed data URL" plus the original exception text and
the offending image_data prefix (not full payload) to aid debugging; update any
callers of this code in accessibility_vision.py to propagate or handle the
raised error as appropriate.

In `@nodes/src/nodes/accessibility_describe/IInstance.py`:
- Around line 55-64: When handling AVI_ACTION.END in the method that builds
Question and constructs image_base64/image_data_url, first validate that
self.image_data is not None (and ideally has content/length) before calling
bytes(self.image_data); if it is None, short-circuit with a clear error path
(raise a descriptive ValueError or log via the same logger and return/abort) so
bytes(self.image_data) cannot raise TypeError. Update the block around the
AVI_ACTION.END branch that references self.image_data, image_base64 and
image_data_url to perform this defensive check and handle the empty/missing
image case consistently with the surrounding error handling.
- Around line 50-53: When handling AVI_ACTION.WRITE in the method that appends
image chunks, guard against self.image_data being None (which can happen if
WRITE arrives before BEGIN or BEGIN failed) by initializing it to an empty
bytes/bytearray before performing the append; ensure the initialization is
compatible with the incoming buffer type and then proceed to self.image_data +=
buffer and return self.preventDefault(). Update the WRITE branch that currently
references AVI_ACTION.WRITE, self.image_data, and preventDefault() to include
this defensive check and initialization.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 28d8cc78-a9d2-49cd-ac32-18cdfb9d269c

📥 Commits

Reviewing files that changed from the base of the PR and between c41dc4b and 742ac8d.

📒 Files selected for processing (2)
  • nodes/src/nodes/accessibility_describe/IInstance.py
  • nodes/src/nodes/accessibility_describe/accessibility_vision.py

- Simplify redundant length check: `if question.questions:`
- Wrap data URL parsing in try/except for clear error on malformed input
- Guard WRITE action against None image_data before append
- Guard END action against None image_data before processing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@nodes/src/nodes/accessibility_describe/accessibility_vision.py`:
- Around line 204-215: The loop inside the function constructs the same contents
array on every retry; move the deterministic Content/Part construction (using
image_bytes, mime_type, prompt_text and the Content/Part classes) so it is
created once before the for attempt in range(max_retries + 1): loop and then
reused inside the try block, leaving only the API call and its error handling
inside the loop (keep the same exception handling and retry logic in
functions/methods that call the API).
- Around line 127-130: Replace the explicit str(e) in the exception raise with
the f-string conversion flag (!s) to satisfy Ruff/style guidance: in the block
that initializes genai.Client (the code assigning self._client =
genai.Client(api_key=api_key) inside the class/constructor in
accessibility_vision.py), change the raise ValueError(f'Failed to initialize
Google AI client: {str(e)}') from e to use f'Failed to initialize Google AI
client: {e!s}' and keep the "from e" chaining intact.
- Around line 195-202: The image data URL parsing currently catches ValueError
and IndexError but not the binascii.Error raised by base64.b64decode; update the
except clause that surrounds header, b64_data = image_data.split(',', 1),
mime_type = header.split(':')[1].split(';')[0], and image_bytes =
base64.b64decode(b64_data) to also catch binascii.Error so invalid base64
content is converted into the same user-friendly ValueError describing a
malformed data URL (refer to header, b64_data, mime_type, image_bytes and
base64.b64decode in accessibility_vision.py).

In `@nodes/src/nodes/accessibility_describe/IInstance.py`:
- Around line 57-69: The current END handling (when action == AVI_ACTION.END)
only checks self.image_data is None but not whether it's empty, so an empty
bytearray will be encoded into an empty base64 payload and sent; update the
guard in the END branch to treat empty sequences as invalid by checking if not
self.image_data (or len(self.image_data) == 0) and call self.preventDefault()
(or return) in that case before building image_base64 and image_data_url
(references: AVI_ACTION.END, self.image_data, image_base64, image_data_url,
mimeType, preventDefault).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7dd78477-dffc-4381-bf0f-2fad68beaec9

📥 Commits

Reviewing files that changed from the base of the PR and between 742ac8d and f5b8694.

📒 Files selected for processing (2)
  • nodes/src/nodes/accessibility_describe/IInstance.py
  • nodes/src/nodes/accessibility_describe/accessibility_vision.py

- Use f-string conversion flag {e!s} instead of {str(e)}
- Add binascii.Error to data URL parsing exception handler
- Move contents array construction outside retry loop
- Guard END action against empty image_data (not just None)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@nodes/src/nodes/accessibility_describe/accessibility_vision.py`:
- Line 160: Change the double-quoted f-string to a single-quoted f-string for
consistency: update the return that currently reads f"Model '{self._model}' is
currently unavailable. Please try a different model." to use single quotes
around the whole f-string (keeping the inner single quotes around {self._model}
escaped or adjusted as needed) in the method that references self._model in
accessibility_vision.py so it adheres to the nodes/**/*.py string style
guidelines.
- Around line 182-183: The loop "for context_item in question.context" can raise
TypeError when question.context is None; update the code in
accessibility_vision.py to guard before iterating (e.g., check if
question.context is truthy or replace with iterating over (question.context or
[])) and ensure the existing "No image provided..." validation branch is reached
when question.context is None; reference the question.context usage in that
for-loop to locate and fix the issue.

In `@nodes/src/nodes/accessibility_describe/IInstance.py`:
- Around line 58-85: The END branch in IInstance currently sets self.image_data
= None only after a successful call to self.IGlobal.chat.chat(question), so if
chat(question) raises the image buffer remains and later WRITE can corrupt
payloads; wrap the call to self.IGlobal.chat.chat(question) (and any subsequent
processing that depends on it) in a try/finally and ensure self.image_data =
None is executed in the finally block (keeping the existing behavior of emitting
text when successful and returning self.preventDefault()); update the END
handling code around the question/chat/answer logic to always clear
self.image_data even on exceptions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 797dc67d-5c10-4fec-98ad-d6397441a8a4

📥 Commits

Reviewing files that changed from the base of the PR and between f5b8694 and 00c9391.

📒 Files selected for processing (2)
  • nodes/src/nodes/accessibility_describe/IInstance.py
  • nodes/src/nodes/accessibility_describe/accessibility_vision.py

- Use single-quoted f-string for model error message consistency
- Guard against None question.context before iteration
- Wrap END action inference in try/finally to always reset image_data

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@chinesepowered

Copy link
Copy Markdown
Contributor Author

This is awesome... let's get coderabbit to quit complaining about it and get it merged!

Resolved all coderabbit 🐰

@jmaionchi jmaionchi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

really nice addition @chinesepowered -- thank you!

@jmaionchi
jmaionchi merged commit f0794a6 into rocketride-org:develop Mar 23, 2026
10 checks passed
AviNormie pushed a commit to AviNormie/rocketride-server that referenced this pull request Nov 9, 2026
…ers (rocketride-org#317)

* feat(nodes): add accessibility_describe node for visually impaired users

Adds a new Accessibility Describe node powered by Google Gemini Vision
that generates scene descriptions optimized for blind and visually
impaired users.

Features:
- Structured output: hazards, spatial layout (clock positions), text/OCR,
  people detection, and navigation guidance
- Safety-first prompting — always leads with potential hazards
- Configurable spatial format (clock positions vs relative directions)
- Configurable hazard priority level
- Supports Gemini 2.5 Flash, 2.5 Pro, and 2.0 Flash models
- Retry logic with exponential backoff
- Follows existing vision node patterns (image lane in, text lane out)

Built as part of the AltVision project — AI glasses for the visually
impaired using real-time Gemini Live streaming + RocketRide pipelines.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(nodes): address CodeRabbit review feedback on accessibility_describe

- Use built-in dict/tuple instead of deprecated typing.Dict/Tuple
- Add exception chaining (raise ... from e)
- Merge startswith calls into single tuple argument
- Move deterministic data URL parsing outside retry loop
- Rename _chat to chat to match class attribute declaration
- Add class docstrings to IGlobal
- Move image_data init into __init__ to avoid mutable class-level default
- Add explicit return None for unmatched actions in writeImage
- Implement prioritizeHazards and spatialFormat config consumption
- Fix test image paths to use existing testdata images

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(nodes): address remaining CodeRabbit review comments

- Remove redundant else:break after continue in retry loop
- Define AccessibilityVisionError custom exception instead of generic Exception
- Add public prompt property to Chat, stop accessing private _prompt externally
- Chain last_error in final raise for full traceback preservation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(nodes): address new CodeRabbit review comments

- Simplify redundant length check: `if question.questions:`
- Wrap data URL parsing in try/except for clear error on malformed input
- Guard WRITE action against None image_data before append
- Guard END action against None image_data before processing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(nodes): address latest CodeRabbit review round

- Use f-string conversion flag {e!s} instead of {str(e)}
- Add binascii.Error to data URL parsing exception handler
- Move contents array construction outside retry loop
- Guard END action against empty image_data (not just None)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(nodes): address CodeRabbit review - quoting, null guard, cleanup

- Use single-quoted f-string for model error message consistency
- Guard against None question.context before iteration
- Wrap END action inference in try/finally to always reset image_data

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:nodes Python pipeline nodes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants