[brennanb2025]

	Files	Added	Deleted	Net
Test	0	0	0	0
Prod	2	$\color{#1a7f37}{\Huge{\mathbf{+}}}$​222	$\color{#cf222e}{\Huge{\mathbf{−}}}$​23	$\color{#1a7f37}{\Huge{\mathbf{+}}}$​199

Problem

Linux computer inspection can time out after 30 seconds on a Flutter app even though its controls are readable. The provider reads every control's position while collecting its semantic information, so stalled native position reads can consume the whole request. Tracked in STA-11477.

User-facing change

Inspection returns readable controls and supported actions when child positions stall. Unavailable positions become null frames; they do not mark the readable tree truncated. Requested screenshots and responsive positions remain available. The same observation path runs after an action, preserving the fresh result without replaying the input.

Mechanism

Resolve the app/window and read fresh window bounds after any restoration, using the existing native allowance. Standard window roles do not need position reads to establish their identity; other roles retain the existing positive-bounds predicate and timeout. Collect the semantic tree, focus, selection and metadata, then capture the requested screenshot.

Only then start a fresh three-second phase for emitted controls' positions. Each native component/extent read gets at most 800ms, skips when less than 50ms remains, disables startup grace for that call, and restores the 800ms/15s baseline in finally. Associate native nodes with exact record objects through browser compaction. Reuse the selected root rectangle; keep no discovery or restoration rectangle cache.

Intended differences and limits

• Complete structured semantics remain available for subsequent indexed actions. This differs from passive flat-text or partial-tree capture in the common pattern because those outputs do not supply Orca's action-target identity contract.
• Required window identification, root bounds, semantic reads and explicit restoration retain their existing native behavior. The three-second budget applies only to optional descendant geometry; it is not a whole-request deadline.
• Coordinate fallback still requires live positions and keeps its existing timeout behavior. Semantic actions can target frameless controls.
• No saved schema, wire fields, capability, minimum server version, action replay or bridge deadline changes.

Review and validation

Two review rounds and a three-seat architecture ruling corrected five initial regressions: semantic work exhausting the position budget, slow nonstandard windows being rejected, benign protected-name labels blocking an app, stale bounds after restoration, and incorrect frames after browser compaction. A fresh full review passed, followed by a narrow check of the final main merge at 4dae758.

All 17 targeted Python tests pass. Independent main/original/final probes verify the five corrections; these use simulated native boundaries and do not prove installed-library timing. The scoped diff is 72 changed production lines and 173 changed test lines; the larger test harness pins the incident and confirmed regressions.

Final-head CI passed, including static analysis/typecheck, all five test shards, relay integration, packaging and native smoke. A dedicated performance/resource scan found no proven introduced issue. Final real Linux QA passed on Ubuntu 24.04 with PyGObject 3.48.2, libatspi 2.52.0 and GTK 3.24.41, using the exact packaged final-head runtime through Orca's paired RPC route.

• A real native stalled-position fixture returned all 68 controls in 3.308 seconds on the branch versus 18.576 seconds with the main provider through the same packaged server. Both completed within the unchanged 30-second bridge limit. A frameless semantic click changed persisted target state exactly once and returned its new value.
• Official Flutter Gallery 2.10.2 returned 42 controls on main and head; paired head inspection took 0.305 seconds. The reporter-owned ERP was unavailable, so Gallery supplies toolkit coverage; Gallery itself did not reproduce stalled positions. The real-libatspi fixture separately exercises that failure and remains a private QA artifact.
• Two healthy GTK windows, name/PID selectors, screenshots, secure-value redaction, blocked-app behavior, restoration, responsive coordinate fallback and previous target state through real target/server restarts passed. Plain coordinate input changed saved state identically on main/head after the private test registry's display setup was corrected.
• Main-provider RPC comparison used the supported provider-path override in the final packaged server, not an entire main package. Final-head restart removed the override. No renderer or agent folder changed; the real packaged CLI/RPC consumer was exercised.
• All owned Linux processes exited, the light-job gate is released, real config hashes match, and the remote rig was removed. Local evidence remains with the owning review lane.

The inspection change does not fix required root/semantic stalls or coordinate-input timeouts. A three-second descendant-position budget is not a whole-request deadline.

The existing duplicate browser action-index issue is tracked separately in STA-11731. Real GTK set-value fails identically on main and this head because the installed GI Accessible lacks is_editable_text; this unchanged baseline issue is tracked in STA-2596, assigned to Brennan with fresh native evidence. Neither text nor numeric set-value success is claimed. Per the review lane ruling, set-value acceptance for this PR is main parity; the separate defect remains open.

The unchanged internal frame-origin inconsistency when root bounds are unavailable is tracked in STA-11781. Current action consumers reread live positions or safely refuse; the automated thread was resolved after an independent exact-main/head trace.

Visual QA

Main GTK after coordinate input:

Branch GTK after the same input and previous-build state:

Flutter Gallery (main/head byte-identical):

[coderabbitai]

📝 Walkthrough

The Linux AT-SPI runtime adds a shared timeout budget for geometry reads. It renders accessibility records before collecting their geometry, then assigns window-relative frames when geometry is available. Window discovery accepts standard window roles without requiring readable geometry. Tests cover snapshot data, timeout handling, window selection and restoration, accessibility actions, and compacted browser records.

Priority: ➖ Normal

Merge Risk: 🟡 Moderate · up to 5fdee

Some snapshots can report incorrect control positions, so the coordinate contract should be fixed before merging.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 28 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Title check		The title clearly and concisely describes the main change: preserving readable Linux controls when position reads stall.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Description check		The description is detailed and covers the problem, user-facing behavior, implementation, limits, testing, validation, and visual proof. It does not use the template headings exactly, and it reference…

Full details: Description check

Explanation

The description is detailed and covers the problem, user-facing behavior, implementation, limits, testing, validation, and visual proof. It does not use the template headings exactly, and it references STA-11477 without a linked Fixes # issue, but the required information is otherwise substantially present.

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Repository UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 7d8dbad7-57fa-460c-b5c1-cdda25f97ef0

📥 Commits

Reviewing files that changed from the base of the PR and between 48660ae and 5fdee96.

📒 Files selected for processing: 2

• native/computer-use-linux/runtime.py
• native/computer-use-linux/runtime_render_test.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 2 remain after this review.

[brennanb2025]

Linux inspection could lose an otherwise readable control tree when child position reads stalled until the 30-second bridge deadline. This change collects semantics first and treats descendant positions as bounded optional enrichment. Controls and supported semantic actions remain available with missing frames; requested screenshots and responsive positions remain available.

The full review loop is complete on 4dae75816256cd5eda21677520e9f1a25f12148a, including the final main merge. Two review rounds and an architecture panel corrected five initial regressions: request age consuming the position budget, slow nonstandard windows being rejected, benign protected-name labels blocking an app, stale bounds after restoration, and frames attached to the wrong records after browser compaction. Independent final review, readiness and performance passes found no remaining introduced issue. All 17 targeted Python tests and current-head CI pass; no unresolved review threads remain.

Real Linux QA used Ubuntu 24.04, PyGObject 3.48.2, libatspi 2.52.0 and GTK 3.24.41, the exact CI AppImage and Orca's actual paired RPC route. The main-provider comparison used the supported provider override in the same packaged server; it was not a full main package. Final restart used the unmodified packaged branch provider.

Real check	Main provider	Branch
Native stalled descendant positions, screenshot enabled	68 controls in 18.576s	68 controls in 3.308s; late controls retain semantics and null frames
Healthy Flutter Gallery	42 controls in 0.145s	42 controls in 0.141s; paired route 0.305s
Plain coordinate input on identical saved GTK state	Saved count 1→2 in 0.116s	Saved count 1→2 in 0.384s, fresh returned tree and matching screenshot

A frameless semantic click through paired RPC changed durable state exactly once (count 1→2) and returned the changed tree in 3.319s. Two GTK windows, name/PID selectors, requested screenshots, password redaction, blocked-app behavior, restoration and state carried through real target/server restarts passed. After restart, stalled inspection retained 68 controls in 3.318s, then healthy recovery took 0.311s. All owned processes exited, the Linux light-job gate is free, real config hashes match, and the remote rig is removed.

Limits and separate issues:

• The reporter-owned ERP was unavailable. Authorized official Flutter Gallery supplies real Flutter toolkit coverage; Gallery itself did not stall. The separate real-libatspi fixture exercises stalled calls and remains a private QA artifact.
• Text and numeric set-value fail identically on main and head because installed GI lacks is_editable_text. Native mutation success is not claimed. The lane explicitly accepted main parity for this unchanged action; fresh evidence is recorded in existing STA-2596, assigned to Brennan.
• Existing duplicate action indexes are tracked in STA-11731. The unchanged internal frame-origin inconsistency when root bounds are unavailable is tracked in STA-11781; current actions reread live positions or refuse safely. The automated review thread was resolved after an independent main/head trace.
• Required window/root/semantic reads and coordinate fallback retain baseline timeout behavior. The fresh three-second budget covers optional descendant positions, not the entire request. No saved schema, protocol, minimum server version or input replay changes.

For Brennan's review

The Flutter main/head captures are byte-identical; both labels below intentionally use the same uploaded image. GTK main/head images show the same retained previous-build text, masked password and successful coordinate count. Initial occluded GTK captures were excluded.

Main: Flutter Gallery	Branch: Flutter Gallery

Main: GTK after coordinate action	Branch: GTK after coordinate action

Branch second GTK window:

Three review points: native-call timeout restoration, preserved window-identification allowance, and fresh root, semantics, screenshot, then optional child positions.