Skip to content

fix: improve browser input readiness and observed form state - #261

Merged
iuyo5678 merged 7 commits into
mainfrom
fix/browser-interaction-state
Sep 16, 2026
Merged

iuyo5678 merged 7 commits into
mainfrom
fix/browser-interaction-state

Conversation

@iuyo5678

@iuyo5678 iuyo5678 commented Sep 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Hidden pages can receive native input before their renderer is ready, and observations do not distinguish checkbox state from submitted values.

  • Prepare hidden click, press and wheel input with bounded rendering readiness and temporary focus emulation, then restore focus state after success, cancellation or failure. Readiness waits use the caller's remaining deadline and respect configured timeout defaults.
  • Record whether the actual click, key press or wheel event was dispatched. Failures before dispatch remain distinguishable from unknown input outcomes, including errors returned by an action and cancellation responses passing through the daemon. Parameter errors, cancellation and action failures retain their diagnostics; only readiness failures receive the readiness hint. Click and press join wheel in cancelling at the daemon deadline and retaining the session queue during bounded cleanup.
  • Give acknowledged wheel input with unconfirmed paint a dedicated input_paint_unconfirmed reason and CLI guidance to observe the scroll position before continuing. Keep effect_state: unknown, existing error codes and no automatic replay, including after deadline or user cancellation.
  • Preserve checkbox/radio checked state separately from values, and render disabled state from both DOM attributes and usable AX state, including inherited disabled state. Clicking retains browser-native behavior; there is no additional AX query or disabled preflight on each click.
  • Invalidate tab refs on root-document and attachment changes, reject observations spanning a document change, and preserve root refs when only a child frame changes.
  • Reject full-page capture on an already-hidden page before changing its scroll position or styles. Keep explicit CLI guidance when input may already have taken effect or temporary state could not be restored.

Validation

  • Extension TypeScript compile and build passed; extension tests: 1,741 passed, 97 optional tests skipped.
  • The isolated real-Chrome fixture covers visible/hidden input and native, inherited and ARIA disabled controls. With DOM events explicitly enabled, ordinary observe-to-click preserves refs and produces no document invalidation; navigation produces DOM.documentUpdated and invalidates old refs. It also verifies that session detach clears a leftover focus override.
  • The browser fixture runs in its own CI job, with failures still failing the workflow. Frontend lint, typecheck, tests and build complete independently.
  • Daemon queue integration tests: 9 passed, including click/press deadline cleanup and preservation of input effects and paint diagnostics after deadline or user cancellation.
  • Rust error-rendering tests: 19 passed. Workspace clippy with all targets, Rust formatting and repository Biome checks passed.

Scope and limitations

Readiness and acknowledgement do not prove that every application has processed an input. Failed post-input confirmation remains an unknown outcome and does not trigger automatic replay. Cleanup has a bounded grace period; an unconfirmed cleanup remains an explicit error.

File-transfer orchestration, child-frame event initialization, fill/hover readiness and a new focus-lease layer are outside this change. Hidden file-transfer failures and incomplete stale-ref handling after out-of-process iframe navigation were also observed on unmodified main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants