[Lower priority] Keyboard state introspection (visibility, type, dismiss) #102
Description
Activity
Adding a concrete use case for this:
When automating Android app login flows, the keyboard frequently covers bottom-of-screen buttons (e.g., "Next" or "Submit"). Currently the only workaround is dropping to raw ADB:
adb shell input keyevent KEYCODE_BACK
This breaks the abstraction that agent-device provides. A simple
agent-device keyboard dismiss(or automatic keyboard dismissal beforepress/find...clickwhen the target is behind the keyboard) would eliminate the most common reason for falling back to raw ADB in Android automation flows.For context, this came up while using agent-device to automate a WebView-based login flow on Android emulator (API 36). The sequence was:
fill @e2 "phone-number"— keyboard appearsfind "Next" click— fails silently because keyboard covers the button- Had to
adb shell input keyevent KEYCODE_BACKto dismiss, then retry the click
Priority: Lower / backlog.
Ability to query: is the keyboard up? What type (e.g. email, number)? Can the agent dismiss it reliably? This would improve robustness of flows that depend on keyboard state (e.g. fill then dismiss keyboard then tap button). May require snapshot extensions or platform-specific APIs.
Acceptance criteria (high level):