You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
One SwiftUI toolbar control reaches press/wait as two actionable elements (AMBIGUOUS_MATCH) #2480
$ agent-device snapshot -i --json # nodes with id scoring_home_button Other "scoring_home_button" rect [20, 63, 36, 36] (no hittability evidence) Button "scoring_home_button" rect [20.667, 63, 35, 36] (no hittability evidence)
$ agent-device press 'id="scoring_home_button"' --settleError (AMBIGUOUS_MATCH): Selector matched 2 distinct actionable elements: id="scoring_home_button"Candidates: @e3 [other] "Home" @e4 [button] "Home"
$ agent-device wait'id="scoring_home_button"' 3000Error (AMBIGUOUS_MATCH): selector matched multiple elements
Root cause (TypeScript)
packages/selectors/src/interaction-targeting.ts:
candidatesFormSingleAncestryChain passes - the pair really is one ancestor-descendant chain.
findPreferredActionableDescendant filters same-rect children with a verified-hittability requirement. Regular iOS snapshots omit hittability evidence, so that branch can never fire on iOS.
The rects differ by <= 1 pt, beyond RECT_EQUALITY_TOLERANCE = 0.5, so areRectsApproximatelyEqual is false too.
So the wrapper resolves to itself, the control resolves to the semantic target, the classifier calls the pair ambiguous, and the documented "collapse duplicate accessibility wrappers" guarantee cannot hold for the most common SwiftUI toolbar shape.
Root cause (runner side). The native wait refusal is produced by apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift:1683 and RunnerTests+Interaction.swift:238, which refuse multi-match without the shared structural-equivalence rules.
Impact
An agent's existence check reports "the screen never appeared", and its press refuses, for a control that is uniquely addressable for a human. In our harness this produced false scenario failures that looked like app bugs.
Fix
Branch fix/unverified-hittability-wrapper-chain (fork prateekranka/agent-device) adds packages/selectors/src/interaction-targeting-wrapper-chain.ts plus a small classifier change: when every candidate lacks hittability evidence and all rects agree within 1 pt, collapse to the deepest semantic touch target. Candidates carrying any hittability fact keep today's rules. Verified red -> green on a simulator; gate 304 files / 2010 tests.
Runner question. Should the runner delegate multi-match to the shared equivalence rules, or mirror them? I can prepare that patch; it needs a simulator lane run.
Activity