Skip to content

Selector queries have no fallback producer on the iOS Simulator, and any fallback needs producer equivalence #2274

Closed

Description

@benjaminkomen

Split out of #2273 at triage request, to keep that issue scoped to the typed failure contract.

What this asks for

A selector query on the iOS Simulator has no fallback producer. Where the snapshot path degrades,
snapshot switches to the private-ax producer and still returns the element, and a selector query
for that same element fails.

This issue tracks giving the selector path a fallback, and the equivalence it would have to hold.

The observation behind it

Measured on agent-device 0.20.10, iOS Simulator, in a remote EAS Simulator session. Full method
and diagnostics are in #2273.

Six screens, 30 is visible calls each. Three screens answered every call. Three answered almost
none, all with XCTEST_RECORDED_FAILURE.

The split follows one thing only, which each capture states in its own header.

screen nodes capture truncated and fell back queries
screen 1 189 no answer
screen 2 177 no answer
screen 3 188 no answer
screen 4 901 no answer
screen 5 178 yes fail
screen 6 170 yes fail
screen 7 149 yes fail

Node count does not predict it in either direction. Screen 4 holds 901 nodes and answers. Screen 7
holds 149 and fails.

A capture taken immediately before each probe held the element every time. So the element was
present and readable while the query for it failed.

Why this is not simply "add the fallback"

Triage notes that #2197 establishes the Simulator snapshot producer and presentation contract
first, and that this work should follow it. We agree, and we want to add one requirement from a
caller's side.

Producer equivalence. A selector resolved through the fallback producer must match identifiers
and labels the same way the primary producer matches them. If the two producers expose different
identifiers, or propagate labels differently, then a query answered from the fallback could report
an element absent when it is present.

For us that would be worse than the failure we reported. A failure stops a flow and names itself. A
false negative passes a check that should have failed, or fails a check that should have passed, and
neither names the producer.

We cannot measure that equivalence ourselves. We tried. We captured one screen twice in one
run, once clean and once degraded, and the two captures turned out to be different states of the
screen rather than one state read two ways. So we have no data on how the two producers differ, and
we are not claiming any.

Acceptance tests we would find convincing

  • An element that is present is never reported absent, on either producer, for identifier and label
    selectors alike.
  • A true negative is reported identically by both producers, and with the same error code.
  • A screen that forces the fallback answers the same selector query as the same screen before it
    degraded.

What would unblock us sooner

Nothing here is urgent for us if #2273 lands. A typed reason that separates "the query could not
run" from "the element is absent" lets a flow retry or read the screen instead, which is enough to
work around this. The fallback is the better fix and the slower one.

Activity

  1. thymikee commented on Sep 3, 2026

    @thymikee
    Member

    Thanks for splitting this out and for stating the equivalence requirement explicitly.

    We checked the motivating behavior against current main (e2ce98556b). The get, is, and wait paths no longer use the direct XCTest querySelector shortcut that was present in 0.20.10. They now resolve selectors from a fresh snapshot capture, so the existing bounded private-AX snapshot fallback is already in the read path. Those shortcuts were retired in #1875, #1877, and #1883.

    This conclusion does not assume #2197 changes querySelector. The specific is visible path measured in this report no longer invokes it. The only remaining direct querySelector consumer is an offscreen-target safety probe used during mutation routing; its failure is treated as an unavailable probe and cannot produce a selector-read false negative.

    Focused selector-read, public-handler, and offscreen-probe coverage passes on current main (40 tests). We did not have access to the reported Expo app and EAS Simulator screen, so this is route-level confirmation rather than an exact-app live replay. The change will ship in 0.21, which we plan to release in the coming days.

    The producer-equivalence requirement remains valuable for #2197: snapshot-backed selector reads must preserve identifier and label semantics when its Simulator producer changes. I suggest retaining that as acceptance coverage for #2197 or a focused follow-up, while treating the original missing-fallback behavior as already resolved on main. Confirmation on the affected screens after 0.21 would still be useful.

  2. thymikee commented on Oct 2, 2026

    @thymikee
    Member

    Closing as resolved by the shortcut retirement. The behavior measured here (an is visible failing where the snapshot path still returned the element) depended on the direct XCTest querySelector read, which is gone from the get/is/wait routes (#1875, #1877, #1883): those commands now resolve selectors from a fresh snapshot capture (src/commands/interaction/runtime/selector-is.ts, selector-read-shared.ts, wait-selector.ts), so the existing bounded private-AX snapshot fallback is already in the read path, with backend conformance pinned (contracts/fixtures/ios-snapshot-backend-conformance.json). The only remaining direct-query consumer is the best-effort offscreen-target probe, which degrades to null rather than answering for a capture.

    If a current release still shows a selector read failing where snapshot -i returns the node, please open a fresh issue against that version; the fallback-producer-equivalence requirement stays valid as a design bar for any new producer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-infoWaiting on reporter or external input

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions