Skip to content

test(ios): add live system permission-prompt accept/dismiss/recovery journey #320

Open
Open
test(ios): add live system permission-prompt accept/dismiss/recovery journey#320
Labels
enhancementNew feature or requestready-for-agent

Description

@thymikee
thymikee
opened on Mar 28, 2026

Purpose

Add the one remaining live iOS system-permission journey that fixture-backed simulator coverage cannot replace: native permission-prompt accept, dismiss, and recovery through the public CLI.

#1408 already owns and enforces the fixture-app smoke/full split, stable selectors, cold deep links, keyboard lifecycle, background/foreground resume, modal handling, interrupted system UI, long-list recovery, host-focus preservation, and catalog-level command coverage. Existing Settings/system-app coverage remains additive and must not be removed.

Required journey

Using the repository fixture app on an iOS simulator:

  1. Reset one real OS permission to an undecided state and launch the app path that requests it.
  2. Prove alert get observes the native system prompt and exposes actionable buttons.
  3. Exercise dismissal/denial through the public alert command and verify the prompt is gone plus the app/session remain usable.
  4. Reset to undecided, trigger the prompt again, accept it through the public command, and verify the granted native permission state plus continued app interaction.
  5. Exercise the no-prompt recovery case and require the documented typed outcome rather than a hang, stale modal route, or leaked session/device ownership.

Do not substitute a fixture-owned modal or synthetic alert: the test must cross the SpringBoard/native permission boundary.

Observable completion

  • The journey runs in the appropriate iOS simulator E2E lane against an exact built fixture artifact.
  • Assertions cover the native permission state, prompt disappearance, app usability after each branch, and clean session/lease teardown.
  • The test is observed red with one relevant production or test invariant deliberately broken before the final green run.
  • Focused checks and the affected gate pass; GitHub remains authoritative for the exact-head native lane.

Non-goals

  • Migrating or deleting existing Settings/system-app replays.
  • Re-testing fixture behaviors already owned by test(ios): expand simulator e2e coverage #1408.
  • Broad permission-matrix coverage across every permission type; one representative real native prompt is sufficient for this contract.

Depends on the existing fixture build/install infrastructure from #1408.

Activity

  1. thymikee
    added
    enhancementNew feature or request
    ready-for-agent
    on Jun 25, 2026
  2. thymikee
    added a commit that references this issue on Jul 16, 2026

    refactor(ci): extract setup-fixture-app so any job can use the cached…

    0aad123

  3. thymikee commented on Jul 16, 2026

    @thymikee
    thymikee
    on Jul 16, 2026
    MemberAuthor

    The main blocker for this issue is now removed. Update from the layer-3 conformance work in #1289.

    What changed

    "Suggested Starting Point" #1 — build a small fixture/demo app with stable ids — was already done: examples/test-app (Agent Device Tester, com.callstack.agentdevicelab) has stable testIDs across Home / Catalog / Product detail / Checkout form / Settings.

    What was actually missing was that no CI job could get that app onto a device — nothing in the repo built or installed it, which is why every replay still drives com.apple.Preferences. #1289 needed a real app to run a Maestro differential against, so it built that capability, and it is now a reusable composite action:

    - uses: ./.github/actions/setup-fixture-app
  with:
    runtime-version: ${{ env.IOS_RUNTIME_VERSION }}
# outputs: app-path, app-id, cache-hit

    It builds the app (Expo Release, no Metro needed), installs it on the booted simulator, verifies the install, and fails loudly if it is missing.

    The cache is shared across jobs, which is what makes this affordable. Building costs ~22 min (79% of a 30 min job); the built .app is cached keyed on the app's sources, dependency graph, iOS runtime and Xcode version. GitHub caches are per-repository and readable across workflows — a run restores from its own branch or the default branch — so once a run on main populates it, every workflow gets the hit and only the first pays the build. Cache-hit path is seconds.

    Proven working: https://github.com/callstack/agent-device/actions/runs/29514541333 built and installed it, and the Maestro differential drove real flows against it (tapOn home-open-form → Checkout form, percentage swipes, optional assertions).

    What this unblocks, concretely

    Point 2's split is now actionable. The current suite shows exactly the tax this issue describes — from test/integration/replays/ios/simulator/01-settings.ad:

    click "role=cell label=General || role=button label=General"
wait  "label=About || label=\"Software Update\" || text=\"Manage your overall setup and preferences\""

    That selector hedging exists only because Apple reshuffles Settings between iOS versions. Against the fixture those become single stable ids.

    Candidates to port (all six currently drive Settings):
    01-settings · 02-deep-navigation · 03-scroll-discovery · 04-text-input-keyboard · 05-app-lifecycle · 06-swipe-gestures

    The fixture already covers the patterns point 4 asks for: modal open/close, async loading, long-list scroll, text entry + keyboard dismiss, choice groups, destructive-confirm modal.

    One caution, matching point 2

    Do not drop system-app coverage entirely. Settings is currently the only thing exercising real OS chrome — system alerts, permission prompts, home/app-switcher, deep links — which an RN app cannot reproduce, and which is where real bugs have surfaced (the SpringBoard system-alert probe work). This issue already says as much; worth restating now that porting is easy, because "port everything" is the tempting next step. Target shape: fixture app for behavioral coverage (stable, fast, deterministic), a thin system-app canary retained for OS-integration reality.

    Suggested sequencing

    1. Land test: replace the hand-typed Maestro fixture with a generated conformance oracle #1289 (the action arrives with it).
    2. Port 02–04 and 06 (pure interaction coverage) to the fixture; measure the flake/duration delta.
    3. Reduce 01-settings to the minimal OS-integration canary rather than deleting it; keep 05-app-lifecycle on a system app if it covers home/app-switcher.
    4. Point 5's "does not steal host focus" regression is independent of this and still outstanding.

    Also worth evaluating (not a blocker)

    rnrepo distributes prebuilt native artifacts for RN libraries. The fixture qualifies (RN 0.85.3, New Arch, Expo CNG plugin) and it would speed the cache-miss rebuild — but the cache already takes the hit path to zero, and only 3 of 17 deps are candidates (gesture-handler, screens, safe-area-context), so it is incremental, not the main lever. It also adds an external artifact host to CI's trust surface, which is worth deciding deliberately rather than as a build-speed side effect.

  4. thymikee commented on Jul 16, 2026

    @thymikee
    thymikee
    on Jul 16, 2026
    MemberAuthor

    Decision: the Settings coverage stays. Correcting my previous comment, which went further than agreed.

    I suggested "reduce 01-settings to the minimal OS-integration canary rather than deleting it" (sequencing step 3). That is not the call — the Settings suite is kept.

    So fixture-app coverage is additive, not a migration:

    • Keep the existing system-app replays as they are. They cover real OS chrome — system alerts, permission prompts, home/app-switcher, deep links, launch/close — which an RN fixture cannot reproduce, and which is where real bugs have surfaced.
    • Add fixture-app cases for behavioral coverage where stable ids buy determinism: selectors, fill, scroll, keyboard, modals, lifecycle.
    • Nothing is removed or trimmed as part of this.

    This does narrow the win compared to my earlier framing — the selector-hedging tax in the Settings flows ("role=cell label=General || role=button label=General") stays where it is. The gain is that new and expanded coverage can be written against a controlled app instead of adding more OS-churn surface, which is closer to what this issue's point 2 actually says ("keep system-app coverage where it validates platform-specific behavior") than my "port and reduce" reading.

    The enabling capability is unchanged and still lands with #1289: ./.github/actions/setup-fixture-app, with the build cached and shared across workflows.

  5. thymikee
    added a commit that references this issue on Jul 16, 2026

    test: replace the hand-typed Maestro fixture with a generated conform…

    856d5d4

  6. thymikee commented on Aug 17, 2026

    @thymikee
    thymikee
    on Aug 17, 2026
    MemberAuthor

    From the #1781 sweep: most of this landed piecemeal without referencing the issue — the fixture-app smoke/behavior split exists (test/integration/ios-simulator-e2e/behavior-coverage.ts, incl. host-focus-preservation), and Android got fixture-app coverage in #1482. Still unverified from the original list: deep-link cold start, background/foreground resume, modal/sheet, permission recovery. Suggest rescoping the body to those four or closing as superseded by #1408/#1482 — maintainer's call.

  7. thymikee
    changed the title [-]test: move replay coverage toward stable mobile app patterns[/-] [+]test(ios): add live system permission-prompt accept/dismiss/recovery journey[/+] on Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions