Skip to content

test(e2e): agent-device through a gateway on real devices #518

Closed

Description

@V3RON

Part of #511.

Scope

After this PR, the slow lane proves agent-device through a gateway on real
devices, iOS and Android: a gateway and one worker on one machine, the real
agent-device CLI pointed at the gateway. It also moves simlock to the
agent-device build that sends the lease header: the vendored
@agent-device/proxy tarball is packed again from simlock-integration at
253dca023edf13410f7928c76e9e604e5e5871a0 (callstack/agent-device#3594
merged in; version 0.21.28-dev).

In short

flowchart LR
  A[agent-device CLI, real] -->|connect proxy| G[simlock gateway process]
  G -->|uplink| W[simlock worker process, agentDevice.enabled]
  W --> D[agent-device daemon, real]
  D --> S[simulator / emulator, real]
Loading

The suite runs only where the existing "agent-device through simlock" slow
lane row's checks pass, with the same SIMLOCK_E2E_AGENT_DEVICE build.
Unset, both describes are skipped.

If this goes wrong, the slow lane passes while agent-device through a
gateway is broken on real devices, or fails for a reason in the test rig.

Technical spec

Modules touched

  • e2e/slow-agent-device-gateway.test.ts (new): one describe per
    platform, tagged slow plus ios or android, skipped like
    e2e/slow-agent-device.test.ts's.
    • A gateway (driver: "none", agentDevice.enabled) and a worker joined
      with a worker token, with agentDevice: { enabled: true, command: <SIMLOCK_E2E_AGENT_DEVICE> }, maxDevices: 1 and a warm-up in
      beforeAll, as the existing suite pins them. Its fleet start waits until
      simlock worker list on the gateway shows the worker connected with
      agent-device available, up to 120 s.
    • A test that changes a config (the gateway's or the worker's
      agentDevice.enabled) restarts that daemon with it, waits up to 120 s
      for simlock worker list to show the state it expects, and in
      afterEach restores the config and waits the same way for agent-device available.
    • The devices test runs in a nested describe whose beforeAll
      restarts the worker with the device limit (capacity.config.limits) at
      2, warms up two devices and waits until the gateway's simlock worker list shows the worker available with capacity 2, up to 420 s; its
      afterAll restores the limit to 1 the same way, as the existing suite's
      "with the device limit raised to 2" block does.
    • Agents are made with makeAgent pointed at the gateway's
      /v1/agent-device.
  • e2e/helpers/agent-device-targets.ts (new, exported from
    e2e/helpers/index.ts): hasIosRuntime, succeeds, Target, IOS,
    ANDROID, FIRST_SNAPSHOT, ok, failureText, waitForFree,
    listDevices, listLeases, listRequests, listRows and the row types
    DeviceRow, LeaseRow, RequestRow move here from
    e2e/slow-agent-device.test.ts, which imports them from it. setLimits
    and warmUp, closures inside its defineSuite today, become module
    functions there that take the daemon and target they used to close over.
    No helper is copied.
  • vendor/: agent-device-proxy-0.21.28-dev-253dca0.tgz, packed with
    pnpm --filter @agent-device/proxy pack at that commit and renamed to
    carry the short commit, as the pin check in e2e/helpers/agent-device.ts
    reads it; the old tarball removed. package.json's
    @agent-device/proxy dependency and vendor/README.md name the new file
    and commit; the lockfile follows.
  • e2e/slow-agent-device.test.ts: adjusted only where the new build's
    output changed (its snapshot change, agent-device commit 746283685
    "make a node's ref optional"), so it passes with the new build.
  • docs/internal/agent-rules/toolchain.md: the "agent-device through
    simlock" row names the new test file in its Run line.

Contract and event changes

None.

Other code on the same state

  • e2e/slow-agent-device.test.ts uses the same build and the same
    machine; the slow-lane script runs one file at a time, so the two never
    share devices.

Rules in play

  • toolchain.md: slow lane, one lane per machine, run through
    scripts/slow-e2e.sh.
  • testing.md: each line fails on a named assertion for its own reason.

Tests

Seam: real processes and real devices through scripts/slow-e2e.sh,
modelled on e2e/slow-agent-device.test.ts. Each line runs on iOS and on
Android.

  • through the gateway, an agent opens a device, installs the Appduct fixture app as an upload, opens it, takes a snapshot that shows tab-status, saves a PNG screenshot and closes; within 240 s the device is free and the lease is gone from the gateway's simlock list
  • through the gateway, a screenshot saved as an artifact downloads to the agent's machine as a PNG
  • with the device limit raised to 2 and two devices ready, agent-device devices through the gateway after open lists exactly the agent's device
  • with agentDevice.enabled unset on the gateway, its /v1/agent-device/health answers 404 and no lease is made
  • with the worker's agentDevice.enabled unset (the gateway's simlock worker list shows agent-device off), open through the gateway fails within 5 s with a message containing no worker connected to this gateway can serve agent-device, and no lease is made
  • simlock worker list on the gateway shows the worker with agent-device available

Done when

  • On a Mac with an iOS runtime and the toolchain row's checks, scripts/slow-e2e.sh e2e/slow-agent-device-gateway.test.ts passes every test in the iOS describe.
  • On a machine with emulator, adb and the row's checks, the same command passes every test in the Android describe.
  • With SIMLOCK_E2E_AGENT_DEVICE unset, the same command writes <log>.exit 0 and shows both describes skipped.
  • On the same two machines, scripts/slow-e2e.sh e2e/slow-agent-device.test.ts passes every test in its iOS describe and in its Android describe, with the new build.

Out of scope

  • A second worker on another machine.

Depends on

Approval

  • Approved for delivery

Written by an agent.

Activity

  1. added
    task:draftScope written; technical spec or its spec check missing.
    on Oct 13, 2026
  2. changed the title [-]feat(agent-device): a disconnected worker is not an ended lease, and a gateway restart keeps the device[/-] [+]test(e2e): agent-device through a gateway on real devices[/+] on Oct 13, 2026
  3. added
    task:readySpec checked. An agent may implement it once its Depends on allow.
    and removed
    task:draftScope written; technical spec or its spec check missing.
    on Oct 13, 2026
  4. self-assigned this
    on Oct 13, 2026
  5. added 3 commits that reference this issue on Oct 14, 2026
    3793b83
    64f3a4f
    8d699f9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

task:readySpec checked. An agent may implement it once its Depends on allow.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions