Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/internal/KNOWN-PITFALLS.md
Original file line number Diff line number Diff line change
Expand Up @@ -786,9 +786,10 @@ is 60s, so an MCP client that does not reset its timeout on progress
notifications gets `MCP error -32001: Request timed out` on the slim lease
even though the daemon completes it. Simlock relays boot progress as MCP
progress notifications precisely so clients can pass
`resetTimeoutOnProgress: true` (or a longer timeout) on `lease_simulator`;
`e2e/slow-ios-slim.test.ts` shows the call shape. The warm pool hides this
for every lease after the first.
`resetTimeoutOnProgress: true` (or a longer timeout) on `lease_simulator`,
as in `client.callTool(request, undefined, { resetTimeoutOnProgress: true,
timeout: 600_000 })`. The warm pool hides this for every lease after the
first.

**`launchctl disable` accepts labels that do not exist.** Verified on iOS
26.4 and 27.0 simulators: disabling `system/com.apple.does.not.exist` exits
Expand Down
14 changes: 7 additions & 7 deletions e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,13 +53,13 @@ running test body silently never fires — see the comment in `helpers/env.ts`.

## Known gaps

- **The slow lane does not pass reliably as a whole run.** Each of the three
flows passes on its own, but a combined `pnpm run test:e2e:slow` has failed on
every attempt so far: a daemon occasionally outlives `daemon stop` and trips
teardown's stray-process assertion, and the no-implicit-download flow has hit
its 300s timeout. Not yet root-caused, and not yet distinguished from the
reclaim stall recorded in [../docs/internal/KNOWN-PITFALLS.md](../docs/internal/KNOWN-PITFALLS.md).
Treat the slow lane as a manual, one-flow-at-a-time tool until that is fixed.
- **The slow lane does not pass reliably as a whole run.** A daemon
occasionally outlives `daemon stop` and trips teardown's stray-process
assertion; not yet root-caused, and not yet distinguished from the reclaim
stall recorded in [../docs/internal/KNOWN-PITFALLS.md](../docs/internal/KNOWN-PITFALLS.md).
The slim lane also fails while a 50-label disable batch can outrun its fixed
60s limit on a cold boot ([#206](https://github.com/callstackincubator/simlock/issues/206)). Treat the slow lane as a manual,
one-flow-at-a-time tool until both are fixed.
- `daemon status` ignores `--json` and always prints raw JSON; documented by an
`it.fails` in `daemon-lifecycle.test.ts`.
- Android is exercised through the fake driver in the fast lane; only
Expand Down
54 changes: 0 additions & 54 deletions e2e/capacity-cleanup-nuke.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -157,60 +157,6 @@ describe("capacity, warm pool, cleanup, and nuke", () => {
await waitForLeaseCount(env, 0);
});

it("reports overLimit when a lowered running limit can't yet be met, without evicting existing leases", async () => {
const env = await withDaemon({
configOverrides: { limits: { maxRunning: 3, ios: { maxDevices: 3, maxRunning: 3 } } },
});
await env.driverScript.set({
ios: { knownModels: ["iPhone 16"], availableOsVersions: ["18.4"] },
});

// `--detach`, so no holder process has to stay alive across the `daemon stop` that
// `withConfig` triggers to pick up the new limit. The leases themselves survive that
// stop either way now (ADR 0004 §3), which is what lets this exercise "a lowered limit
// doesn't evict an existing lease" against a genuinely still-leased device.
const leaseOne = await env.cli([
"lease",
"--platform",
"ios",
"--device",
"iPhone 16",
"--os",
"18.4",
"--agent-id",
"overlimit-one",
"--detach",
]);
const grantOne = leaseOne.json as { lease: { id: string } };
const leaseTwo = await env.cli([
"lease",
"--platform",
"ios",
"--device",
"iPhone 16",
"--os",
"18.4",
"--agent-id",
"overlimit-two",
"--detach",
]);
const grantTwo = leaseTwo.json as { lease: { id: string } };

await env.withConfig({ limits: { ios: { maxRunning: 1 } } }, async () => {
const afterLower = await status(env);
expect(afterLower.capacity.ios.overLimit).toBe(true);

// Pre-existing leases must never be evicted just because the limit dropped
// below what is currently held.
const leases = (await env.cli(["list", "--leases"])).json as { id: string }[];
expect(leases.map((lease) => lease.id)).toEqual(
expect.arrayContaining([grantOne.lease.id, grantTwo.lease.id]),
);
});

await env.cli(["release", "--all", "--yes"]);
});

it("demotes an idle device, previews cleanup without executing on --dry-run, and nuke touches only registry devices", async () => {
const env = await withDaemon({
configOverrides: {
Expand Down
47 changes: 0 additions & 47 deletions e2e/contention.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -69,55 +69,8 @@ describe("contention & queueing", () => {
waiterE.kill("SIGTERM");
await waiterE.waitForExit(15_000);
});

it("tells a waiter queued behind a reclaim how long that reclaim runs", async () => {
const env = await withDaemon({
configOverrides: { limits: { maxRunning: 1, ios: { maxDevices: 1, maxRunning: 1 } } },
});
await env.driverScript.set({
ios: {
knownModels: ["iPhone 16"],
availableOsVersions: ["18.4"],
estimateMs: { reclaim: 34_000 },
latencyMs: { reclaim: 4_000 },
},
});

const holder = await env.cli([...LEASE_ARGS, "--agent-id", "agent-holder", "--detach"]);
expect(holder.code).toBe(0);
const holderGrant = holder.json as { lease: { id: string } };

// Release hands the caller back before the purge (#58), so the device is `reclaiming` with
// its full latency still ahead when the next requester arrives -- the case the stage exists
// for, and the one nothing exercised before this covered the whole path out to the CLI.
const release = await env.cli(["release", holderGrant.lease.id]);
expect(release.code).toBe(0);

const waiter = env.cliBackground([...LEASE_ARGS, "--agent-id", "agent-waiter"]);
await waitFor(() => waiter.progressEvents().some(isReclaimingWithEta(34_000)), {
label: "agent-waiter told the reclaim's ETA",
});
expect(waiter.progressEvents().some((event) => isQueuedAt(event, 1))).toBe(true);

const granted = JSON.parse(await waiter.firstStdoutLine(15_000)) as {
device: { spec: { model: string } };
};
expect(granted.device.spec.model).toBe("iPhone 16");

waiter.kill("SIGTERM");
await waiter.waitForExit(15_000);
});
});

function isReclaimingWithEta(etaMs: number): (event: unknown) => boolean {
return (event) =>
typeof event === "object" &&
event !== null &&
(event as Record<string, unknown>).push === "progress" &&
(event as Record<string, unknown>).stage === "reclaiming" &&
(event as Record<string, unknown>).etaMs === etaMs;
}

function isQueuedAt(event: unknown, position: number): boolean {
return (
typeof event === "object" &&
Expand Down
67 changes: 0 additions & 67 deletions e2e/daemon-lifecycle.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,29 +30,6 @@ describe("daemon lifecycle & recovery", () => {
expect(countOccurrences(after, "Daemon started")).toBe(startedCountBefore);
});

it("simlock config renders the android.emulator block with its defaults", async () => {
const env = await withDaemon({ mode: "running" });

const config = await env.cli(["config"]);

expect(config.code).toBe(0);
expect((config.json as { android?: unknown }).android).toEqual({
emulator: { audio: true, bootAnimation: true, gpu: "auto", headless: false },
});
});

it("simlock config renders ios.defaultMode, full by default, and no on/off switch under ios.slim", async () => {
const env = await withDaemon({ mode: "running" });

const config = await env.cli(["config"]);

expect(config.code).toBe(0);
const ios = (config.json as { ios: { defaultMode: string; slim: Record<string, unknown> } })
.ios;
expect(ios.defaultMode).toBe("full");
expect(ios.slim).not.toHaveProperty("enabled");
});

it("recovers from a kill -9 that leaves a stale socket behind", async () => {
const env = await withDaemon({ mode: "running" });
expect(existsSync(env.socketPath)).toBe(true);
Expand Down Expand Up @@ -86,50 +63,6 @@ describe("daemon lifecycle & recovery", () => {
expect(statusHuman.stdout).toContain("Daemon running");
});

it("writes the documented structured startup lines to daemon.log", async () => {
const env = await withDaemon({ mode: "running" });

const contents = await readFile(env.logPath, "utf8");
const lines = contents
.trim()
.split("\n")
.filter((line) => line !== "")
.map((line) => JSON.parse(line) as Record<string, unknown>);

expect(lines.length, "daemon.log is empty after startup -- see report").toBeGreaterThan(0);

const startRecord = lines.find((line) => line.message === "Daemon started");
expect(startRecord, "no 'Daemon started' record in daemon.log").toBeDefined();
expect(startRecord).toMatchObject({
fields: expect.objectContaining({
version: expect.any(String),
protocolVersion: expect.any(Number),
socketPath: env.socketPath,
}),
});

const modules = new Set(lines.map((line) => line.module));
expect(modules.has("daemon.driver-discovery"), "no driver-discovery log lines").toBe(true);
expect(modules.has("daemon.connection-host"), "no connection-host log lines").toBe(true);
});

it("rotates daemon.log to daemon.log.1 once past config.log.rotateBytes", async () => {
const env = await withDaemon({ mode: "auto" });

await env.withConfig({ log: { rotateBytes: 200 } }, async () => {
await waitFor(() => existsSync(`${env.logPath}.1`), {
timeout: 15_000,
label: "daemon.log.1 created after low rotateBytes",
});
// Rotation renames daemon.log away and then opens a fresh one; with a 200-byte cap
// nearly every startup line rotates, so a one-shot check can land in that gap.
await waitFor(() => existsSync(env.logPath), {
timeout: 15_000,
label: "a fresh daemon.log opened after rotation",
});
});
});

it("stops while a detached lease is still outstanding", async () => {
const env = await withDaemon();
await env.driverScript.set({
Expand Down
79 changes: 0 additions & 79 deletions e2e/daemon-log.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,6 @@ async function logLines(env: TestEnv): Promise<LogLine[]> {
.map((line) => JSON.parse(line) as LogLine);
}

async function operationLines(env: TestEnv, operation: string): Promise<LogLine[]> {
return (await logLines(env)).filter(
(line) => line.message === "operation" && line.fields?.operation === operation,
);
}

const IOS = { ios: { knownModels: ["iPhone 16"], availableOsVersions: ["18.4"] } };

async function leaseDetached(env: TestEnv): Promise<string> {
Expand Down Expand Up @@ -65,79 +59,6 @@ async function reservePort(): Promise<number> {
}

describe("daemon.log", () => {
it("a lease and its release over the socket each leave one operation line naming the requester", async () => {
const env = await withDaemon({ agentId: "log-agent" });
await env.driverScript.set(IOS);

const leaseId = await leaseDetached(env);
const released = await env.cli(["release", leaseId]);
expect(released.code, released.stderr).toBe(0);

await waitFor(async () => (await operationLines(env, "lease.release")).length > 0, {
label: "the lease.release line",
});
const requests = await operationLines(env, "lease.request");
const releases = await operationLines(env, "lease.release");
expect(requests).toEqual([
expect.objectContaining({
level: "info",
fields: expect.objectContaining({
principal: "log-agent",
requesterId: "log-agent",
durationMs: expect.any(Number),
}),
}),
]);
expect(releases).toEqual([
expect.objectContaining({
level: "info",
fields: expect.objectContaining({
principal: "log-agent",
leaseId,
durationMs: expect.any(Number),
}),
}),
]);
});

it("a lease request for an unknown model leaves an operation line with its error code", async () => {
const env = await withDaemon();
await env.driverScript.set({
ios: { failures: { resolveSpec: { type: "UnknownModelError", model: "Bogus Phone" } } },
});

const lease = await env.cli([
"lease",
"--platform",
"ios",
"--device",
"Bogus Phone",
"--detach",
]);
expect(lease.code).not.toBe(0);

expect(await operationLines(env, "lease.request")).toEqual([
expect.objectContaining({
level: "info",
fields: expect.objectContaining({ code: "UNKNOWN_MODEL" }),
}),
]);
});

it("at the default level simlock status leaves no operation line; at debug it leaves one", async () => {
const quiet = await withDaemon();
expect((await quiet.cli(["status"])).code).toBe(0);
expect(await operationLines(quiet, "status.get")).toEqual([]);

// `daemon start` asks for status itself, so the count is taken around this one call.
const verbose = await withDaemon({ configOverrides: { log: { level: "debug" } } });
const before = (await operationLines(verbose, "status.get")).length;
expect((await verbose.cli(["status"])).code).toBe(0);
const after = await operationLines(verbose, "status.get");
expect(after).toHaveLength(before + 1);
expect(after.at(-1)).toMatchObject({ level: "debug" });
});

it("a scripted boot failure leaves a line naming the device and the driver's error text", async () => {
const env = await withDaemon();
await env.driverScript.set({
Expand Down
Loading
Loading