feat(fleet): a lease request may require a capability

docs/CLI.md

@@ -96,6 +96,7 @@ command starts it again) to bring the platform up.
 | 11 | `DEVICE_BUSY` | `device add` or `device remove` of a device that is leased, being reclaimed or held by another operation, such as another `device add`; try again once that ends |
 | 11 | `COMPONENT_BUSY` | `component remove` while a component install or removal runs or waits on that platform; try again once it ends |
 | 12 | `NO_DRIVER` | no driver registered for the requested platform |
+| 12 | `CAPABILITY_MISSING` | through a gateway, no worker that could serve the request has every capability it requires available; direct to a worker, that worker lacks one (a request names capabilities only through the programmatic client) |
 | 12 | `RUNTIME_MISSING` | runtime not installed and no `--allow-download`, or no installed image of the `--image-tag` asked for |
 | 12 | `UNKNOWN_MODEL` | unknown device model for the platform |
 | 12 | `INSUFFICIENT_DISK_SPACE` | not enough free disk space to install a component |
@@ -947,6 +948,7 @@ catalog has not arrived, is neither.
 | ... and no known worker lists the model, or, for a class, a model of it | `UNKNOWN_MODEL` (exit 12) at once |
 | ... and no known worker has the runtime, or can pair it with the model (or with a model of the class) | `RUNTIME_MISSING` (exit 12) at once; `downloadable` is `false`, and `osVersion` is the range as you typed it, or `default` when you named none |
 | A known worker could serve it, but none that takes requests can | `NO_CAPACITY` (exit 11) at once |
+| The request requires capabilities (programmatic client only), and no connected worker that could serve it has all of them available | `CAPABILITY_MISSING` (exit 12) at once; `capabilities` names the required ones |
 | A worker that takes requests can serve it but is busy | Waits in the queue; `NO_CAPACITY` only with `--no-wait` |
 
 "At once" holds with and without `--no-wait` and `--timeout`, and the same

docs/CLIENT.md

@@ -177,6 +177,18 @@ code, not something this client does for you. There is no heartbeat
 to declare and no connection-liveness mode to opt into: a renew arriving
 before the deadline is the whole mechanism.
 
+**Requiring a capability.** `requestLease` also takes `requires`: up to 8
+lowercase capability names, none twice, such as `["agent-device"]`. Simlock
+places the lease only on a worker whose status lists every one of them as
+`available`. Through a gateway, a request that no connected worker able to
+serve it can take fails at once with `CAPABILITY_MISSING`, and so does one
+already waiting when the last such worker loses the capability or restarts. If
+no worker stays connected, it fails with `NO_CAPACITY` instead. Direct to a worker that lacks one it fails the same way.
+The error's
+`details.capabilities` names the missing capabilities. `requires` cannot be
+combined with `idempotencyKey` (`BAD_REQUEST`). The CLI, the MCP tool and the
+HTTP lease body do not take it.
+
 **Getting an answer back after a disconnect.** Pass `idempotencyKey` to
 make a request repeatable:
 

docs/EVENTS.md

@@ -15,7 +15,7 @@ through `simlock events` and `simlock events --follow`.
 | `lease.renewed` | lease id, new deadline | a `lease.renew` succeeded — whether it came from `simlock lease renew`, `POST /v1/leases/{id}/renew`, or the renew timer a running `simlock lease` / MCP session keeps over its own lease. There is one renew path and this is it | LeaseLifecycle | implemented |
 | `lease.released` | lease id, device id, reason (explicit/killed/device-lost), owner id | an explicit `lease.release` (which is what a `simlock lease` holder does on its way out), (killed) an operator `release --all` or `nuke`, or (device-lost) a leased device could not be recovered after it stopped running outside simlock, or a daemon start found its device not running, or (device-lost) a physical device was not present for the grace period (`health.stableObservations` reads in a row, `health.probeIntervalMs` apart; the device becomes `absent` and nothing is wiped), and the device of that lease was wiped and returned to the pool, left waiting in `reclaiming` on a platform the daemon could not list, or marked missing. Closing a connection is not a release and never emits this | LeaseLifecycle | implemented |
 | `lease.expired` | lease id, device id, owner id | the lease's deadline passed with no `lease.renew` behind it — the grant-time TTL, or the TTL of the last renew, simply ran out. This is the one way a lease ends without somebody asking, and the only bound on a holder that was killed outright | LeaseLifecycle | implemented |
-| `lease.rejected` | request id, requester, request spec (as on `lease.requested`: a request that named a class or nothing has no model, and one that named an OS range carries it as typed), reason (timeout/no-wait/unresolvable-spec/no-worker/already-leased/lease-id-taken/boot-timeout/killed/cancelled/daemon-restarted/worker-failed), and for `worker-failed` also `code` and `worker` | a request ended without a grant. A worker emits it only for its own, local requests: a request a gateway sent it ends in `lease.declined` there instead. `already-leased` means the requester already has a waiting request, or already holds `lease.maxPerRequester` leases. A worker declines a gateway's request (`lease.declined`) as `already-leased` only for a waiting request, never for a lease it holds. A request refused before it was stored (`killed`, `already-leased`, `lease-id-taken`) emits no `lease.requested`, and its `lease.rejected` carries the id it would have been stored under (on a gateway, `lease-id-taken` can also end a request that was stored, when a worker's grant carries an ID the gateway already routes elsewhere: that one has its `lease.requested`); on a gateway, `no-worker` is a request no worker that takes requests can serve, `NO_CAPACITY` at once, and `unresolvable-spec` is one no known worker has the platform, model, or runtime for (for a physical request, an enrolled device that fits it), and such a request emits no `lease.queued`. A request a worker refused as unable to serve (`RUNTIME_MISSING`, `UNKNOWN_MODEL`, `NO_DRIVER`) while another worker was busy waits in the gateway queue, unless it is a `noWait` request, which is rejected at once with reason `no-wait` and emits no `lease.queued`; a request that finds no worker left emits `unresolvable-spec`, after its `lease.queued` when it had queued; `worker-failed` is a request that failed on the worker it went to (a refusal that is final, a failure after the worker began work, an unreachable worker, or a dispatch that timed out), with `code` the error code the caller got and `worker` the worker's id; `daemon-restarted` is a request still waiting when the daemon stopped, settled as failed when it starts again. The reason list can grow: a consumer must tolerate a reason it does not know; `cancelled` is an explicit single-request cancel (backing `DELETE /v1/lease-requests/{id}`) of a still-queued waiter — one with device work already in flight is reported `not-cancellable` instead, the same envelope the queue timeout already uses; a physical request has no such work, only a presence read, so it is cancelled in any state | LeaseAcquisitionCoordinator / WaitQueue / LeaseStartup (worker) / FleetLeaseCoordinator (gateway) | implemented |
+| `lease.rejected` | request id, requester, request spec (as on `lease.requested`: a request that named a class or nothing has no model, and one that named an OS range carries it as typed), reason (timeout/no-wait/unresolvable-spec/no-worker/already-leased/lease-id-taken/boot-timeout/killed/cancelled/daemon-restarted/worker-failed), and for `worker-failed` also `code` and `worker` | a request ended without a grant. A worker emits it only for its own, local requests: a request a gateway sent it ends in `lease.declined` there instead. `already-leased` means the requester already has a waiting request, or already holds `lease.maxPerRequester` leases. A worker declines a gateway's request (`lease.declined`) as `already-leased` only for a waiting request, never for a lease it holds. A request refused before it was stored (`killed`, `already-leased`, `lease-id-taken`) emits no `lease.requested`, and its `lease.rejected` carries the id it would have been stored under (on a gateway, `lease-id-taken` can also end a request that was stored, when a worker's grant carries an ID the gateway already routes elsewhere: that one has its `lease.requested`); on a gateway, `no-worker` is a request no worker that takes requests can serve, `NO_CAPACITY` at once, and `unresolvable-spec` is one no known worker has the platform, model, or runtime for (for a physical request, an enrolled device that fits it), or one that requires capabilities (`CAPABILITY_MISSING`) no connected worker able to serve it has available, and such a request emits no `lease.queued`. A request a worker refused as unable to serve (`RUNTIME_MISSING`, `UNKNOWN_MODEL`, `NO_DRIVER`, `CAPABILITY_MISSING`) while another worker was busy waits in the gateway queue, unless it is a `noWait` request, which is rejected at once with reason `no-wait` and emits no `lease.queued`; a request that finds no worker left emits `unresolvable-spec`, after its `lease.queued` when it had queued; `worker-failed` is a request that failed on the worker it went to (a refusal that is final, a failure after the worker began work, an unreachable worker, or a dispatch that timed out), with `code` the error code the caller got and `worker` the worker's id; `daemon-restarted` is a request still waiting when the daemon stopped, settled as failed when it starts again. The reason list can grow: a consumer must tolerate a reason it does not know; `cancelled` is an explicit single-request cancel (backing `DELETE /v1/lease-requests/{id}`) of a still-queued waiter — one with device work already in flight is reported `not-cancellable` instead, the same envelope the queue timeout already uses; a physical request has no such work, only a presence read, so it is cancelled in any state | LeaseAcquisitionCoordinator / WaitQueue / LeaseStartup (worker) / FleetLeaseCoordinator (gateway) | implemented |
 | `lease.declined` | request id, gateway request id (`fleetRequestId`), requester, request spec (as on `lease.rejected`), reason (the values of `lease.rejected`) | a worker refused or failed a request a gateway sent it, for any reason and however far the work got. The gateway owns that request's outcome and may try another worker, so the worker declines rather than rejects, and never queues such a request. A worker that was asked and has no room declines with `no-wait`; one that cannot serve the request, or fails it, declines with the reason it would have rejected a local request with. A request still open when the worker stopped is declined with `daemon-restarted` at the next start | LeaseAcquisitionCoordinator / WaitQueue / LeaseStartup (worker) | implemented |
 
 On a **gateway**, `lease.requested`, `lease.queued` and `lease.rejected` are its own fleet queue's facts,

docs/HTTP-API.md

@@ -536,6 +536,7 @@ known and a reconnecting one stays known from its last catalog.
 | ... and no known worker lists the model | `422 UNKNOWN_MODEL` |
 | ... and no known worker has the runtime, or can pair it with the model | `422 RUNTIME_MISSING`, with `downloadable: false` and `osVersion: "default"` when `os` is absent |
 | A known worker could serve it, but none that takes requests can | `503 NO_CAPACITY` |
+| The request requires capabilities (not a body field; the programmatic client sends it), and no connected worker that could serve it has all of them available | `422 CAPABILITY_MISSING`, with `capabilities` naming the required ones |
 | A worker that takes requests can serve it but is busy | queues; `503 NO_CAPACITY` only with `noWait: true` |
 
 A request that is already waiting is held to the same table: when the only
@@ -1599,7 +1600,7 @@ Every failure is the same shape the daemon protocol uses:
 | 403 | `FORBIDDEN` (role doesn't permit the route — including a `worker` token on any `/v1` route other than `/v1/uplink`, and an `agent`/`operator` token at `/v1/uplink`; a `/v1/lease-requests/*` route whose request another token sent; or `POST /v1/leases/{id}/renew`/`DELETE /v1/leases/{id}`/`POST /v1/leases/{id}/exec` naming another requester's still-live lease), `DOWNLOADS_DISABLED` (`POST /v1/components/install` under `downloads.policy: "never"`) |
 | 404 | `UNKNOWN_WORKER` (`POST`/`DELETE /v1/workers/{id}/drain` naming a worker the gateway does not know), `UNKNOWN_LEASE_REQUEST` (unknown request id), `UNKNOWN_LEASE` (unknown lease id, expired/released, **or `GET /v1/leases/{id}`/`GET /v1/leases/{id}/events` naming another requester's lease** — see [`GET /v1/leases/{id}`](#get-v1leasesid)) , `UNKNOWN_DEVICE` (`DELETE /v1/devices/{id}` naming a device that is not enrolled) |
 | 409 | `REQUESTER_ALREADY_LEASED` (body names the existing lease id; fleet-wide on a gateway), `LEASE_ID_TAKEN` (a `leaseId` an active lease or a waiting request holds; body carries `leaseId`; on a gateway, one of the gateway's own leases or requests, or a worker that refused it), `IDEMPOTENCY_CONFLICT` (an `Idempotency-Key` repeated with a different device or `leaseId`), `REQUEST_NOT_CANCELLABLE` (body names the lease id if the request had already been granted), `WORKER_CONNECTED` (`DELETE /v1/workers/{id}` while its uplink is open), `COMPONENT_NOT_OWNED`, `COMPONENT_IN_USE` (body carries `devices` and `foreignDevices`), `COMPONENT_BUSY` (the three refusals of `DELETE /v1/components/{platform}/{version}`), `DEVICE_BUSY` (`POST /v1/devices`, `DELETE /v1/devices/{id}` of a leased, reclaiming or held device; body carries `id` and `reason`) |
-| 422 | `UNKNOWN_MODEL`, `RUNTIME_MISSING`, `NO_DRIVER`, `DEVICE_NOT_ENROLLABLE` (`POST /v1/devices`; body carries `id`, `platform`, `reason`, and `canonicalId` when known), `PASSTHROUGH_REFUSED` (a refused `exec` verb or `devicectl` command, a caller-supplied `--set`/`-P`, a bare `adb shell`), `UNKNOWN_PASSTHROUGH_TOOL` |
+| 422 | `UNKNOWN_MODEL`, `RUNTIME_MISSING`, `NO_DRIVER`, `CAPABILITY_MISSING` (a request required capabilities no worker able to serve it has available; body carries `capabilities`), `DEVICE_NOT_ENROLLABLE` (`POST /v1/devices`; body carries `id`, `platform`, `reason`, and `canonicalId` when known), `PASSTHROUGH_REFUSED` (a refused `exec` verb or `devicectl` command, a caller-supplied `--set`/`-P`, a bare `adb shell`), `UNKNOWN_PASSTHROUGH_TOOL` |
 | 501 | `UNSUPPORTED_IN_GATEWAY_MODE` (an operation that acts on one machine, asked of a gateway: `POST /v1/components/install`, `GET /v1/components`, `DELETE /v1/components/{platform}/{version}`, `POST /v1/devices`, `DELETE /v1/devices/{id}`), `UNSUPPORTED_IN_WORKER_MODE` (an operation on a gateway's workers, asked of a single host: `POST`/`DELETE /v1/workers/{id}/drain`, `DELETE /v1/workers/{id}`, `POST /v1/components/install` with `workers`) |
 | 503 | `NO_CAPACITY` (with `noWait: true`, or, on a gateway, when no worker that takes requests can serve the request; response carries `Retry-After`), `WORKER_UNREACHABLE` (a gateway could not reach the worker holding this lease or request) |
 | 504 | `EXEC_TIMEOUT` (a `device.exec` command outlived `exec.timeoutMs`), `DOWNLOAD_TIMEOUT` (a runtime download, waiting for another download included, outlived `downloads.timeoutMs`) |