Source control: GitHub API transport, credentials and settings (A1) #638

Summary

Direct GitHub API transport with credentials, a header-driven quota ledger, a credential-scoped host pause, and Settings → Source Control. Phase A1 of #535. Ported from T3 Code (commit 83a82a46b; PRs #16319, #16322, #16986 there). gh is used only to obtain a token.

Credentials (crates/services/src/github/credentials.rs)

Per host (lowercased), in order: saved token (Settings, write-only, stored in secrets.json under the @github key (a key no profile id can take, so the file stays the flat profile map older builds read and preserve), read fresh each time) → GH_TOKEN / GITHUB_TOKEN for github.com and *.ghe.com; GH_ENTERPRISE_TOKEN / GITHUB_ENTERPRISE_TOKEN for other hosts only when GH_HOST equals that host → gh auth token --hostname H [--user ACCOUNT] via the services process helper, 10 s timeout, GH_PROMPT_DISABLED=1. Cache: gh token 5 min, "not signed in" 10 s, CLI failure 0; capacity 32; keyed by host or host+account. A disabled host fails with a typed error regardless of token. A 401 drops the cache so the next call re-reads. Identity: GET /user, cached 10 min per credential fingerprint (SHA-256 of the token, host-prefixed; the fingerprint is the only thing that leaves the module). A pinned credential refuses requests to another host. Tokens never appear in logs, argv, query variables or replicated state.

Transport (github/api.rs)

ureq (existing services dependency, rustls), called through HostCx::unblock. 8 concurrent requests; the permit is held by the executing job until I/O completes even if the awaiting task is dropped (cancellation is result suppression, not socket cancellation). Whole-response deadline 30 s (ureq 2 does not bound DNS resolution; a stalled lookup is bounded by the system resolver, and the job keeps its permit until it returns), body cap 8 MiB, X-GitHub-Api-Version: 2022-11-28, bearer auth, user-agent: tcode. URL mapping: github.com → https://api.github.com; *.ghe.com → https://api.HOST; GHES → https://HOST/api/v3, GraphQL at /api/graphql. REST: optional JSON body, If-None-Match, 304 accepted only when the caller sent a validator. GraphQL: POST {query, variables}; all user values are variables, never interpolated. Classification order: rate-limited (429; GraphQL RATE_LIMITED; GraphQL errors with remaining 0; untyped "rate limit exceeded"; 403 with remaining 0 / retry-after / "rate limit" body) → 401 unauthorized → 404 or GraphQL NOT_FOUND → any other GraphQL error (even with partial data) → 2xx/304 success → REST error with extracted messages. No automatic retry in the transport. Logs: path without query, operation, status, numeric quota headers, GraphQL document hash; never variables.

Quota ledger and host pause (github/quota.rs)

Per (host, resource, credential): store limit, remaining, reset from every response that names a resource; the latest arrival in the same window wins; no local debit (a free 304 or a failed request never drains a local counter). Background reserve 10% of the limit: background callers are admitted only above the floor; interactive callers may use the reserve but never an observed zero. A refusal spends no request and yields retry_at. Separate host-wide pause per (host, credential) on any refusal from either API: retry-after or x-ratelimit-reset when present, else 30 s·2^(n−1) capped at 15 min; generation-leased so an older success cannot clear a newer refusal; background callers get the pause as an error and wait, they never sleep in the transport.

Shared GraphQL building (github/graphql.rs)

Aliased documents (one alias per item, typed variables declared once, shared variables once) and a cursor pager (sequential pages, repeated-cursor guard, max_pages, explicit truncated result).

Settings

github.hosts[host] = {enabled, account} with a per-host patch (not whole-map replacement), Command::SetGitHubToken {host, token: Option}; replicated state shows only "token set" and the credential source (saved, env, gh login). Settings → Source Control card: per-host switch, account picker when gh has several logins, token field, source shown, "env token overrides the choice" note. Strings in both locales.

First consumer

services::version_check moves its unauthenticated release request onto the transport under its own anonymous ledger namespace; it keeps working when Source Control is disabled.

Evidence required

Offline through the real client entry: host mapping; token precedence and enterprise host binding; GraphQL 200-with-errors; typed and untyped rate limits; reserve versus interactive; 304 reuse; 401 re-fetch on the next call; an older success not clearing a newer refusal; pinned-host refusal; body cap and deadline; "a free 304 cannot drain a local balance". gh via an executable fixture at the process boundary (as the Cursor probe does). Live: gh auth token on a real keyring and one authenticated GET /user. Visual check of the card in both themes.

Reference

T3 Code: apps/server/src/sourceControl/GitHubCredentials.ts, GitHubApi.ts, githubQuota.ts, SourceControlRateLimit.ts, githubGraphQl.ts, GitHubApi.test.ts, githubQuota.test.ts.

Additions kept beyond this text

Reviewed in #650 against "necessary to deliver this issue, or clearly worth far more than it costs". Everything else that #647 added was removed there.

• Phone example --local (Phone example: --local runs the shell against an in-process host for screenshots #649): runs the phone shell against an in-process host on a throwaway profile, for screenshots without pairing. Documented in CONTRIBUTING → Verifying real behaviour. Kept by the lead's decision.
• "Add host" field in the card (necessary): without it, a GHES host that has no gh login and is not GH_HOST gets no row, so its saved token cannot be set. Upstream's token form also takes a host (GitHubTokenSettings.tsx).
• Command::RefreshGitHubCredentials (necessary): opening the Source Control section reruns gh discovery (account list and credential source), because gh logins change outside Tcode. There is no separate refresh button; re-entering the section rescans.
• Secret commands never enter the client's persisted outbox (necessary): Command::contains_secret. Otherwise a token would also land in the client's outbox file, not only in secrets.json. The same predicate covers the existing SetProfileSecret and CreateThirdPartyProfile (Principle 8). Because these commands are held only in memory, dispatching one while disconnected is refused, so a restart cannot drop it silently.
• Fail-closed secret writes (worth it, about 5 lines): a secrets.json that cannot be read is never overwritten. Before this, the first write after a parse failure erased every stored secret.
• secrets.json temp file created with mode 0600 (worth it, about 8 lines): tokens are never readable by other users during the write.
• Host validation (necessary): normalize_host plus the invalid_github_host command error. Hosts become storage keys and URL authorities.
• Discovery generation guard (worth it, about 3 lines): an older discovery result never overwrites a newer one.
• RequestOptions::{timeout, body_limit} (necessary): version_check keeps its existing 10 s and 1 MiB bounds on the shared transport. Both are capped at 30 s and 8 MiB.
• GitHubApi::new(credentials, ureq::AgentBuilder) (necessary): the network boundary the offline evidence above requires. Tests point the resolver and TLS at a loopback fixture. Production uses GitHubApi::host.
• ureq pooling off and redirects 0 (necessary for "no automatic retry"): ureq 2 retries requests on stale pooled sockets.
• sha2 dependency (necessary): real SHA-256 for fingerprints and GraphQL document hashes.
• Small port-only response checks: a 2xx GraphQL body that is not an object is InvalidResponse; a truncated GraphQL or JSON body is BodyTooLarge; REST responses carry truncated; a pinned credential re-checks that its host is still enabled.

Where this port follows upstream over the text above

• Background callers are admitted at the 10% floor (githubQuota.ts:85–92).
• NOT_FOUND applies only when all GraphQL errors are NOT_FOUND (GitHubApi.ts:329–335).
• Interactive callers bypass the host pause but not an observed zero (GitHubApi.ts:405).

Minor known differences from upstream, left as they are:

• RateLimited.retry_at reports the ledger's effective pause, not the raw header.
• The pause is checked before the quota.
• Non-2xx GraphQL responses get REST message extraction.
• GraphQL error decoding is lenient.
• No caller-supplied credential scope.
• A 401 invalidates every cached account for the host.
• Timeouts are capped at 30 s, where upstream lets callers raise them for large patch reads.