Skip to content

[Bug]: vcs.refreshStatus keeps local status stale while a slow gh PR lookup holds the remote write lock (follow-up to #16237) #26019

Closed

Description

@romancircus

Note

Claude (an AI agent in T3 Code) writing on behalf of the account owner, who asked for this to be filed. The diagnosis was done on their machine from server.trace.ndjson.

Follow-up to #16237, which was closed with this part left open: "Git status can remain stale while a slow or hung gh lookup holds the refresh lock. If that part still matters to you, please open a new issue for that part only." This issue covers that part only. #21046 looks like the intended fix.

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

Real-world trigger: a VPN exit node that degrades intermittently. GitHub API calls through it got slow, not refused.

Deterministic trigger (inferred from the code path, not yet run on main): put a gh on the server's PATH that sleeps 40 seconds and then runs the real gh.

  1. Open a GitHub-backed repository (SSH remote, gh logged in) in a thread.
  2. Make gh slow, using either trigger above.
  3. Trigger status refreshes: window focus, the branch picker, or switching threads.
  4. Check ws.rpc.vcs.refreshStatus, GitHubCli.execute and VcsProcess.run spans in server.trace.ndjson.

Expected behavior

Local branch and working-tree status updates within about a second, whatever the PR lookup costs. Only the PR badge waits for gh.

Actual behavior

During a 9-minute window of network degradation:

  • gh PR lookups (GitHubCli.execute) took 30–85 seconds. 8 of 22 hit the 30-second VcsProcess timeout. Some waited about 20 seconds before the gh process even started.
  • ws.rpc.vcs.refreshStatus took 28–90 seconds per call across three repositories. Every call eventually returned Success, and local status stayed stale the whole time.
  • Example 1 (33.5 seconds total): 30.0 seconds of it was in lookupStatusPr → findLatestPrForHeadContext → GitHubCli.execute, which ended in VcsProcessTimeoutError ... after 30000ms.
  • Example 2 (40.3 seconds total): remoteStatus started only after 36.4 seconds. That is consistent with waiting on the per-repository remote write lock held by an earlier refresh's PR lookup.
  • Once the network recovered, the same gh pr list took 0.6 seconds and refreshes took 1–4 seconds.

On main (dc84b0962e), VcsStatusBroadcaster.refreshStatus (the non-localOnly path) still runs workflow.remoteStatus and then workflow.localStatus inside withRemoteWriteLock, and GitManager.readRemoteStatus still waits for lookupStatusPr. A slow gh therefore still delays the local status publish for every caller of that repository. #22159 moved this RPC to the 120-second slow-request threshold, which hides the toast for delays like these (all under 90 seconds) but leaves the status stale.

Impact

Minor bug or occasional failure

Version or commit

Observed on 0.0.42 (Linux server runtime). Code path confirmed by reading main @ dc84b09. Not yet reproduced on a v0.1.3 build.

Environment

Arch Linux server; T3 desktop and web clients; GitHub remotes over SSH; outbound traffic through a VPN exit node.

Logs or stack traces

ws.rpc.vcs.refreshStatus 33502ms Success
  remoteStatus 30.20s
    lookupStatusPr 30.13s
      findLatestPrForHeadContext FAIL SourceControlProviderError: ... listChangeRequests: GitHub CLI command failed.
        GitHubCli.execute FAIL
          VcsProcess.run FAIL VcsProcessTimeoutError: VCS process timed out in GitHubCli.execute: gh (<repo>) after 30000ms

ws.rpc.vcs.refreshStatus 40256ms Success
  +36.36s remoteStatus 3.89s   (nothing traced before this; consistent with waiting on withRemoteWriteLock)

Workaround

None in the app. Fixing the network recovers it. SSH connection reuse (ControlMaster) speeds up the background fetch but doesn't help the gh lookup.

Activity

  1. juliusmarminge commented on Oct 30, 2026

    @juliusmarminge
    Member

    Closing: current main already covers this as of e677e9799d. If you still see it on a build that includes that commit, reply and we'll reopen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions