Skip to content

Show agent sign-in notices before the first chat message - #37191

Merged
brennanb2025 merged 8 commits into
mainfrom
brennanb2025/claude-signin-notice-1101
Nov 2, 2026
Merged

brennanb2025 merged 8 commits into
mainfrom
brennanb2025/claude-signin-notice-1101

Conversation

@brennanb2025

@brennanb2025 brennanb2025 commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 7 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​455 $\color{#cf222e}{\Huge{\mathbf{−}}}$​1 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​454
Prod 6 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​145 $\color{#cf222e}{\Huge{\mathbf{−}}}$​77 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​68

Problem. A signed-out Claude user opening a new chat saw a normal model list and no sign-in notice, so they only found out they weren't signed in when their first message failed. Codex already warned before Send because its model check reads the account; Claude's model check never asked whether the CLI was signed in, and Pi's explicit "no provider" state was treated as a generic listing failure.

What changes for users. When Claude or Pi itself reports that it is signed out, the chat now shows the existing sign-in notice before Send (for Claude: "Claude isn't signed in. Run claude auth login, or choose an account in Claude Accounts settings."). Models stay listed, Send stays available, and the real start result is still what decides whether a message goes through; this check never holds or refuses a message. An agent-reported "authentication required" error during model discovery now shows the same per-agent sign-in notice instead of a generic failure.

How it works. Claude's existing model-discovery process now also asks the CLI for its startup account information, in the same process as the model and settings requests. It reports "not signed in" only when the CLI says there is no token, no API key, no account, a first-party provider, and no custom endpoint (including one set in Claude's settings files). API keys, bearer tokens, key helpers, Bedrock/Vertex/Foundry and custom endpoints stay quiet. If the account reply is slow, the models already received are kept and the process is closed after a short grace period. Pi is treated as signed out only when it reports its "unknown" provider and has no available models. Results reuse the existing per-account, per-folder, per-host catalog cache, so no extra background polling is added.

Intended differences.

  • Unknown, slow or failed account checks stay quiet and Send remains available (lane product ruling: a sign-in check never refuses or holds a send; the start result is the authority).
  • Grok: no pre-Send sign-in inference. Grok's own status command can report "not authenticated" for a signed-in user whose token is mid-refresh, and its startup auth list omits deployment keys, so neither is a reliable signed-out fact. Grok behaves as on main until Send.
  • While a signed-out notice shows, the composer doesn't preselect a default model (it shows "Model"), the same as signed-out Codex already does; all agents behave the same here.
  • Mobile behavior is unchanged; this uses the existing desktop notice. The separate startup auth classifier, invalid-command advisories and account-retirement behavior are outside this change.

Verification. Recorded real CLI output tests (Claude 2.1.310, Pi), a repeated-read contract with 10 and 100 chats on one signed-out account proving at most one re-check per 30 s, and CI. A real-app run on a test machine with real CLIs: signed-out Claude showed the notice before Send in a git project and a folder workspace and failed at Send as before; signed-in Claude and Codex each completed two real replies with no notice; the listing process exited within about 2 seconds; main showed no notice before Send.

Tracked in STA-13193.

@brennanb2025
brennanb2025 marked this pull request as ready for review November 1, 2026 22:09
@coderabbitai

coderabbitai Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 338e8dbc-57e1-4382-8c0b-3e44041b34fb
Commits

Reviewing files that changed from the base of the PR and between d921575 and 39bc807.

Files selected for processing: 1
  • src/main/native-chat/agent-model-catalog/agent-model-catalog-scaling.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 2 remain after this review.

Walkthrough

The ACP, Claude, and Pi catalog probes now report selected authentication-related conditions as notSignedIn unavailable results. The Claude probe also reads request-ID-keyed control responses, identifies qualifying signed-out account configurations, and can return ready partial listing output.

Merge Risk: Low · up to 39bc8

The Pi catalog may show a signed-out notice before provider discovery completes; this warrants bounded follow-up.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main user-facing change: showing agent sign-in notices before the first chat message.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description clearly explains the problem, user impact, implementation, intended differences, scope boundaries, linked issue, and verification. It does not reproduce the template headings or checkl…
Full details: Description check

Explanation

The description clearly explains the problem, user impact, implementation, intended differences, scope boundaries, linked issue, and verification. It does not reproduce the template headings or checklist items, and it omits explicit visual-proof and risk sections, but the required information is mostly present.

Full details: Docstring Coverage

Explanation

Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 5 functions across 11 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
Generate docstrings
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

brennanb2025 and others added 3 commits November 1, 2026 15:02
…plies arrive late

Grok goes back to one listing process with no Grok-specific sign-out guess;
an agent-authored auth-required reply still reports notSignedIn. Pi reports
notSignedIn only when it has no provider and nothing listable. Once Claude's
required replies are in, the listing deadline returns them instead of failing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


Review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a472fcce-5897-4121-a488-f9f432c96cf6
Commits

Reviewing files that changed from the base of the PR and between 70dae97 and 8d8edab.

Files selected for processing: 13
  • src/main/acp/__fixtures__/session-new-auth-required.json
  • src/main/acp/acp-model-catalog-probe.test.ts
  • src/main/acp/acp-model-catalog-probe.ts
  • src/main/claude/__fixtures__/catalog-signin.json
  • src/main/claude/claude-command-catalog-probe.ts
  • src/main/claude/claude-control-responses.ts
  • src/main/claude/claude-model-catalog-probe.test.ts
  • src/main/claude/claude-model-catalog-probe.ts
  • src/main/native-chat/agent-model-catalog/agent-model-catalog-probe-runner.test.ts
  • src/main/native-chat/agent-model-catalog/agent-model-catalog-probe-runner.ts
  • src/main/native-chat/agent-model-catalog/agent-model-catalog-scaling.test.ts
  • src/main/pi/rpc-model-catalog-probe.test.ts
  • src/main/pi/rpc-model-catalog-probe.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

Comment thread src/main/pi/rpc-model-catalog-probe.ts Resolved
brennanb2025 and others added 4 commits November 1, 2026 15:20
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
N and 10N distinct chats (distinct read claims, one account-scoped
Claude fingerprint) read with waitForListing against the real probe and
store on an injected clock: one listing before the held-reason TTL,
exactly one more once it is due (held until every read has decided),
none within the next TTL, each result carrying notSignedIn and models.

Checked locally that it bites: making heldReasonDue ignore the TTL fails
it (2 listings inside the TTL). Removing the in-flight dedup (the
probeDue isListing guard plus the lister's early return) does not: store
refresh then runs once per chat, but a newer flight replaces each queued
one behind the two admission slots, so still one process per TTL.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The shared reader required type 'control_response' and prefiltered on
that text, dropping success frames main's model and slash-command
parsers accepted (the released-phone wire test's fake listing).
It now prefilters on request_id and keeps the structure limit and
success-only reading.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@brennanb2025

Copy link
Copy Markdown
Contributor Author

Review loop status

Problem. A signed-out Claude user opening a new chat saw models and no sign-in notice; they only learned they weren't signed in when the first message failed. Codex already warned before Send because its model check reads the account; Claude's never asked, and Pi's explicit "no provider" state was treated as a generic listing failure.

What changes for users. When Claude or Pi itself reports it is signed out, the existing sign-in notice now shows before Send. Models stay listed, Send stays available, and the actual start result still decides. API keys, bearer tokens, key helpers, Bedrock/Vertex/Foundry and custom endpoints (including settings-file endpoints) stay quiet. Grok is unchanged from main: its status command can say "not authenticated" for a signed-in user mid token refresh, so it isn't used.

Fixed during review. Removed a Grok status check that could show a false notice and slowed every Grok listing; Pi now reports signed out only when it also has no models; models already received survive the listing deadline; the shared Claude reply reader accepts exactly what main accepted (a released-phone compatibility test caught the stricter version); the repeated-read contract now drives the real trigger (10 and 100 chats on one signed-out account: one re-check per 30 s); test fidelity and CI typing fixes.

Deferred / not in this PR.

  • While a signed-out notice shows, the composer doesn't preselect a default model ("Model" instead of "Opus Med"). This is what signed-out Codex already does on main; the references differ from each other here, so it's kept consistent across agents.
  • Existing on main, made easier to reach: after signing in from an Orca terminal, the notice stays until the next window focus, turn or chat start (same as Codex today).
  • Existing on main, outside this mechanism: Claude's startup auth classifier is narrower than this advisory check, so a keyless custom endpoint could be called signed out at start (source trace only, no working repro); resolver errors for a missing/invalid command and notice clearing while an account binding retires remain separate issues.
  • Concern without a failing input: the shared reader caps reply size/nesting (64K tokens, depth 16); recorded settings replies are far below it.

Verified. CI green on 39bc807. Targeted real-app run on a test Mac with real CLIs (QA head 39bc807, main a7d6926): signed-out Claude shows the notice before Send in a git project and a folder workspace, Send stays enabled and fails at start as before; signed-out Codex unchanged; signed-in Claude and Codex each got two real replies with no notice; the listing process exits within about 2 s with nothing left running; main shows no notice before Send. Real config hashes unchanged. Pi and the ACP auth-required mapping are covered by tests fed recorded real CLI output (Pi also reproduced against the real CLI); no Pi run in the app.

For Brennan's review

Signed-out Claude, branch Signed-out Claude, main
f1-branch-claude-signedout-notice.png f1-main-claude-signedout.png

Send while signed out (branch): f1-branch-claude-send-result.png
Signed-in Claude, two replies: f2-claude-two-replies.png
Codex, two replies: f3-codex-two-replies.png

Where judgment matters:

  1. Claude signed-out predicate: which account facts count as signed out and which endpoints are exempt.
  2. Runner grace and deadline: keeping received models when the optional account reply stalls.
  3. Pi sentinel: "unknown" provider plus no models as the sign-out fact.

@brennanb2025
brennanb2025 merged commit 1783cf5 into main Nov 2, 2026
51 checks passed
@brennanb2025
brennanb2025 deleted the brennanb2025/claude-signin-notice-1101 branch November 2, 2026 05:47
nwparker pushed a commit that referenced this pull request Nov 2, 2026
#37191 added an import of the old path after #37289 moved the module, breaking
main's typecheck for every PR.
OrcaWin added a commit that referenced this pull request Nov 8, 2026
… sign-in probe and onboarding trust prompt

- setup-script-import: imported setups are saved from the Review setup
  script dialog since #37937.
- github-url-smart-input-transition: the GitLab source lists through
  gitlab:listWorkItems since #32005, so stub that channel.
- structured-chat-tabs-open-close: the Claude catalog probe also sends an
  initialize request for sign-in discovery since #37191.
- golden-core-flows: onboarding launches the chosen agent in the fresh ~
  workspace (#37898), whose first launch asks the per-repo agent folder
  trust question (#38946); answer it before adding a project.
OrcaWin added a commit that referenced this pull request Nov 8, 2026
… sign-in probe and onboarding trust prompt (#39983)

- setup-script-import: imported setups are saved from the Review setup
  script dialog since #37937.
- github-url-smart-input-transition: the GitLab source lists through
  gitlab:listWorkItems since #32005, so stub that channel.
- structured-chat-tabs-open-close: the Claude catalog probe also sends an
  initialize request for sign-in discovery since #37191.
- golden-core-flows: onboarding launches the chosen agent in the fresh ~
  workspace (#37898), whose first launch asks the per-repo agent folder
  trust question (#38946); answer it before adding a project.

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant