Skip to content

fix: stop the test suite from calling live limits endpoints with the user's login - #355

Merged
chattymin merged 1 commit into
chattymin:mainfrom
cte13:fix/tests-no-live-limit-calls
Sep 25, 2026
Merged

chattymin merged 1 commit into
chattymin:mainfrom
cte13:fix/tests-no-live-limit-calls

Conversation

@cte13

@cte13 cte13 commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

Summary

swift test can call Anthropic, Google, and Cursor with the contributor's real login.

UsageStore.init defaults every limits provider to the real one, so any test that doesn't inject a stub runs it. The automatic path reads no Keychain item, which makes this look harmless, but credentials are also plain files:

  • ~/.claude/.credentials.json
  • the Antigravity token file
  • Cursor's login in state.vscdb

On a machine that has any of them, the no-prompt path succeeds. The suite then makes live requests with that account. This uses up the account's rate limit, so the real app running alongside hits 429 and backs off. For Antigravity, it also spends the user's refresh token. On my machine, the new Claude and Antigravity checks below reached the gate instead of skipping. So both credentials are reachable from swift test, and before this change those tests could make real calls.

This adds one gate, AppEnv.allowsLiveLimitsFetch, and checks it at every limits network boundary:

Provider Boundary
OAuthLimitsProvider usage call, and the profile identity call
AntigravityRateLimitsProvider quota call, and the Google token refresh
URLSessionSessionKeyClient the real claude.ai client (tests inject their own)
CursorRateLimitsProvider perform, used by both the usage call and the token refresh

The gate is open in the app bundle (AppEnv.isBundledApp, the same check that already guards notifications and Keychain reads), with PTB_PARITY=1 (already used by the parity smoke tests), or when a test explicitly opts in to drive real network code against a stubbed transport. It sits at the network boundary, not before the credential read, so KeychainAutoPathTests still sees real reads.

Trade-offs

  • Raw dev binaries: running .build/debug/PokeTokenBar directly now shows no live limits unless you run it with PTB_PARITY=1. Bundles built by scripts/build-app.sh are unaffected. This matches how the app already disables other side effects outside a bundle.
  • Stubbed-network tests opt in: tests that exercise a provider's real network code against URLProtocol or an injected session set AppEnv.allowLiveFetchForTesting and reset it in defer. Only the 3 such tests in CredentialSwitchCacheTests need it.
  • The source scan is a heuristic: it checks every *LimitsProvider.swift that uses URLSession for the gate, ignoring commented-out lines. It exists so the next provider can't ship without the gate. I'm happy to drop it if you'd rather not have a source-scanning test.

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Documentation
  • Other:

Checklist

  • swift build and swift test pass locally (1273 tests, 0 failures)
  • PR title and description are written in English
  • No copyrighted assets, secrets, or private tooling references are committed (see CONTRIBUTING)
  • Tests were added or updated for this change: LiveCredentialCallGateTests
    • testGateIsClosedUnderSwiftTest runs on every machine, including CI.
    • The Claude and Antigravity fetches must stop at the gate. They skip when the machine has no credential, rather than counting as a pass.
    • testEveryNetworkingLimitsProviderIsGated is the source scan.
    • Fault injection: commenting out the Cursor gate fails the scan, and forcing the gate open fails testGateIsClosedUnderSwiftTest. I did not test by removing the Claude or Antigravity gates, because that would make the very live calls this PR prevents.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PNenigaFnwvcTs2kDND16Z

…user's login

UsageStore defaults every limits provider to the real one, so any test that
does not inject a stub runs it. Credentials are also plain files
(~/.claude/.credentials.json, the Antigravity token file, Cursor's
state.vscdb), so on a contributor's machine the no-prompt path succeeds and
`swift test` calls Anthropic, Google and Cursor with that account: spending
its rate limit (which the real app then backs off from) and, for Antigravity,
its refresh token.

Gate every network boundary of the limits providers with
AppEnv.allowsLiveLimitsFetch (app bundle, or PTB_PARITY=1, or an explicit
test opt-in for stubbed transports):
- OAuthLimitsProvider: usage call and the profile identity call
- AntigravityRateLimitsProvider: quota call and the Google token refresh
- URLSessionSessionKeyClient: the real claude.ai client (tests inject theirs)
- CursorRateLimitsProvider: perform (usage and token refresh)

The gate sits at the network boundary, not before the credential read, so
KeychainAutoPathTests keeps observing real reads. A raw `swift build` binary
now shows no live limits unless run with PTB_PARITY=1.

LiveCredentialCallGateTests checks the gate is closed under `swift test`,
that Claude and Antigravity stop at it when a credential exists, and scans
every networking *LimitsProvider.swift so the next provider cannot ship
without it. CredentialSwitchCacheTests opts in for its three stubbed-transport
tests.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PNenigaFnwvcTs2kDND16Z
@chattymin
chattymin merged commit 671c1ec into chattymin:main Sep 25, 2026
2 checks passed
@chattymin

Copy link
Copy Markdown
Owner

Thank you for your contribution!

cte13 added a commit to cte13/PokeTokenBar-iOS that referenced this pull request Sep 25, 2026
chore: sync fork with upstream (15 PRs, incl. the fork's chattymin#353/chattymin#354/chattymin#355)
donky-ey pushed a commit to donky-ey/PokeDexBar that referenced this pull request Oct 15, 2026
…user's login (upstream chattymin#355)

UsageStore defaults the Claude limits provider to the real one, so any
test not injecting a stub ran it; with ~/.claude/.credentials.json
present the no-prompt path succeeds and swift test calls Anthropic with
the user's account. Gate the network boundary (not the credential read,
so Keychain-path tests keep observing real reads) behind
AppEnv.allowsLiveLimitsFetch: app bundle, PTB_PARITY=1, or an explicit
test opt-in for stubbed transports. A source scan test requires the
gate in every networking limits provider.
donky-ey pushed a commit to donky-ey/PokeDexBar that referenced this pull request Oct 15, 2026
…in#199)

Two Claude accounts on one Mac overwrite the single Keychain item, and
the limits section silently flips between them. Fetch the token owner's
identity from the oauth profile endpoint (same host and auth as the
usage call, no extra Keychain read), cache it per access token, and
render an 'Account email - org' caption under the plan row. Best
effort: if the profile call fails, limits render exactly as before. The
profile call sits behind the chattymin#355 live-fetch gate.
donky-ey pushed a commit to donky-ey/PokeDexBar that referenced this pull request Oct 15, 2026
…attymin#241)

Auto-polling never reads the Keychain by design, so once the cached
token expires and ~/.claude/.credentials.json is absent, official
limits freeze until the user presses the refresh button — which raises
the password dialog on unnotarised builds. This adds an opt-in second
credential path: paste a claude.ai sessionKey cookie under Settings >
Advanced and limits come from claude.ai/api/organizations/{org}/usage
(same schema as oauth/usage, LimitStatus reused). The path touches no
Keychain, so auto-polling keeps limits current. Without a key nothing
changes: ChainedLimitsProvider falls through to the OAuth path, and a
configured-but-dead key never lets the fallback open the Keychain.
The chattymin#355 gate covers the new network client (the provider scan test
caught the ungated client on the first full-suite run).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants