Repository navigation
Description
Summary
At e1d4cc9 (v0.2.1), none of the providers let the caller set a request timeout, so every request uses the SDK default: up to 10 minutes for OpenAI and Anthropic, and no timeout at all for Gemini (its client defaults to httpx.Timeout(None)).
This means RetryPolicy(api_timeout_error=True), which the adapter accepts as a drop-in for TypeSafeClient, can't realistically trigger, and a stalled request hangs the caller instead of failing fast so it can fall back.
Reproduction
A local server that accepts connections but never replies, with RetryPolicy(max_retries=1):
import socket, threading, time
from typesafe_sdk import RetryPolicy
from system_one_adapter import Noul, SystemOneAdapterClient
from system_one_adapter.providers.openai import OpenAIProvider
server = socket.socket()
server.bind(("127.0.0.1", 0))
server.listen()
held = []
threading.Thread(target=lambda: [held.append(server.accept()) for _ in iter(int, 1)], daemon=True).start()
provider = OpenAIProvider("gpt-4o-mini", base_url=f"http://127.0.0.1:{server.getsockname()[1]}/v1", api_key="test")
client = SystemOneAdapterClient(structured_outputs=True, llm_answer_mode="discrete", retry=RetryPolicy(max_retries=1))
start = time.monotonic()
try:
client.system_one(state="hi", questions={"q": Noul(instructions="Is it a greeting?")}, model=provider)
except Exception as error:
print(f"{type(error).__name__} after {time.monotonic() - start:.2f}s")- Expected: a way to bound each request, failing with
TypeSafeAPITimeoutError. - Actual on
main: no output; I killed it after 30 s. Each attempt waits up to 10 minutes. - With the proposed fix (
OpenAIProvider(..., timeout=0.5)):TypeSafeAPITimeoutError after 1.5s(two attempts plus backoff).
Proposed fix
An optional timeout= (seconds) on all six providers, following the existing max_tokens= option on AnthropicProvider:
client.system_one(state, questions, model=OpenAIProvider("gpt-4o-mini", timeout=2.0))- Omitted, it keeps today's behavior.
Noneis never passed through, since the SDKs read it as "never time out". - Zero or negative values raise
ValueError, likemax_tokens. - Gemini receives it as
sdk_configuration.timeout_ms. - README section (including that the timeout is per request, not per
system_onecall) and anUnreleasedchangelog entry.
The fix is one commit on my fork:
- Branch: https://github.com/jessicafalcon/system-one-adapter-python/tree/feat/provider-timeout
- Diff: main...jessicafalcon:system-one-adapter-python:feat/provider-timeout
- Apply:
curl -L https://github.com/jessicafalcon/system-one-adapter-python/commit/5dbe2075b54a24d3405df2d435b9ad9e9032b58b.patch | git am
Happy to adjust the API (for example, also accepting an httpx.Timeout) or for a maintainer to take it from there.
Test plan
uv run pytest tests/test_provider_timeouts.py: new tests cover all six providers. The ones checking that the timeout reaches the SDK fail onmainand pass with the fix.uv run pytest: 445 passed.uv run pyrefly check: 0 errors.- The reproduction above, run on
mainand on the branch.
Activity
aoright commentedon Sep 27, 2026
I ran an independent verification of @jessicafalcon's patch (commit 5dbe207) on macOS with Python 3.14.3 and confirmed the timeout behavior end-to-end.
Verification Summary
Unit Tests:
uv run pytest tests/test_provider_timeouts.py: 21/21 passed (parameter passing across all providers, millisecond conversion for Gemini, rejection of non-positive timeouts).uv run pytest(453 passed),uv run pyrefly check(0 errors).End-to-End RetryPolicy Behavior:
OpenAIProvider("gpt-4o-mini", timeout=0.2, ...)withRetryPolicy(max_retries=1, api_timeout_error=True)against an in-memory socket server that accepts connections without responding.TypeSafeAPITimeoutErrorat total elapsed time ~1.46s. Both sync (SystemOneAdapterClient) and async (AsyncSystemOneAdapterClient) behave consistently.In-Suite Integration Testing & pytest-recording:
pytest,pytest-recordingintercepts socket creation becausepyproject.tomlconfiguresaddopts = "--block-network -p no:tach".socket.connect()raisesRuntimeError: Network is disabledfrompytest_recording/network.py:124."allowed_hosts": ["127.0.0.1"], to thevcr_configfixture intests/conftest.py(as discussed in--block-networkmakes the whole test suite unrunnable on Windows #44) allows local loopback tests to run cleanly underpytestwhile keeping external provider calls strictly blocked.Integration Branch
I added end-to-end timeout integration tests covering sync and async client behavior on branch:
https://github.com/aoright/system-one-adapter-python/tree/test/verify-provider-timeout (commit 5b2a5bd)
With
"allowed_hosts": ["127.0.0.1"], all 455 tests pass cleanly.