Skip to content

fix: raise RuntimeError on second ctx.stream() consumption - #62

Draft
shloimy-wiesel with Copilot wants to merge 20 commits into
betafrom
copilot/bug-fix-ctx-stream-hang
Draft

shloimy-wiesel with Copilot wants to merge 20 commits into
betafrom
copilot/bug-fix-ctx-stream-hang

Conversation

Copilot AI commented May 30, 2026 •

Copy link
Copy Markdown
Contributor

Calling ctx.stream() more than once silently deadlocks — the [DONE] sentinel is consumed on the first iteration, leaving the queue permanently empty and the second consumer blocked forever.

Changes

  • StreamContext.__init__ — adds _streamed: bool = False flag
  • StreamContext.stream() — raises RuntimeError immediately on re-entry before touching the queue; sets _streamed = True on first call
# Before: hangs forever
async for _ in ctx.stream(): pass  # drains OK
async for _ in ctx.stream(): pass  # blocks indefinitely

# After: fails fast
async for _ in ctx.stream(): pass  # drains OK
async for _ in ctx.stream(): pass  # RuntimeError: ctx.stream() can only be consumed once
  • TestEdgeCases — two new tests: one asserting RuntimeError is raised on the second call, one confirming the error surfaces on __anext__() before any queue wait

semantic-release and others added 19 commits March 22, 2026 09:39
Automatically generated by python-semantic-release
* "Claude PR Assistant workflow"

* "Claude Code Review workflow"
…penAI adapter, and request types

Add the full StreamContext feature surface for AI SDK v6 compatibility:

Stream collection & persistence:
- collect=True accumulates text, reasoning, tool calls, sources, files,
  data parts, token counts, and timing into StreamRecord (ctx.record)
- on_finish callback for post-stream DB persistence
- Per-call collect=False to skip recording ephemeral writes
- RuntimeError when collect=True on non-collecting context

StreamContext enhancements:
- Generic[_InfoT] typed custom_information (ctx.info) for request-scoped metadata
- ctx.run() safe task runner: auto-finish, auto-error, GC-safe
- ctx.error() emits ErrorEvent then terminates stream
- ctx.abort() emits AbortEvent per v6 spec (with optional reason)
- ctx.write_file() for FileEvent support
- ctx.write_data() for dynamic data-{name} parts (any JSON-serializable value)
- start_tool_input/stream_tool_input_delta/finish_tool_input for streaming tool input
- start_metadata parameter for start event messageMetadata
- Auto-count streaming tokens with configurable count_func + set_usage() override
- Timing fields (created_at, finished_at, duration_ms) on StreamRecord

New event types:
- ErrorEvent, AbortEvent, FileEvent, DataEvent (dynamic type)
- ToolInputStartEvent, ToolInputDeltaEvent, ToolInputAvailableEvent

OpenAI contrib module (contrib/openai):
- consume_openai_stream() maps OpenAI chunks to ctx.write_* calls
- convert_to_openai_messages() converts UIMessage parts to ChatCompletionMessageParam
- Duck-typed — no hard openai package dependency

Request body types (types.py):
- ChatRequest, UIMessage, and all v6 part models (TextUIPart, ToolUIPart, etc.)
- Discriminated union for MessagePart with dynamic prefix support
- extra="allow" for forward compatibility

Bug fixes:
- Tool name deduplication in consume_openai_stream
- Tool call ID consistency using ToolCallHandle.toolCallId
- write_data() accepts any JSON-serializable value, not just dict
- DataEvent.encode() uses model_dump_json() for non-native types

Also adds chatbot-backend example (FastAPI + Redis + 5 tools),
hierarchical AGENTS.md knowledge base, and 237 passing tests.
Automatically generated by python-semantic-release
Automatically generated by python-semantic-release
Automatically generated by python-semantic-release
Automatically generated by python-semantic-release
feat: add tool_calls_in_reasoning option to StreamContext

When enabled, begin_tool_call and start_tool_input skip closing the
reasoning block, allowing tool calls to be emitted inline within
reasoning for frontends that support interleaved rendering.

Agent-Logs-Url: https://github.com/shloimy-wiesel/ai-sdk-stream-python/sessions/5576925b-99a1-4b5e-8f28-bd20be5b267b

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: shloimy-wiesel <144027408+shloimy-wiesel@users.noreply.github.com>
Automatically generated by python-semantic-release
…les (#61)

* docs: replace asyncio.create_task pattern with ctx.run() in all examples

Update the three locations that still showed the old manual
asyncio.create_task + try/finally pattern to use the recommended
ctx.run() approach: the __init__ module docstring quickstart,
the context.py module docstring FastAPI example, and the live
example/backend/routes/chat.py endpoint.

The example backend also drops the _background_tasks GC-protection set
and the asyncio import, which ctx.run() makes unnecessary.

Co-authored-by: shloimy-wiesel <shloimy-wiesel@users.noreply.github.com>

* fix: replace lambda with named _work function and update StreamContext docstring

- example/backend/routes/chat.py: replace `lambda c: ...` with a named
  `async def _work(c: StreamContext)` consistent with the docstring examples
- context.py: update StreamContext class docstring to mention `ctx.run()`
  instead of the old "spin up a background task" phrasing

Co-authored-by: shloimy-wiesel <shloimy-wiesel@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: shloimy-wiesel <shloimy-wiesel@users.noreply.github.com>
@vercel

vercel Bot commented May 30, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
ai-sdk-stream-python Ready Ready Preview, Comment, Open in v0 May 30, 2026 7:47pm

Track a `_streamed` flag on StreamContext. The `stream()` async
generator raises `RuntimeError("ctx.stream() can only be consumed
once")` on its second call, preventing silent deadlocks from retry
middleware or accidental re-iteration.

Add two tests in TestEdgeCases:
- test_stream_raises_on_second_consumption
- test_stream_raises_before_iteration_on_second_call
Copilot AI changed the title [WIP] Fix ctx.stream() hanging on second consumption fix: raise RuntimeError on second ctx.stream() consumption May 30, 2026
Copilot AI requested a review from shloimy-wiesel May 30, 2026 19:47
@shloimy-wiesel

Copy link
Copy Markdown
Owner

@claude review this pr

@claude

claude Bot commented May 30, 2026 •

Copy link
Copy Markdown

Claude finished @shloimy-wiesel's task in 1m 58s —— View job


PR Review: fix: raise RuntimeError on second ctx.stream() consumption

  • Gather context and understand the request
  • Read changed files (context.py, tests/test_context.py)
  • Run diff and analyze changes
  • Post review feedback

Summary

This PR fixes a real bug: double-consuming ctx.stream() previously deadlocked silently because the None sentinel was consumed on the first pass, leaving the second consumer blocked on an empty queue forever. The fix is correct and well-targeted.

What's Good

Implementation (context.py:772–777): The _streamed flag + early RuntimeError is the right approach. Since asyncio is single-threaded, there's no TOCTOU race between the check and the flag set — no await separates them, so no other coroutine can interleave.

Tests (test_context.py:611–639): Two tests, each covering a distinct behavior:

  1. test_stream_raises_on_second_consumption — verifies the error is raised when iterating a second time
  2. test_stream_raises_before_iteration_on_second_call — verifies the error surfaces on __anext__(), not when the queue would block

This distinction matters: because stream() is an async def with yield, calling ctx.stream() returns the generator object without executing any code. The check only runs on the first __anext__(). The second test explicitly documents this Python async generator behavior, which is the right call.

Version bump: pyproject.toml → 0.4.0, which is reflected in uv.lock. Appropriate for a behavioral change.


One thing to flag

The collect_stream helper (used in both new tests) breaks on [DONE] without calling aclose() on the generator:

# tests/test_context.py:29–38
async def collect_stream(ctx: StreamContext) -> list[dict]:
    async for chunk in ctx.stream():
        if chunk.strip() == "data: [DONE]":
            break   # generator abandoned here, aclose() deferred to GC
        ...

This is pre-existing behavior, not introduced by this PR, and it doesn't cause any correctness issues here — _streamed = True is already set on the first __anext__() so the second call correctly raises. But if stream() ever grows a try/finally cleanup block, this pattern would defer that cleanup non-deterministically.

Not a blocker for this PR, but worth a follow-up async with contextlib.aclosing(ctx.stream()) as gen: refactor in the test helper.


Verdict

Approve. The fix is minimal, correct, and well-tested. No issues blocking merge.

@shloimy-wiesel
shloimy-wiesel changed the base branch from main to beta May 30, 2026 20:43

This branch was successfully deployed

1 active deployment
Preview — 7d370de3 Deployed May 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: ctx.stream() hangs on second consumption

2 participants