fix(eval): resolve awaitable data before iteration

[Gareth.Y (linjiaye929)]

Summary

EvalData explicitly accepts an awaitable iterator or a factory returning one, but the evaluation runner iterates the coroutine without awaiting it. Passing an async data loader to either Eval() or EvalAsync() therefore raises TypeError: 'coroutine' object is not iterable before evaluating any rows.

• Await the resolved data source once, after invoking the factory and before iterator normalization.
• Cover direct awaitables, true async-function factories, sync factories returning awaitables, ordinary sync iterators/factories, and existing async-generator behavior through both public entry points.
• Verify the loader runs once and its original exception propagates.

The implementation change is three lines; public signatures are unchanged.

Reproduction

import asyncio
from braintrust import EvalAsync

async def rows():
    return iter([{"input": "hello"}])

asyncio.run(EvalAsync(
    "local-data",
    data=rows,
    task=lambda input: input,
    scores=[],
    no_send_logs=True,
    enable_cache=False,
))

Validation

Python 3.12.14, local editable SDK, test dependencies from the frozen test dependency group:

• Baseline regression matrix: 8 failed, 6 passed.
• Fixed regression matrix: 14 passed.
• test_framework.py, test_parameters.py, and test_score.py: 158 passed, 12 skipped. Eleven skips require optional Pydantic; one requires Python 3.14 lazy annotations.
• Tests ran with CI=1, cassette replay, the Braintrust pytest plugin disabled, and an external guard rejecting live socket connections.
• Ruff 0.15.21 format/check and git diff --check passed.

No live service/provider calls were used. The full core suite, full version matrix, type-checker sessions, and make fixup were not run.

AI assistance

Prepared with AI assistance; the public-API reproduction, baseline failures, fix, and targeted tests were independently reviewed.

CI status

As checked on 2026-10-13 at 04:16:21 UTC for head fe71940ef4137eea624a3a61f91d4fea2e9e2794, both upstream workflow runs report action_required:

• checks
• integration-tests

CI has not passed. The local validation above does not represent a successful upstream CI run.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-13T04:22:56.640626Z	fe71940	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "Codex (@codex) review" or "Codex (@codex) security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.