You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat: report instrumentation coverage for agent runs #30
Users need to understand which tool and LLM calls were captured during a StepFork run and which external boundaries are outside its replay guarantees.
Scope
Explore an instrumentation coverage report based on information StepFork can reliably observe. Start with known instrumented calls and explicit capture metadata. Do not claim to detect every uninstrumented call automatically without a reliable mechanism.
Review the existing trace format (docs/trace-format.md), run receipts, replay behavior, and PRODUCT_ROADMAP.md before proposing a design. Document limitations and avoid breaking existing trace compatibility.
Acceptance criteria
The report has a clearly defined and documented meaning of coverage.
It distinguishes observed instrumentation from calls that cannot be verified.
An offline example demonstrates the report.
Tests cover the reported coverage data and its limitations.
Existing trace and receipt formats remain compatible unless a migration is explicitly justified.
No false guarantee is made that every external call has been captured.
Testing
Add deterministic tests and an offline example. Run the relevant tests and the full test suite.
This is a larger feature. Please open a discussion or comment with a proposed design before beginning substantial implementation.
Contributions are welcome. Please comment with your proposed design before opening a PR.
Activity