Skip to content

Benchmark Pi's progress commits on OPFS through the output gate #89

Description

@MiguelsPizza

Pi commits progress, not just outcomes:

  • partial model output into a live document at most every 100 ms with one commit in flight (generation.ts);
  • tool output at most every 100 ms and 100 KiB/s (output.ts).

Each commit is a transaction through the output gate. On do-runtime that means OPFS writes through the SAH-pool VFS with a rollback journal, about ten per second per active stream, more with parallel tools or subagents.

Add a benchmark next to conformance/bench/sql-latency that drives a PiHarness session through a long streamed answer and a chatty tool. Report:

  • commits per second;
  • p50 and p99 commit latency;
  • gate stall time seen by an unrelated RPC on the same actor.

Compare it with the same answer produced by Think. If the cost matters, the throttle belongs in Pi's settings or in the harness, and an issue goes upstream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions