Skip to content

Harness reliability: run termination, context-safety margins, compaction fidelity #1170

Description

@anandgupta42

Long-running non-interactive sessions exhibit several reliability failure modes:

  1. No termination path — runs that finish their task keep consuming turns until wall-clock timeout; a majority of long-run wall time is burned after the work is done, and clean-exit rates sit near 15%.
  2. Context-window overflow at the provider — the chars/4 token estimate undercounts dense SQL/JSON by up to ~1.5x, so requests sized near the declared window are rejected by the provider (400), and a single oversized tool result can jump a small conversation past the hard limit in one step.
  3. Task drift under repeated compaction — iterated summarization loses the original task statement, producing re-grounding loops and zero-write sessions.
  4. Embedder gaps — external drivers invoking run don't get run-mode semantics without env plumbing; explicit opt-outs were not consistently honored.

Tracking issue for the harness reliability changeset addressing these (run termination protocol, compaction fidelity + context safety margin, per-tool-result dispatch cap, run-mode defaults, accounting/attribution).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions