What happened. On one fleet, the controller's disk filled for about two hours. Kernel-dispatched baseline evals that ran in that window finished with Slurm state COMPLETED but wrote no result file (the write failed with "No space left on device"). Two runs then ended with ending: negative-result and the note:
measure baseline: job <id> ended COMPLETED without a result
Neither run had a research outcome: the agent never saw a comparison, and its candidate was never measured. Their results had to be excluded by hand.
Expected. When an eval finishes without producing a result, the kernel can't tell whether the tree under test failed or the infrastructure did. A baseline eval with no result is never the agent's fault. It should be treated as an infrastructure failure: re-dispatch it (uncharged, bounded by a retry budget) or park the run with a clear operator-visible note. It should not end the run as a negative result. A candidate eval with no result is arguably the same unless its stderr shows the candidate's code failing.
Where. measure.py _no_result_note builds the note; the caller ends the run as negative-result.
Repro. Make the eval job's output directory unwritable (or full) so the job exits 0 without writing result.json; wake the run.
Reported by Claude Opus 5.5 from a live fleet.
What happened. On one fleet, the controller's disk filled for about two hours. Kernel-dispatched baseline evals that ran in that window finished with Slurm state COMPLETED but wrote no result file (the write failed with "No space left on device"). Two runs then ended with
ending: negative-resultand the note:Neither run had a research outcome: the agent never saw a comparison, and its candidate was never measured. Their results had to be excluded by hand.
Expected. When an eval finishes without producing a result, the kernel can't tell whether the tree under test failed or the infrastructure did. A baseline eval with no result is never the agent's fault. It should be treated as an infrastructure failure: re-dispatch it (uncharged, bounded by a retry budget) or park the run with a clear operator-visible note. It should not end the run as a negative result. A candidate eval with no result is arguably the same unless its stderr shows the candidate's code failing.
Where.
measure.py_no_result_notebuilds the note; the caller ends the run as negative-result.Repro. Make the eval job's output directory unwritable (or full) so the job exits 0 without writing
result.json; wake the run.Reported by Claude Opus 5.5 from a live fleet.