Repository navigation
feat(swe_agents): add opt-in unfinished_episodes="fail" - #4036
Open
linj-glitch wants to merge 3 commits into
Open
linj-glitch wants to merge 3 commits into
linj-glitch wants to merge 3 commits into
Conversation
yaoyu-33
added
area:agent
linj-glitch
marked this pull request as draft
October 5, 2026 06:10
linj-glitch
force-pushed
the
linj/swe-agents-unfinished-as-failures
branch
from
October 8, 2026 00:15
2b90900 to
3bc3e7d
Compare
Episodes that OpenHands stops at agent_max_turns (max_iteration) or that run out of context (context_window) are masked from the loss, and episodes its loop detector stops (stuck_in_loop) are graded on whatever patch is on disk. For evaluation that is reasonable, but under RL it means running out the budget never costs anything: a policy that wanders for the full turn budget simply drops out of the batch, and one that loops after writing a working patch still earns reward 1. In an internal GRPO run the share of held-out tasks that hit the turn cap grew from 19 to 24 of 100 within 11 steps, and loop-detector exits grew from 0 to 29% of tasks while a sizeable fraction of them still scored. `unfinished_episodes: fail` keeps all three kinds in the loss with reward 0. `resolved` still reports the measured test verdict, and the new `unfinished` metric marks these episodes in both modes. OOM kills and agent / eval wall-clock timeouts stay masked either way, since those depend on the cluster as much as on the policy. The default, `mask`, keeps today's behavior. Signed-off-by: Lin Jia <linj@nvidia.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
linj-glitch
force-pushed
the
linj/swe-agents-unfinished-as-failures
branch
from
October 8, 2026 00:20
3bc3e7d to
4520359
Compare
Signed-off-by: Lin Jia <linj@nvidia.com>
linj-glitch
marked this pull request as ready for review
October 13, 2026 23:08
github-actions
Bot
added
the
sla:triage-overdue
This was referenced Oct 22, 2026
Collaborator
Author
|
/ok to test 09daaa1 |
Signed-off-by: Lin Jia <linj@nvidia.com>
Collaborator
Author
|
/ok to test 6cdddb4 |
7 tasks done
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Add
unfinished_episodes: "fail"for training. Turn-cap, context-window, and loop-detector exits remain in the loss with reward 0. The default"mask"preserves existing behavior: turn-cap/context exits are masked and loop exits receive their measured patch reward. The newunfinishedmetric records either mode, whileresolvedretains the measured test verdict. Independent infrastructure masks remain unchanged.No separate issue is needed for this focused training-policy option. Benchmark configurations should retain the default; opting in can lower reward-derived scores for unfinished episodes whose patch passes tests.
Validation
At
6cdddb4f5714170eaad98bf52f42e563a3d71ca8, including mainf36e126547866601bee9c0671c9b59484ef83723, Python 3.13.14 imports were verified against the isolated PR checkout:responses_api_agents/swe_agents/tests: 230 passed. Cases cover both modes, all three unfinished outcomes, measured verdict retention, legacy metrics, final reward, and independent infrastructure masks.pre-commit run --all-files: passed.Rollouts
The author previously ran the component policies in real OpenHands GRPO training. This is historical author-reported evidence; no new live model/agent/verifier rollout is claimed for this head. The default behavior is unchanged.
Checklist
Required current-head CI and maintainer approval remain merge gates.