Skip to content

fix(ci): per-SHA reset and dispatch race fix in CI poller - #59

Merged
chihsuan merged 1 commit into
mainfrom
worktree-ci-poller-retry-fix
May 21, 2026
Merged

fix(ci): per-SHA reset and dispatch race fix in CI poller#59
chihsuan merged 1 commit into
mainfrom
worktree-ci-poller-retry-fix

Conversation

@chihsuan

Copy link
Copy Markdown
Member

Context

CI escalation fired on the same poll that dispatched the agent, killing a 43-second run before it could fix anything (RSM-3739, PR #57). The lifetime ci_retry_count also stuck issues in escalated across new commits.

TL;DR

Reset CI dispatch/rerun history per head SHA, protect in-flight dispatches from being escalated, and stop poll_error from clobbering the state-machine status.

Summary

  • New head SHA observed → clear dispatched_shas, rerun_attempted_shas, downgrade escalatedwatching. Lifetime ci_retry_count is preserved; only green resets it.
  • Reorder handle_ci_failure so a SHA already in dispatched_shas short-circuits to :already_handled before the escalation branch, preventing the next poll from killing the in-flight agent.
  • poll_error_attrs no longer writes status: "poll_error" — only :error, :consecutive_errors, :next_poll_at. Statuses like escalated/rerun_requested survive a transient GitHub error.
  • Add tests for new-SHA reset, dispatched-SHA race protection, and status preservation across poll errors.
  • Update docs/configuration.md to describe the new retry semantics.

Alternatives

  • Reset lifetime ci_retry_count on every new SHA — rejected: escalation would never fire when an agent rapidly pushes new broken commits.
  • Escalate atomically when retry_count + 1 == max_retries (skip the last dispatch instead of killing it) — rejected: drops one dispatch attempt and is harder to reason about than "protect the SHA you just dispatched."
  • Track full per-SHA history (counts, results) — rejected as bigger than needed; the SHA list already dedupes dispatches.

Test Plan

  • make all
  • mix test test/symphony_elixir/ci_poller_test.exs (18 tests passing locally)

@chihsuan
chihsuan merged commit 137dcce into main May 21, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant