Skip to content

W3-03c: crash/power-loss drill kit - #379

Merged
mergify[bot] merged 3 commits into
mainfrom
claude/w3-03c-drill-crash
Oct 4, 2026
Merged

mergify[bot] merged 3 commits into
mainfrom
claude/w3-03c-drill-crash

Conversation

@Cuarroc

@Cuarroc Cuarroc commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

Dieses Paket liefert den Absturz-Drill für deinen Windows-PC: Du beendest ProjectA hart an einem kritischen Übergang, startest es neu, und das Skript prüft, ob der Lauf auf „reconciling" steht und nichts neu gestartet wurde. Deine Datenbank wird nur über eine Kopie gelesen, Tokens werden geschwärzt. Die Schritte stehen auf Deutsch in docs/drills/crash-drill.md; ohne Continuous findet der Drill keinen offenen Lauf und zählt dann als „nicht abgedeckt".

Report

What changed

  • Added scripts/lib/crash-drill.mjs for state snapshots from a temporary database copy, restart judgement, and before/after evidence bundles.
  • Added Node and PowerShell launchers plus German operator instructions.
  • Added six Node tests for reconciliation, silent transitions, redispatch, delivery rollback, bundle handling, and the reviewed failed-run sibling case.
  • Review fix: a killed open run now passes only while it remains reconciling; terminal failed no longer counts as safe because it releases the active-run redispatch barrier.

Evidence

  • node --test scripts/lib/crash-drill.test.mjs before the review fix -> exit 1 (5 passed, the new failed-run regression test failed as expected).
  • node --test scripts/lib/crash-drill.test.mjs at 02b3e33 -> exit 0 (6 passed).
  • bash scripts/ci/red-first.sh at 02b3e33 -> exit 0 (7 exact test specifications red/missing at base and green at head).
  • bash scripts/ci/gates.sh lane prepush at 02b3e33 -> exit 0 (229 s; frontend 480/480, HQ 533/533, Rust 1769/1769 with 7 skipped).
  • PA_PREPUSH=light git push -u origin claude/w3-03c-drill-crash -> exit 0 (branchpush lane green).
  • git ls-remote origin refs/heads/claude/w3-03c-drill-crash -> exit 0, remote SHA 02b3e338731e9edeaecac3a9a240c89a153ea98e.
  • Diff: 216 lines (limit 300).

NICHT ABGEDECKT

  • Windows half (#[cfg(windows)], the .ps1 launcher is not run on Windows here); the Windows lane runs in the merge queue, and the drill itself is run by the user on the installed app.
  • Real power loss (kill is the proxy); recovery time series (single snapshot).
  • Transitions claimed / intent / checkpoint / completion exist only for development runs (Continuous ON, frozen until M4); not triggerable from the UI with Continuous OFF.
  • Schema assumptions (development_runs, development_launches, development_deliveries, development_capture_checkpoints) were checked against the store source, not an installed-app database; missing tables yield empty lists.
  • External runtime services (Updater endpoint, OmniRoute, mirror) are not checked by the local gates.

Architecture drift

No new finding. The change stays in the existing Node drill/bundle pattern and introduces no frontend state, Tauri result/error, Store/lock, IPC, or Rust-to-frontend event pattern.

Review disposition

  • Reviewer: Codex GPT-5.6 Sol (OpenAI), independent of the Claude author family.
  • Reviewed candidate: 4e808b074dd67ddc9fa0016f40969ee8eeed2a4c; fix head and delta rechecked at 02b3e338731e9edeaecac3a9a240c89a153ea98e.
  • R1, medium, scripts/lib/crash-drill.mjs:38: the candidate accepted an interrupted run that became failed with any detail. Countercheck against development_runs_one_active_task and startup reconciliation showed that only reconciling retains the active-run barrier; the candidate could therefore report pass while a new run for the same task was allowed. Accepted with red-first test commit 527a761 and fix commit 02b3e33.
  • No high finding remains open. Review complete (tier B).

Open points

  • The installed Windows drill and the merge-queue Windows lane remain pending.

Before/after snapshots of run, launch and delivery state from a copy of
the database, a restart judge (reconciling, no redispatch, no silent
change), PowerShell launcher and German steps.

Test-First: scripts/lib/crash-drill.test.mjs::an open run that becomes reconciling passes
Test-First: scripts/lib/crash-drill.test.mjs::an open run left open or completed silently fails
Test-First: scripts/lib/crash-drill.test.mjs::a new run for a task with a reconciling run is a redispatch
Test-First: scripts/lib/crash-drill.test.mjs::a delivery must not advance without a process and a drill needs an open run
Test-First: scripts/lib/crash-drill.test.mjs::before and after bundles read a copy and pass for a reconciled run

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@Cuarroc
Cuarroc marked this pull request as ready for review October 4, 2026 20:55
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-04T21:01:42.988005Z 4e808b0 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4e808b074d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +55 to +57
const state = snapshotState(live);
bundle.addFile('state.json', JSON.stringify({ transition, ...state }, null, 2));
bundle.step('snapshot state after the kill (copy, read-only)', { detail: `transition ${transition}; ${state.runs.length} runs` });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Validate the requested transition before recording evidence

When the user supplies any of the six transition labels, runBefore merely writes that string beside a database-wide snapshot, while judgeRestart never uses it. Consequently, a run killed during intent but labeled checkpoint can pass once any open run becomes reconciling; claimed and completion can likewise be falsely represented by an unrelated open run. Bind the evidence to a specific run and verify the corresponding launch, delivery, or checkpoint state before accepting the label.

AGENTS.md reference: AGENTS.md:L16-L18

Useful? React with 👍 / 👎.

Comment on lines +19 to +20
for (const s of ['', '-wal', '-shm']) if (existsSync(dbPath + s)) copyFileSync(dbPath + s, join(tmp, `projecta.db${s}`));
const con = new DatabaseSync(join(tmp, 'projecta.db'), { readOnly: true });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Take an atomic snapshot of the live WAL database

During the documented after phase ProjectA is running, so it can commit or checkpoint between these three independent copies of the main database, WAL, and shared-memory files. The resulting temporary set may combine different SQLite states, producing an error, stale rows, or a misleading restart verdict. Use a coherent SQLite online snapshot or quiesce the writer before using this data as runtime evidence.

AGENTS.md reference: AGENTS.md:L16-L18

Useful? React with 👍 / 👎.

export const TRANSITIONS = ['claimed', 'intent', 'process-starting', 'input-delivery', 'checkpoint', 'completion'];
const OPEN = ['intent', 'launched'];
const QUERIES = {
runs: 'SELECT id, task_id, status, claim_fence, process_id, terminal_detail FROM development_runs ORDER BY id',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Exclude unstructured terminal details from evidence bundles

When any historical blocked run has an agent-supplied failure reason containing a bare credential, this query exports it through terminal_detail into state.json and the after bundle. The shared redactor only recognizes named key/value or authorization-header forms, so a value such as a bare ghp_... token survives unchanged despite the documentation saying the bundle is redacted. Omit this arbitrary field or apply fail-closed secret sanitization before persisting it.

AGENTS.md reference: AGENTS.md:L44-L46

Useful? React with 👍 / 👎.

Comment on lines +72 to +74
if (processList) bundle.addFile('processes-after.txt', processList);
const problems = judgeRestart(before, after);
bundle.step('judge restart: reconciling, no redispatch, no silent change', { command: `transition ${before.transition}`, exitCode: problems.length ? 1 : 0, detail: problems.join('; ') || 'ok' });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Verify that no provider process survived or restarted

During both phases the launcher captures a process list, but runAfter only stores it and the verdict checks database run IDs rather than OS processes or launch identities. A provider child can therefore survive the ProjectA kill, or a process can restart under the existing run, while that run becomes reconciling; the drill still reports pass even though its central claim is that nothing restarted before process and database state agree. Compare the recorded run/process identity with both process snapshots and fail when the old process remains or a replacement appears.

AGENTS.md reference: AGENTS.md:L16-L18

Useful? React with 👍 / 👎.

Comment on lines +44 to +45
const delivery = new Map(after.deliveries.map((d) => [d.run_id, d.state]));
for (const d of before.deliveries) if (d.state === 'started' && delivery.get(d.run_id) === 'enqueued') problems.push(`run ${d.run_id}: delivery went started -> enqueued without a live process`);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Fail when durable transition records disappear

If a committed delivery row present before the restart is missing afterward, delivery.get(d.run_id) is undefined, so this condition does not report any problem; launch rows and checkpoint counts are not compared at all. The drill can therefore return pass after losing the durable launch, delivery, or checkpoint record that the selected crash transition was meant to exercise, provided the run itself becomes reconciling. Compare the target run's durable records across snapshots and reject deletions or regressions.

AGENTS.md reference: AGENTS.md:L16-L18

Useful? React with 👍 / 👎.

@mergify

mergify Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Queued — the merge queue status continues in this comment ↓.

Test-First: scripts/lib/crash-drill.test.mjs::a failed run cannot pass because it releases the redispatch block
Test-First: scripts/lib/crash-drill.test.mjs::a failed run cannot pass because it releases the redispatch block
@Cuarroc

Cuarroc commented Oct 4, 2026

Copy link
Copy Markdown
Owner Author

@Mergifyio queue

@mergify

mergify Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

This pull request spent 26 minutes 56 seconds in the queue, including 14 minutes 16 seconds running CI.

Required conditions to merge

@mergify mergify Bot removed the queued label Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant