[Silthus]

Problem

• A developer who schedules an email (N3, #128881) cannot cancel or move it while it waits (Silthus/personal#303).

Warning

🟡 Merge risk: medium · two-way door until Django exposes cancel and reschedule to customers

• Today's sends with a reused file, from teams on the files allowlist in US and EU, now get an S3 copy at accept, so the file outlasts any move. A failed copy answers 503 delivery_unavailable and frees the key; added accept latency at volume is not measured. No stored job carries files in either region (prod-shape).
• The new routes have no caller until Django mints their token. Every send stores statusAt; the N3 cross-version run and the shadow diff show N3 reads those jobs and nothing else changes.
• The upsert guard meets no stored row: no job is canceled in either region (prod-shape).
• A scheduled send's idempotency key now lives until a day after the last time a reschedule can move it to, creation plus 31 days; keys expire on their own and the base reads them the same way.
• Switch-off: none: live on deploy, fix forward.

Rollout

• Stages: lands with N1 to N3 in one merge, so no N3-only pod ever runs.
• Watch: cdp_transactional_email_requests_total{outcome="delivery_unavailable"}; no alert pages on it.

Merge first: #128881 (with #127745 under it), #128871, then #128868.

Landing: #129915 merges alone and its migration runs in US and EU first. #127745 merges next. Then one /trunk merge on #128929 lands N1 to N4.

Changes

• New cdp-api routes cancel or move a scheduled email before a worker claims it. No caller exists until Django mints their token.
• Cancel: 200 canceled, also on a repeat; 409 already_sending after a claim; else 404, also for a job id on a queue other than email.
• A repeat cancel writes the canceled status row and run row again, without metrics, so a retry after a crash between the commit and those rows still shows the email canceled.
• Reschedule: 200 scheduled; a past time sends now; past 30 days from creation 400; 409 once claimed or canceled.
• The repeat 200 and the 409s hold while the job row exists; after the janitor deletes it, 404.
• Each route checks a JWT of a new audience, pinned to the team, the message and the action, so a token minted for one cancel cannot cancel or move another send.
• Cancel writes a canceled status row and the run's terminal row, at the time of the run's running row, so history shows the run canceled on the day it was listed.
• The canceled row and a reschedule's new scheduled row name the recipient and the workflow, and the new row carries the new scheduled_at, so the latest row per message is enough for the waiting-sends list.
• Two reschedules of one email that overlap keep the time of the one that committed last. The new scheduled row takes its status time from the database right after the row lock, not from when the row is written, and at least one millisecond after the job's previous scheduled row. The send and each reschedule keep that row's time in the job state, so a lagging database clock cannot put the first reschedule behind the send.
• Every send with files keeps them until a week after the last day a reschedule can move it to, 37 days after creation. A send that goes out at once can still be moved while it waits, so it gets the same rule. A file uploaded just before the send lasts that long anyway; only an older, reused file gets an S3 copy at accept (Michael's decision, 2026-11-12).
• Riskiest part: an overwriteExisting upsert leaves a job canceled before any claim alone, so a rerun cannot revive it while its row exists. After that, a manual rerun can (spec, out of scope).

Note

The rerun premise holds (2026-11-04): a rerun upserts with overwriteExisting, so the guard is needed. Details below.

How did you test this code?

• Manager tests on real Postgres: a held lock plus a bumped transition_count stands for a claim; team scoping; the upsert guard.
• Serial API tests: each answer, the token checks, the status rows, metrics counted once.
• A scratch live run with the base's worker and janitor, below. Not committed.

Live run, cross-version and shadow diff at fd4106b, and the retried cancel at 75f0ddb

Real cdp-api, queue, email worker, maildev and S3 store. The base is N3 at e351094.

Scenario	Result
Cancel before due, twice	200 canceled both times; a canceled status row; no mail.
Cancel or move a sent email	409 already_sending.
Move nearer, later, then back	One mail, at the last time only.
Unknown or another team's message	404.
Bad, expired or wrong-audience token	401.
A time past 30 days from creation	400; exactly 30 days is accepted.
Send with an older file	Copied at accept. Moving it to 30 days from creation touches no file; the mail carries it.
The base schedules, this PR cancels and moves	The moved send goes out with its file; the canceled one never does.
This PR schedules, the base worker runs	Plain and moved sends go out at their times; the canceled one never does.
Shadow diff against the base	Differs only in statusAt, the longer file keep, the key lifetime, the canceled rank and the upsert guard.
A cancel committed without its rows, then retried (at 75f0ddb, worktree box)	Before the retry the email reads scheduled and the run running; the retry answers 200 and writes both canceled rows.

Not run: a rolling deploy on real infrastructure, and a rollback. A replay with the original body after a move or a cancel answers the original scheduled reply, as any idempotent replay does.

Release status

• No feature flag controls this change

Docs update

• None.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5

Skills: /writing-pr-descriptions, /writing-tests, /writing-code-comments

More context for AI agents

• Session: T3 Code stack thread for Silthus/personal#303, Claude Code.
• Decisions: the manager's rescheduleUnclaimed locks the row (FOR UPDATE, no SKIP LOCKED), decodes the JSON state in JS and writes it back, because a jsonb rewrite in SQL fails on a null character in email content. A cancel keeps transition_count 0, which marks the row for the upsert guard. A non-UUID message id answers 404 before any query. Reschedule takes only the time and writes the state's scheduledAt from it. The API cancel and the workflow worker share one canceled-result builder, and the cancel loads the workflow (primary on a replica miss) so its metrics carry the version. A repeat cancel rewrites its rows only for a job canceled before any claim, so a worker's cancel after a claim writes nothing new (the author's choice for a PostHog Review finding, 2026-11-12). The per-item comments are dropped at the author's request; the CI report's comment density is 2%, and the remaining lines moved from master, such as the canceled-result builder's block.
• Open check R4 Q7: feat(cdp): deliver a code send's content on its own transactional email route #127122 still skips a run whose state holds send unless the request sets replay_code_sends. feat(messaging): send an email with inline html from code #127123 sets it from the code-send flag, so with the flag on a code send reruns through the upsert, and a template_id send reruns either way.
• Follow-ups: Django mints the token of the new audience with team_id, message_id and action claims, maps the answers, and adds canceled and scheduled to EmailDeliveryStatus (Django stack). The waiting-sends list reads the latest email_status_events row per message (Django stack). A moved send that later sends writes its terminal row at the new time, as a fetch retry does today. The history query filters scheduled_at before it collapses rows, so the two days can disagree; a separate change can fix the query.
• Read closely: cancelUnclaimed, rescheduleUnclaimed and the upsert guard in manager.ts.

🤖 Generated with Claude Code

[github-actions]

🤖 CI report

✅ Trunk lane — non-backend lane

This PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Comment density — 2% of added code lines are comments (22 of 1096)

This section warns when comments are more than 3% of the code lines a PR adds, and alerts above 6%. Before agent-assisted PRs, the typical share was about 2%. Only full-line comments count. Docstrings, generated files, snapshots, migrations, and workflow files are left out.

Comments that restate the code, record how the change came about, or narrate the next line add noise for the next reader. Keep the comments that explain a reason the code cannot show, and remove the rest. See .agents/skills/writing-code-comments/SKILL.md for the house rules.

Files with the most added comment lines:

File	Comment lines	Added lines
nodejs/src/cdp/utils/invocation-utils.ts	13	35
nodejs/src/cdp/services/cyclotron-v2/cyclotron-v2.test.ts	6	234
nodejs/src/cdp/services/cyclotron-v2/manager.ts	2	117
nodejs/src/cdp/cdp-api.serial.test.ts	1	432

This check does not block merging. It updates on every push and clears when the share drops.

[posthog]

PostHog Review · Standard · reviewed 66423db: 2 Should fix.

This is the only automatic review while this pull request is a draft. Comment @posthog review to review new changes sooner. Otherwise, a review of the new changes runs when you mark it ready for review.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

[parameterai]

Risk: No findings

This increment rewrites a canceled send's status rows when a cancel request repeats: the manager's already-canceled fallback now returns the job row when it was canceled before any claim (transition_count 0), and the service re-records the canceled run without double-counting metrics. The changes are correctly team-scoped and token-pinned; I found no new security issues.

_{Sentinel reviewed 75f0ddb · Review settings}

[posthog]

PostHog Review · Standard · reviewed 9537cbd: 1 Should fix.

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

Reviews

• 75f0ddb — No new issues
• fd4106b — No new issues
• 9571f8b — No new issues

[posthog]

PostHog Review · Standard · reviewed 4b7ead5: no findings.

[stamphog]

Not approved — this change needs a human reviewer.

Re-adding the stamphog label gives the same result unless the changed files change.

stamphog can't approve this pull request on its own, because two gates refused it. The deny-list gate flagged it for matching the auth category, which fits the new JWT audience and the changes to jwt-utils.ts and internal-api-auth.ts. The tier gate classified it as T2-never (1429 lines across 17 files, a feature), a class that always needs a human reviewer. The size gate passed, so splitting the PR won't clear either refusal. Please ask a human reviewer, ideally someone who owns the auth and cyclotron code, to review it.

Gate mechanics and policy version

Gate		Result
prerequisites	✓	all clear
deny-list	✗	matches: auth
size	✓	500L, 13F substantive, 1429L/17F incl. docs/generated/snapshots — within ceiling
tier	✗	classified as T2-never: T2-never (1429L, 17F, single-area, feat)
stamphog 2.8.0		.stamphog/policy.yml @ unknown · reviewed head 75f0ddb

[Silthus]

/trunk merge

[trunk-io]

😎 Stack merged successfully - details.

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-11-12 10:34 UTC	Run
prod-us	✅ Deployed	2026-11-12 10:56 UTC	Run
prod-eu	✅ Deployed	2026-11-12 11:07 UTC	Run

_{This table shows the posthog release only. Hogstation shows every service that this PR reaches, Kargo services included.}