Skip to content

feat(cdp): list waiting scheduled sends and count them - #129010

Closed
Silthus wants to merge 12 commits into
scheduled-sends/n5-indexfrom
scheduled-sends/n5-list-scheduled
Closed

Silthus wants to merge 12 commits into
scheduled-sends/n5-indexfrom
scheduled-sends/n5-list-scheduled

Conversation

@Silthus

@Silthus Silthus commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Problem

  • A developer who scheduled emails (N3, #128881) cannot see which ones still wait or when they go out (Silthus/personal#303).
  • Operators cannot see how many scheduled sends wait across teams.

Merge first: #129035 (the index, deploys first), #128929, #128881, #128871, then #128868.

Landing: queue this PR with /trunk merge only after #129035 has landed and its index migration is applied in US and EU. A reader of a new index lands after the migration is live, not in the same queue run.

Changes

  • A new cdp-api route lists a team's waiting scheduled emails in send-time order, 100 a page at most, with an opaque cursor. It uses N4's team-only token; Django is the first caller, in a later stack.
  • Each row has id, to, scheduled_at and created_at.
  • The janitor sets cdp_transactional_email_scheduled_waiting once a minute, one count at a time.
  • Riskiest part: the count on the queue table. It reads the partial index from feat(cdp): index email jobs that wait for their send time #129035.

How did you test this code?

  • Manager tests on real Postgres: order, pages, team scoping, what is not listed, the plan uses the index, and state content that Postgres json rejects.
  • Serial API tests for the answer, bad input and the token checks.
  • A scratch live run, below.
Live run and cross-version results at e54e397
Scenario Result
Three sends sent out of order, limit=1 Three pages in time order, the last next is null. A send-now is not listed.
Another team's token Lists only its own sends; its token on this team's path answers 401.
Cancel one The list shows two.
Gauge Matches a direct SQL count of the same predicate.
Base (N4) worker with the new index Sends the job at its time; the list then answers empty.
List before the migration (index dropped) Same answer through a seq scan. The migration's SQL rebuilds a valid index.

Release status

  • No feature flag controls this change

Docs update

  • None.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5

Skills: /writing-pr-descriptions, /writing-tests, /writing-code-comments

More context for AI agents
  • Session: T3 Code stack thread for Silthus/personal#303, Claude Code.
  • Decisions: the index ships in feat(cdp): index email jobs that wait for their send time #129035, because the repo's check refuses a cyclotron migration and Node code in one PR. The recipient is decoded in JS: Postgres json operators fail on a null character or a lone surrogate anywhere in the state. The cursor is base64url JSON of the last row's (scheduled, id); the first page runs the same keyset query from the epoch. The list fetches one extra row to know whether a next page exists. to comes from the job state's event properties. The count is a function the janitor calls with its own pool, because the janitor has no manager. The gauge keeps its last value when a count fails. The CI report's comment density (4%) is why-comments only.
  • Follow-ups: the 100,000 waiting-sends alert lives outside this repo (deploy follow-up). Django maps the list route (Django stack).
  • Read closely: listUnclaimed, and UNCLAIMED_EMAIL in manager.ts, which must match the predicate of the index in feat(cdp): index email jobs that wait for their send time #129035.

🤖 Generated with Claude Code

Silthus and others added 2 commits November 4, 2026 10:32
A developer who scheduled emails can page through the ones that still wait,
in send-time order, through a cdp-api route on the team-only token of the
cancel and reschedule routes. A partial index over the unclaimed email jobs
serves the list. The janitor counts the waiting sends across teams once a
minute into cdp_transactional_email_scheduled_waiting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

✅ Trunk lane — non-backend lane

This PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

⚠️ Comment density — 4% of added code lines are comments (14 of 331)

This section warns when comments are more than 3% of the code lines a PR adds, and alerts above 6%. Before agent-assisted PRs, the typical share was about 2%. Only full-line comments count. Docstrings, generated files, snapshots, migrations, and workflow files are left out.

Comments that restate the code, record how the change came about, or narrate the next line add noise for the next reader. Keep the comments that explain a reason the code cannot show, and remove the rest. See .agents/skills/writing-code-comments/SKILL.md for the house rules.

Files with the most added comment lines:

File Comment lines Added lines
nodejs/src/cdp/services/cyclotron-v2/cyclotron-v2.test.ts 5 105
nodejs/src/cdp/services/cyclotron-v2/janitor.ts 3 33
nodejs/src/cdp/services/cyclotron-v2/manager.ts 3 46
nodejs/src/cdp/services/hogflows/transactional-email.ts 1 26
nodejs/src/cdp/services/hogflows/transactional-sends.ts 1 23
nodejs/src/common/api/middleware/internal-api-auth.ts 1 2

This check does not block merging. It updates on every push and clears when the share drops.

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed 310805e: 2 Should fix.

This is the only automatic review while this pull request is a draft. Comment @posthog review to review new changes sooner. Otherwise, a review of the new changes runs when you mark it ready for review.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

Silthus and others added 5 commits November 4, 2026 11:06
The query schema decodes the cursor, so a malformed or out-of-range
cursor answers 400 from one branch. A manager test row holds content
that Postgres json rejects, so a later SQL read of the recipient fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nds/n5-list-scheduled

# Conflicts:
#	nodejs/src/cdp/cdp-api.serial.test.ts
A count that outlasts the one-minute tick no longer starts a second one
on the janitor's pool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Silthus
Silthus changed the base branch from scheduled-sends/n4-cancel-reschedule to scheduled-sends/n5-index November 4, 2026 11:24
@Silthus

Silthus commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

/depot retry

Both Depot backend runs for b3a0542 were cancelled by the
superseded-run job, and /depot retry was cancelled the same way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@parameterai

parameterai Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Risk: No findings

This PR adds a team-scoped, JWT-protected GET endpoint listing waiting scheduled transactional emails with opaque keyset pagination, plus a janitor gauge counting waiting sends across teams once a minute. The auth path (audience- and team-pinned HS256 JWT, fail-closed on missing secret), the parameterized SQL, and the validated cursor/limit inputs all check out; I found no new vulnerabilities.

Sentinel reviewed 34e99b7 · Review settings

@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team November 5, 2026 07:34
@Silthus

Silthus commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Closing: the waiting-sends list reads the latest row per message in email_status_events (#129915) instead of the job queue. #128929 writes every row it needs, so this layer has nothing left.

@Silthus Silthus closed this Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants