Repository navigation
Sandbox sender: keep sends out of tier history and record the first sandbox delivery #236
Description
Map: Map: let teams send a first email before verifying their domain
Spec: Spec: sandbox sender
Cast: GPT-6.1 Sol (Michael, 2026-10-03)
Question
Do sandbox sends stay out of sending-tier history, and does the first sandbox delivery show up as its own funnel step?
Delivers end to end. Spec section "Sending tiers and funnel":
- Node pushes
email_sandbox_sentalongsideemail_sentfor every live sandbox send (test sends push no app metrics, as today). - The tier sweep subtracts
email_sandbox_sentfromemail_sentper team per day, floored at zero. Bounces and complaints keep counting. - The SES webhook captures
workflows sandbox email delivered(is_test) when aDeliveryevent's configuration set isSES_SANDBOX_CONFIGURATION_SET, for the team named by the verified tracking code. Test sends included. - List the funnel in the PR body:
workflows sandbox sender provisioned→workflows sandbox email sent→workflows sandbox email delivered(first time), withworkflows sandbox email blockedas the drop-off. State that it is not activation.
Test seams. Node: EmailService.executeSendEmail for the companion metric; EmailTrackingService.handleSesWebhook with signed SNS Delivery events on the sandbox and the normal set. Django: recompute_email_sending_tier_for_team over app metrics rows in ClickHouse.
Write scope. nodejs/src/cdp/services/messaging/email.service.ts, nodejs/src/cdp/services/messaging/helpers/ses.ts or email-tracking.service.ts, products/workflows/backend/services/email_sending_tier.py, products/workflows/backend/metrics.py if the metric name belongs there, and tests.
Acceptance criteria
- A team with only sandbox sends computes the same tier as a team with no sends.
- A team with own and sandbox sends counts only the own sends toward history.
- A delivery on the sandbox set fires one
deliveredevent; a delivery on the normal set fires none.
How to build (map #225)
/tddred-green-refactor, outside-in, at the seams named above. Real Postgres, Valkey and ClickHouse; mock only the SES client (Node and boto3) and the PostHog analytics client.- Repo skills apply where the diff touches their area:
/writing-tests,/improving-drf-endpoints,/django-migrations,/writing-ui-components,/writing-kea-logics,/writing-user-facing-copy,/writing-code-comments. - Public repo: state sandbox rules as product rules. Cap values come from the private note the map points at; never quote it. Fixture addresses use
example.com. - Shipped means a draft PR handed off by
/qa-swarm(see the map's Notes for remotes and branches). Never mark ready, enqueue or merge.
Activity
Silthus commentedon Oct 3, 2026
Scope addition (conductor, from the outside-code checklist):
SES_SANDBOX_CONFIGURATION_SETmust also reach the Node service that serves the SES webhook, not only the email worker. Without it theworkflows sandbox email deliveredevent never fires. Cover that service's config in this slice and its test.Silthus commentedon Oct 5, 2026
Started by ticket worker
impl-236on branchfeat/workflows-sandbox-first-delivery(stacked on draft PR PostHog#111820 ata66bb200f89).Silthus commentedon Oct 5, 2026
Note
🤖 Claude Sonnet 5.5 responding on behalf of Michael
Lane owner changed to
impl-236b(Claude Sonnet 5.5) after Opus 5.5 hit rate limits. The implementation and gate onbc81df2ec2cstand; QA round 1 on draft PR PostHog#111904 is starting.Silthus commentedon Oct 5, 2026
Note
🤖 Claude Sonnet 5.5 responding on behalf of Michael
Shipped in draft PR #111904: QA handoff at
d9ef8d19c62, round 2 came back clean with no fix item. Merge risk is low.email_sandbox_sentfromemail_sentper team per day, floored at zero. Bounces and complaints keep counting. A sandbox-only team gets the same tier as a team with no sends.workflows sandbox email delivered(is_test) for aDeliveryon the sandbox configuration set, for the team named by the signed tracking code. A delivery on any other set, or with an unsigned code, captures nothing.@posthog/replay-anonymizerpackage, which this PR does not touch.deliveredevent on SNS redelivery and a swallowed team lookup are not fixed. The funnel needs one event, and capture stays best-effort like thesentandblockedevents.New per-environment setting:
SES_SANDBOX_CONFIGURATION_SETmust reach the CDP API deployment, not only the email worker. Without it the delivered event never fires.Follow-ups, outside this ticket:
email.service.tsclaims the sandbox daily cap tokens again (code from feat(workflows): skip sandbox sends while the sandbox SES tenant is paused PostHog/posthog#111820, raised by CodeRabbit).email_sent, so sandbox sends show as used tier allowance. Subtractemail_sandbox_sentthere too.Gaps: fork CI approval is still pending, as on the earlier tickets. CodeRabbit CLI was signed out, so the local pass was skipped.