Skip to content

feat(skills): move Loops sends to PostHog - #462

Draft
Silthus wants to merge 409 commits into
mainfrom
feat/migrate-email-loops
Draft

Silthus wants to merge 409 commits into
mainfrom
feat/migrate-email-loops

Conversation

@Silthus

@Silthus Silthus commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Parked until after 2026-11-25. Michael chose SendGrid as the first migrate-email provider on 2026-11-14; this PR stays a draft and resumes after that date, with the slim pass from research/2026-11-14-review-bloat-retro before its next review round.

Problem

A person who sends transactional email with Loops and asks to move those sends to PostHog gets no help: migrate-email covers Resend, and each other provider comes in its own PR on top of #472.

With this PR, migrate-email-loops moves Loops transactional sends to PostHog's email send, in each language where both Loops and PostHog have an SDK or the app calls Loops' HTTP API, and wizard migrate loops runs it.

Important

Built on #472, the core migrate-email skill, which has merged: this PR targets main, with main merged in. Each provider PR adds only its own provider and lands in any order. The lists they all touch, config.yaml, AGENTS.md and the skill test, conflict in one line each as they land.
Nothing merges until code sends are live and usable for customers: Michael's rule for #451 and every PR stacked on it.

Impact

Sends that arrive with migratedFrom: 'loops' show whether people use this skill to move off Loops. Read it 14 days after the Wizard runs migrate as a family.

Merge risk

Caution

🔴 Merge risk: high · two-way door until the next context-mill release publishes the skill, one-way once it is published · no switch-off

  • A wrong instruction in this folder silently changes how an agent rewrites someone's Loops sends: the diff compiles and looks right, and the difference shows only in a moved email after their deploy.
  • The skill ships with the next context-mill release once merged, to everyone who runs wizard migrate loops. No flag holds it back.
  • Uncovered: no live run has applied these rules yet (recorded gap). The shadow diff shows the build changes only the Loops bundles and the manifest.
  • Uncovered: Loops has no suppression export. When the app kept no record of its Loops bounce and complaint webhooks, the import checks audience contacts only, so a recipient who gets only transactional sends and bounced or complained at Loops is not imported, and a moved send can mail them. The report lists that import as partial and, while a site stays on Loops, says its weekly re-imports miss those recipients and that the app's record covers them.
  • Suppressions after feat(migrate-email): move Resend sends in every PostHog SDK's language #472: PostHog blocks a moved send only to an address imported as a hard bounce, so an address the app's record shows as a spam complaint gets the moved sends, as the report says. An address the API check finds suppressed is imported as a bounce, since the API doesn't say why: moved sends skip it, as Loops did, even when only a complaint suppressed it.
  • Uncovered: the suppression check, through the API or of the app's record, calls Loops once per address with the team's key, and the app's own Loops sends share that team's rate limit. The skill caps the check at half that limit, tells the person how long it takes and that it shares the sending limit, and asks before it starts, at a quiet hour. A send of the app that still hits the limit during the check gets Loops' rate-limit error, which the app handles as it does today; no run has measured that load (recorded live-run gap).
  • One-way once published: the release makes wizard migrate loops a public command that any agent can run, and a migration's changes persist in each app's code and PostHog account, such as the moved sends, the sender and its DNS records, the imported suppressions, and the PostHog key. No switch-off: a revert only stops new runs, since the next release ships without the Loops variant.
  • Switch-off: none: live once the release publishes it, fix forward.
  • Tier: critical (b: the grade is high)

Summary

references/loops/ holds the three files each provider has:

  • sdk-reference.md: Loops' official libraries in Node.js (and the Nuxt module), Go, PHP and Ruby, the community packages, the HTTP API, SMTP through smtp.loops.so, and Auth.js's Loops provider: how each sends, how it reports errors (API, rate limit and network errors apart), base URL options, the transactional emails stored at Loops with their data variables, arrays and conditional sections, the suppression removal calls, webhooks, and the calls that are not sends.
  • mapping.md:
    • Every Loops email comes into the code. A Loops send names an email stored at Loops, so each site moves by bringing that email into the code and sending it in the code form: templateKey, the rendered subject and html, and from and replyTo from the email's settings. A template made in PostHog (templateId with templateData) is used only when the person asks for one.
    • From LMX to HTML. The content API answers the body in LMX, Loops' markup. A table turns each LMX tag into email HTML (tables, inline styles, the theme's defaults), and a preview send stands in when the API can't answer, such as for an MJML email. Data variables, fallbacks, conditional sections and array blocks become code, with Loops' text-only truth test written out.
    • Fields. email becomes to, and Idempotency-Key becomes idempotencyKey when the code bounds its characters. What keeps a site on Loops: addToAudience: true unless the same code path creates or updates the contact first, attachments, other request headers, a From that holds a data variable, a CC or BCC, contact or event properties, and number or boolean conditions.
    • Tracking. Loops tracks no opens or clicks on a transactional email, at any setting or plan, so every Loops site falls under Step 3's shared rules: "Opens the provider didn't track" asks once about the open pixel, and a template made in PostHog follows the untracked-clicks rule. The Loops folder only states that Loops never tracked them.
    • Deadlines. "HTTP" names the timeouts that count as a site's own deadline, including Go's loops.WithHTTPClient and PHP's setHttpClient, and Step 7 keeps them. An axios timeout and Faraday's timeout on a non-net_http adapter are limits Step 7 says the SDK can't keep.
    • Calls and errors. A Nodemailer send through smtp.loops.so is read as the API send, since its text is the JSON body. Framework mailers, Auth.js, and a send after the code removes the recipient's Loops suppression stay. Each language keeps the site failing the way it did and never widens a catch: 400 reads validation_error, 409 reads idempotency_key_reused, a rate-limit branch takes 429 without quota_limited, and a retry loop on 429 stays.
    • Templates made in PostHog. When the person asks for one, Loops' variables become Liquid, single-quoted. A section stays hidden when its condition variable is left out, for every operator, as at Loops.
    • Webhooks. Each transactional event maps to a $workflows_email_* event, and the report names the handlers that read them.
  • account.md: Loops' DNS records next to PostHog's, a separate migration key per Loops team (Loops keys have no scopes) that the person revokes at the end of Step 5, how to read an email's content from the API or a preview send, and the suppression import: Loops has no export, so it imports from the app's own webhook record or checks each audience contact at no more than 5 requests a second, and marks an audience-only import partial. Its report note says the core's weekly re-import, when it comes from the audience check, never covers a recipient who is not a contact, and names the app's record as what covers them.

Evidence

  • Lane gate on this head: vitest run scripts and node scripts/build.js pass locally.
  • Test seam (migrate-email-skill.test.js): loops joins PROVIDERS, so the shared bundle tests generate the Loops skill and check that every reference resolves and that it holds its three files and the rewrite files.
  • Gaps:
    • live-run: an agent run of this provider's migration on a real app needs the released email SDKs and the production send API. Michael's planned trial on generated codebases runs it after both land. Earlier dry runs on a fixture app moved sites in the template form, before the core moved stored templates into the code; none has run the code form yet.
    • prod-shape: customer-written Loops send code is the input. A survey of real codebases' Loops send shapes ran on an earlier head and shaped the rules for the retry loop, a transactionalId from an env variable, a helper whose callers pass the ID, and HTTP sends; the email content the code form now reads lives only in each person's Loops account. Michael's planned trial covers both.
    • CI: now that this PR targets main, build.yml runs on it and passes on this head, next to the lane gate's local vitest and build.js.

Decisions

Question Answer Who Where
When this merges Not before code sends are live and usable for customers, for #451 and every PR on it 👤 Human this PR
Loops tracking Loops' own open-pixel ask, which Michael chose, moved into the core as Step 3's "Opens the provider didn't track" for every provider; the Loops folder now only says Loops never tracked opens or clicks 👤 Human this PR
Every Loops site Brings its Loops email into the code and sends the code form; a PostHog template only when the person asks. Replaced "moves only to a template the person makes in PostHog" 🤖 Agent this PR
addToAudience: true Keeps the site on Loops unless the same code path already creates or updates the contact, since the send also adds the recipient to Loops' audience 🤖 Agent this PR
Nodemailer through smtp.loops.so Read as the API send, since its text is Loops' JSON body; moving it as plain SMTP would deliver the JSON 🤖 Agent this PR
Suppressions without an export Import from the app's own record, each address checked against Loops first, or check each audience contact at no more than 5 requests a second; mark an audience-only import partial, and say in the report that weekly re-imports from the audience check miss recipients who are not contacts 🤖 Agent this PR
The migration key A separate key per Loops team, revoked at the end of Step 5, because Loops keys can't be limited to reads 🤖 Agent this PR
A missing condition variable in a PostHog template Hides the section for every operator, as Loops does: empty is == '', and not_equal and not_contains check the value is there first 🤖 Agent this PR
A required data variable the code can leave out The render fails before the send, the way the site failed on Loops' 400, but only when the variable appears in content that shows for the site's values: Loops drops a hidden section with its variables. A variable only in hidden sections is named in the plan row, since Loops' docs don't say whether its 400 check skips it. The same holds for a PostHog template's precheck 🤖 Agent this PR
Optional variables read through the API Asked of the person for each variable without a fallback, since the API's dataVariables and LMX don't mark one optional; when they can't tell, the site stays 🤖 Agent this PR
Go context context.Background(), since Loops' call took none and never stopped when a request ended 🤖 Agent this PR
Merge risk High, no switch-off: a wrong rule silently changes someone's sends, ships with the next release, and has no live run 🤖 Agent this PR
A deadline on a Loops request Kept, as Step 7 says. Connect-only limits and network-failure branches follow Step 7's shared rules; this folder names only the Loops settings and error classes they cover. Go's WithHTTPClient timeout limits each try and Loops retried twice, so it keeps PostHog's retries; the default 5-second Go timeout is not the site's own. A retry setting alone is not a deadline 🤖 Agent this PR
A null data variable Treated as left out, or as the fallback when there is one, and named in the plan row: Loops' docs say a null will not work, but not whether Loops then refused the send or left the value out 🤖 Agent this PR
Loops suppressions after #472 Each source maps to PostHog's reason: a hard bounce in the app's record to bounce, a spam report to complaint, and an address the API check finds suppressed to bounce, since the API doesn't say why and PostHog blocks moved sends only for a bounce. The report says those addresses are blocked, complaints included. Step 3's new question is told that Loops has no way to add an address to its list 🤖 Agent this PR
Loops bounce webhooks Both map to $workflows_email_bounced, told apart by $bounce_type (hard or soft); the report says to filter on it before a hard-bounce handler 🤖 Agent this PR
Loops' rate-limit limit and remaining Logged reads become the status code; a handler that paces or requeues work by them keeps its site on Loops 🤖 Agent this PR
A narrow catch on one Loops error Becomes a catch of PostHog's error that checks the same code, name or status and raises the rest, never wider 🤖 Agent this PR

Involvement: qa-swarm, 2 panel rounds (round 1: fresh-eyes, correctness, risk, design, anti-overengineering, sweep, and the team lenses of the author and a code owner; round 2: the lenses with fixes) and 20 final reads, reads 7 to 20 past the cap of six, each with its reason recorded. Read 12, on the fifth merge of #472, found that the core's weekly re-import can't keep its one-week promise for Loops' audience-only import; that is fixed in 2cb7e2b. ReviewHog ran 6 rounds before #472 landed; each finding was fixed or answered. After #472 landed, main is merged in and the suppression mapping follows its shared steps; read 15 found that the report would say complained addresses get the moved sends while the API-checked ones are imported as bounces, fixed in c829289, and read 16 was clean on it. The proof judge then found that the preview-send guard missed a not_contains section, whose second preview shows a value that can't be told from fixed text; fixed in 1f5bd4d, and read 17 was clean on it. ReviewHog's round after #472 found that a migrated retry loop dropped the wait PostHog's error names; it now reads the retry delay under each SDK's name and unit, fixed in 598d376, and read 18 was clean on it. The proof judge then found that the Node.js bullet's thrown error dropped that delay; it now copies retryAfterMs, fixed in ba82a36, and read 19 was clean on it. ReviewHog's next round found that Loops' Go client also retried a 5xx, which PostHog's SDK doesn't; the rule now says so and notes it in the plan row, fixed in ff8ea2b, and read 20 was clean on this head. Shadow diff on this head against main: only the new Loops bundle, the manifest and the skill menu change. Live-run and prod-shape are recorded gaps.

🤖 Agent context

Agent: Claude Code, Claude Opus 5.5 (claude-opus-5-5, high). Reviews on GPT-6.1 Sol (high), the fresh-eyes lens on Opus 5.5.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci feature-flags
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci feature-flags/django
  • /wizard-ci feature-flags/next-js
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a wizard branch:

  • /wizard-ci all wizard:my-branch

Add wizard:<branch> to any command above to pin the wizard branch. It defaults to main.

Results will be posted here when complete.

Silthus and others added 28 commits November 4, 2026 13:12
…domains whose sites move

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s that mirrored additions made

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… unchecked request input is named for the person

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ss set

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ite on the provider

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…to the API error's path

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…g, like Ruby's test mode

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…finds it

Step 6 ticked the key phase as set when the person said they would add
the key themselves. The step now checks the env file with a count-only
grep (or a successful set_env_values in the Wizard). A promise to add it
later, or a refusal, records the key as not set, and the report lists it
before cutover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dotenv loads a key with leading whitespace or spaces around =, so the
count-only grep accepts them too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r' into feat/migrate-email-sendgrid

# Conflicts:
#	context/skills/migrate-email/references/9-report.md
#	scripts/lib/tests/migrate-email-skill.test.js
The count-only grep counted a key that an env loader reads as empty: a
value that is only a comment (= #phs_x), and a key overridden by a later
empty line. The check now reads only the last line that sets the
variable and needs its value to start with phs_, still printing only a
count. The test runs the documented command against fixture env files
in place of matching its wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Silthus and others added 17 commits November 13, 2026 14:57
…plate rule per provider

Resend also drops the text part for an empty text next to react, and for
a text the code builds that can be empty. The shared template rule now
keeps any template whose provider sent its emails with no text part, as
that provider's mapping says.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pty Text as left out

Step 3 reads only the Check part of the template file, so the rule that
keeps a template whose provider sent no text part moves there. Resend's
Go SDK leaves an empty Text out of the request, so Resend made that text
part from the HTML, and such a site moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ain text off

A dashboard copy shows no plain text both for a template without one and
for one whose plain text is an empty string. Ask which, or read the
template from the API; when nobody can tell, the site stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion and keep the answer

A dashboard copy has no key to read the template from the API, so the
question offers to make one. The answer goes into the template's plan
note, so a copy made again in Step 7 asks nothing new.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nly in Step 3

The template's note records its plain text however the template was
read, so Step 7 never asks again, and the key is offered only in Step 3,
before Step 5 asks the person to delete it. A site stays whenever
neither the answer nor the API read settles it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PostHog's bulk add takes only bounce, complaint or manual, and writes its own
reason text, such as "Imported (bounced)". The removal guard now keys on that
prefix, and the import maps each Resend origin to the reason value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…them

PostHog's list records no source an app could check before a removal, a bulk
add skips an address removed earlier, and an app can't call the import tool.
PostHog blocks a transactional send only to an address that hard-bounced, and
records that itself. So each such call becomes a "Before cutover" item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PostHog blocks a transactional send only to an address that hard-bounced, so
the weekly re-import limits mail to bounced addresses, not complaints. A call
that removes an address becomes an "After cutover" item the person acts on
when a moved send fails with recipient_suppressed. A call that adds one names
the action before the deploy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-import only now names bounces alone, and an app call that adds an address
also records it where the moved sends can skip it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A moved send reaches every address an app's add call put on the provider's
list, and an app-side flag misses past adds and survives a reactivation. So
Step 3 asks once which sites rely on the call, and keeps them on the provider.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 3 asks which sites rely on the provider's list for addresses added by
hand or by the app, and a non-answer keeps every site when the app adds
addresses itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the list

The suppression-list question gets its own Step 3 section, asks only about
sites that would move otherwise, keeps them on a non-answer, and says what
would let them move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e work

The question now comes right after the field checks, so a site it keeps
needs no template copy or PostHog template, and the plan note carries what
would let the site move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
posthog-rs's async email send waits between retries with tokio::time::sleep
and sends through reqwest, so a site on async-std or smol stays on Resend.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main equals #451's merged head a090b5b, which this branch already contains, so the tree is unchanged from f98eb3b.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Silthus
Silthus changed the base branch from feat/migrate-email-all-sdks to main November 13, 2026 16:26
Silthus and others added 5 commits November 13, 2026 16:27
Map each Loops suppression source to PostHog's reason value, import an
API-checked address as a bounce so moved sends skip it as Loops did,
tell Step 3's new question that Loops has no way to add an address, and
drop the "every removal counts" text Step 5 no longer reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…moved sends

Addresses from the API check are imported as bounces, so the plan notes
it under the suppression phase and the report qualifies its note that a
complained address gets the moved sends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n't be told apart

A section that only the second preview shows, such as not_contains,
can hold more than the preview's value, so a value shown there keeps
the template on Loops, as a contains section already did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed 1f5bd4d: 1 Should fix.

This is review round 7 on this pull request. When findings keep coming back in the same code, the approach is usually the problem. Before the next fix, check whether a different design would remove the whole class of findings.

Silthus and others added 2 commits November 13, 2026 16:55
…og's retry delay

A Loops retry loop that waited for Retry-After now reads the retry
delay PostHog's error carries since #472, converted to the loop's unit,
and keeps its own path only when PostHog named no wait.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed ba82a36: 1 Should fix.

This is review round 8 on this pull request. When findings keep coming back in the same code, the approach is usually the problem. Before the next fix, check whether a different design would remove the whole class of findings.

…Hog's doesn't

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed ff8ea2b: no findings.

@Silthus
Silthus marked this pull request as draft November 14, 2026 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant