[Silthus]

Problem

Feature flags: workflows-email-domain-agent-setup. Default off; this handoff does not activate them.

• An agent using the PostHog MCP cannot set up a Workflows email sender: the create, verify and DNS tools are all disabled.
• Enabling them alone would not work. Personal API keys and MCP OAuth tokens get "This action does not support personal API key access" on verify, email update and both Domain Connect actions, because none of them is in the viewset's scope lists.
• The endpoints also declare no schemas, so generated tools would get untyped or wrong inputs.
• This is PR B of the email domain setup redesign. PR A (the wizard UI and a read-only status endpoint) ships in parallel, not stacked.

Warning

🟡 Merge risk: medium (merge-risk rubric)

• Requests to the four email sender and Domain Connect endpoints go through new serializers, with or without the flag. If they rejected a payload the web app sends, sender edits or Domain Connect links would break.
• Limits: token access, SES writes from tokens and the address check stay off until workflows-email-domain-agent-setup is on. The web app's payloads match the serializers and tests cover each endpoint. Provider changes already made persist after a revert.

Fork handoff for source #109703. Modernized head 8cbc65746ebc271925c464070c2bd165bb23cf19. This draft prepares review and recreation in the upstream repository.

Changes

Kind	Files	Lines
Production	9	+482 −76
Tests	14	+736 −4
Evals	6	+1164 −2
Docs	2	+138 −0
Generated	8	+902 −252
Total	39	+3422 −334

Start with posthog/api/integration.py and posthog/api/integration_domain_serializers.py.

• Agents can now take a domain from "add a sender" to "verified" over MCP, behind the workflows-email-domain-agent-setup flag. The backend checks the same flag, so turning it on enables the tools and the API access they need together:

Tool	Does
integrations-email-create	Creates the sender and registers its domain
integrations-email-verify-create	Returns the DNS records with per-record status, and verifies
integrations-domain-connect-check-retrieve	Says if the DNS host supports one-click setup
integrations-domain-connect-apply-url-create	Returns a URL the person approves at their DNS host
integrations-email-partial-update	Changes the display name or MAIL FROM subdomain

• Tool descriptions carry the flow: create, get records, then either hand the person the Domain Connect URL or add records with the agent's own DNS tools without a second _dmarc, then verify until success.
• A new setting-up-an-email-domain skill teaches that job, including which records to merge (SPF, existing DMARC) instead of duplicating.
• The skill relies on #109695: the Domain Connect URL leaves out the DMARC record when the domain already has one. It is merged, and this branch includes it.
• The skill and tools treat the MAIL FROM label as domain-wide. SES stores it on the domain identity, so a second sender reuses the existing label instead of moving it for every sender.
• Scoped API keys and MCP OAuth can call the four actions only while the flag is on for the caller. With it off, they still get "This action does not support personal API key access". The check reads (integration:read). Verify, apply-url and email update write (integration:write), because verify and apply-url call SES.
• Session verification, sender updates, discovery and apply-url keep existing access, including email and proxy links while the agent flag is off.
• The four actions now declare request, query and response schemas. URLs and success responses do not change.
• Validation errors keep their message text, but now carry an attr. context and provider_endpoint errors on apply-url can come back together instead of one at a time.
• The generic integrations-create stays disabled. It accepts an untyped config for 30+ kinds, including credentials. integrations-email-create reuses the same POST integrations/ route with kind: email injected and the typed email config as its only input.
• The MCP email tools default provider to ses. The API still requires it.
• The MCP Domain Connect tool only builds URLs for email senders. It always sends context: email, hides context and proxy_record_id, and requires integration_id. The API accepts reverse proxy context only for sessions. Scoped API keys and OAuth receive 403 before signing a proxy URL.
• email/verify and PATCH email on a non-email integration now return 400 instead of 500.
• Domain Connect apply-url now returns 404 for an unknown or non-email integration_id, and for an unknown proxy_record_id from a session. Scoped proxy requests receive 403. Before, a stale id returned 500, and a non-email id returned a "try again later" 400 and reached error tracking.
• With the flag on, PATCH email with a different address returns 400. With it off, it still returns 200 and keeps the old address. The check ignores case, so a sender stored with capitals before addresses were lowercased can still be edited.
• proxy_record_id is now a UUID field, so a malformed value returns a 400 with attr instead of a 500.
• A new sandboxed eval suite, eval_email_domain_setup, checks that an agent with the skill sets up a domain correctly. Details below.
• An MCP codegen fix: param_overrides.<field>.required now also works together with input_schema. Before, it was silently ignored.
• Mechanical: regenerated OpenAPI, frontend types, MCP Orval schemas and tool definitions.

Evals

The suite runs a real agent with the skill against the real email API over MCP. SES, DNS and Domain Connect are simulated per case, so each case decides which records already exist.

Group	Cases
Setup on a new domain	Domain Connect link; records the person adds by hand
Reuse what exists	Already verified sender, no re-create; second sender keeps the domain's MAIL FROM label
Keep existing DNS intact	Existing DMARC kept; existing SPF merged; MAIL FROM label taken by another tool, with no record handed over at the taken name
Errors to relay	Gmail address; domain used by another organization; member who cannot verify

• Scorers grade outcomes, not the order of tool calls: the senders in the project and their verified state, the exact apply URL, the record values, and bounded verify calls. LLM judges check the hand-off and error messages against a reference list of records.
• Every run uses new sending domains. PostHog refuses a domain another organization uses, and each case runs in its own organization in a kept database.
• The suite rejects --trials above 1 for the same reason. Trials of a case share its domain.
• The eval harness now turns on workflows-email-domain-agent-setup for its MCP server, so evals can reach the tools.

Note

Turn the flag on only after this PR has deployed to Django. The MCP server deploys separately, and until Django has the change every tool call returns 403. The flag must exist in PostHog before the tools appear; an unknown flag hides them and denies token access.

Nothing user-visible changes in the web app.

• workflows-email-domain-agent-setup: Default-off server guard for new token authorization; MCP tools use the same flag. Existing session apply-url access remains available.

• Scoped credentials only support email Domain Connect. Session email and proxy access does not depend on the agent flag.

How did you test this code?

• Current test seams: parameterized HTTP requests with PAK, OAuth and session credentials, both flag states, and a mocked signing boundary.

• Before: scoped proxy credentials reached signing, and flag-off sessions received 404. After: scoped proxy calls return 403; baseline sessions succeed.

• Local proof: agent-final-backend-green.log, agent-final-mcp-tests.log, agent-final-openapi-green.log, agent-final-typecheck.log and agent-final-preflight.log.

• 62 focused backend HTTP/domain/scorer/simulated-eval tests passed; domain-109703-gate-green.log.

• The session flag-off test now preserves baseline 200 behavior. PAK/OAuth proxy tests failed before the fix and pass with 403.

• 268 MCP generator/apply-url/tool-filtering tests and MCP TypeScript passed; domain-109703-mcp-tests.log and domain-109703-typecheck.log.

• Full OpenAPI regeneration and skills lint passed; domain-109703-openapi.log and domain-109703-skills.log.

• Strict ci:preflight passed; domain-109703-preflight.log.

Prior validation retained from the original PR (2026-10-06)

Test rationale:

• test_email_domain_actions_with_scoped_api_key (12 parameterized cases, posthog/api/test/test_integration.py): fails if an action drops out of, or moves between, the scope lists, if a member's read key loses the check, or if a response body changes shape. Removing email_verify from the write list fails 2 cases, and removing the kind guard fails the non-email case. The apply-url case also guards that the endpoint validates through the new serializer. It extends the existing API key test class; no existing test called these actions with a key. The apply-url cases guard 404 for unknown or non-email integrations, and 403 for scoped proxy access. One case guards the 400 on an address change. The cases run with the flag on.
• test_email_domain_actions_reject_api_keys_while_the_agent_setup_flag_is_off (4 cases): fails if any of the four actions lets a write-scoped key through with the flag off.
• test_email_update_from_the_app_keeps_the_sender_address (2 cases): fails if editing a sender stored as Hello@… returns 400, which the first version of the address check did. It also fails if the address check runs with the flag off.
• test_integration_domain_serializers.py (SimpleTestCase, no database): per-context required fields on apply-url, a malformed proxy_record_id, and maildev rejected outside DEBUG. The DEBUG check moved from inline choices into validate_provider, so it needed its own guard.
• generate-tools.test.ts: the existing required override test became a parameterized case that also covers input_schema.
• products/workflows/evals/test/test_simulated_email_domains.py: runs the real SES provider and Domain Connect code on the simulation. It fails if verify stops following published records, if the claimed-domain guard stops firing, or if a case can no longer reach an apply URL through the API. products/workflows/package.json now includes evals/test/ in backend:test, so CI runs it.
• products/workflows/evals/test/test_scorers.py: the SPF scorer rejects two separate records and an include after all. The record value scorer rejects a case-changed or truncated TXT token, because TXT values are case-sensitive.
• integrations-domain-connect-apply-url.test.ts (MCP unit test): fails if the tool forwards context: proxy or a proxy_record_id an agent passes.
• test_sandboxed_harness_services.py: the existing override test now also asserts the email domain flag.

Ran locally, on an isolated test database: posthog/api/test/test_integration.py in full, the serializer, Domain Connect and eval tests, the eval harness tests, the MCP unit suite, and repo-wide mypy.
Not run: the eval suite itself, because this devbox has no BRAINTRUST_API_KEY or LLM_GATEWAY_ANTHROPIC_API_KEY. Also not run: a live MCP session against a running stack, real SES, or a real Domain Connect approval.
To run the suite: flox activate -- bash -c "hogli evals eval_email_domain_setup".

This PR comes from a fork, so CI needs a maintainer to approve the runs.

Release status

Fork handoff only. This task does not activate rollout flags. Deployment flag state was not inspected. Upstream CI and the unavailable fresh Opus review remain handoff gaps.

• This change is behind a feature flag and is not available to users

Docs update

Updated the existing OAuth development guide with email-only Domain Connect access for scoped credentials and baseline session access.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5 (claude-opus-5-5[1m])

• Started by Michael's orchestration thread as "PR B" of a settled design for the email domain redesign. PR A owns the wizard and the GET email/status endpoint; whoever lands second reruns hogli build:openapi.
• Skills invoked: /improving-drf-endpoints, /implementing-mcp-tools, /writing-tests, /writing-skills, /writing-evals, /running-ci-preflight, /reviewing-with-coderabbit, /writing-pr-descriptions, /tdd, /qa-swarm.
• Codex review (GPT-6.1 Sol) found 3 issues, all fixed: the create tool still required kind, the update tool left config optional, and the skill reused a sender with a different address on the same domain.
• Adversarial reviews by Fable 5.1 and GPT-6.1 Sol found 8 issues, all fixed: Domain Connect duplicating DMARC (now handled by fix(workflows): keep an existing dmarc record when auto-configuring email dns PostHog/posthog#109695), the per-sender MAIL FROM assumption, re-create resetting verification, SPF records that verify never checks, the @ hostname missing from the help text, wrong permission levels in the skill, a 500 on non-email ids, and a test that only ran as admin and never read bodies.
• Michael asked for evals of the skill. Adversarial reviews of the eval suite by Fable 5.1 and GPT-6.1 Sol found 11 distinct issues, 10 fixed: domains shared across runs and trials, scorers that passed an unverified sender, a shortened apply URL, incomplete records or an SPF include after all, a substring check that failed a correct DMARC answer, a shared discovery cache, an avoided-tool check on a tool the agent cannot see, a judge that penalized the SPF reminder the skill asks for, a member case that never checked the created sender, and DNS state the prompts describe but nobody seeded. Not changed: scorer tests under evals/test/, which follows the notebooks product.
• Merged master twice to clear staleness advisories. The second regen changed only proxy_record_id to a UUID in the generated zod schema.
• A second QA round (six GPT-6.1 Sol lenses and Fable 5.1) found 9 distinct issues. Fixed: the apply-url errors above, a scorer that ignored case in TXT values, a private scorer entry point in tests, the record type label that left out SPF, and three skill gaps (a DMARC lookup timeout read as a missing record, no source for the sender name on update, a member continuing after a 403). Rejected: per-team tenant scoping in the SES simulation, which is documented and exercised on purpose.
• A third QA round found 2 more, both fixed: the silent address change on update, and a missing check on the records handed over in the taken MAIL FROM label case.
• A fourth QA round found a type-annotation gap, fixed, and one open question: adding apply-url to the write scope also opens its reverse proxy branch to integration:write keys. That earlier decision is superseded: the current fix rejects scoped proxy credentials and retains baseline session proxy access.
• CodeRabbit's security review called this PR moderate risk. Its DMARC finding was already fixed by fix(workflows): keep an existing dmarc record when auto-configuring email dns PostHog/posthog#109695, and its MAIL FROM finding is fix(workflows): keep one mail from label per email sending domain PostHog/posthog#111367. Michael then asked for the backend behind the flag too, so the PR can merge with no behavior change for API callers. Checking the risk found the mixed-case regression in the address check.
• A Codex (GPT-6.1 Sol) adversarial review of the flag gate found one issue. Rejected: it reported the DMARC lookup in apply-url as new flag-off behavior, but that code is fix(workflows): keep an existing dmarc record when auto-configuring email dns PostHog/posthog#109695, already on master. Its other point, that the request schemas are not flagged, is by design and listed under Release status.
• CodeRabbit CLI: skipped, signed out with no person available during the run.
• Duplicate search: no open PR covers this.
• Follow-up: once PR A's status endpoint lands, the skill should poll it instead of verify.
• Follow-up: #111367 keeps the domain's MAIL FROM label in step across senders. Until it lands, changing the label on one sender gets undone when another sender on the domain verifies. Michael asked for it to go behind the same flag, so turning the flag on enables both. Keep the flag off until both have deployed. The address check now runs in the API view, so this PR no longer edits update_native_integration.
• Follow-up: #111366 applies MCP exclude_params per tool. Today integrations-email-create removes kind from the shared create body, which only matters if the generic integrations-create tool is enabled.
• Nothing in this PR comes from customer material.

Modernization used Codex, gpt-6.1-sol. Skills: modernize-pr, pr, writing-pr-descriptions, writing-voice, merge-risk, unslop. The source replays onto master at b75567aca6cce9f0499e882463fc7e71d65a064c. Original authorship and prior evidence remain above. The required current Opus review could not run because the Claude API rate limit blocked it.

Current review fixes and final upstream replay used Codex, gpt-6.1-sol. The fresh cross-family review remains blocked by the known Claude rate limit.

[github-actions]

🤖 CI report

ℹ️ MCP agent API — agent-facing tool changes

What this PR changes for agents, from the tool schema snapshots and tool definitions.

Tools added (5): integrations-domain-connect-apply-url-create (604 chars), integrations-domain-connect-check-retrieve (208 chars), integrations-email-create (1,073 chars), integrations-email-partial-update (1,170 chars), integrations-email-verify-create (142 chars)