Skip to content

feat(workflows): sandbox sender integration row and ensure endpoint - #111368

Closed
Silthus wants to merge 7 commits into
PostHog:masterfrom
Silthus:feat/workflows-sandbox-sender-row
Closed

Silthus wants to merge 7 commits into
PostHog:masterfrom
Silthus:feat/workflows-sandbox-sender-row

Conversation

@Silthus

@Silthus Silthus commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Merge risk: medium (qa-swarm danger rubric)

  • Update, delete and verify gain guards for every email integration.
  • The new sender row starts verified, so other verified-sender queries must skip it.

Problem

  • A new Workflows team cannot send any email until it verifies its own domain, which can take up to 48 hours.
  • The planned fix is a PostHog-owned "sandbox sender" that every project gets behind the workflows-sandbox-sender flag. Design and slicing: Silthus/posthog#225.
  • This PR is the first slice: the per-project sandbox sender row, and a guarantee that tools built for team domains leave it alone. Nothing sends through it yet.

Changes

  • A new endpoint, POST /api/projects/{id}/integrations/email_sandbox_sender/, creates or refreshes the project's sandbox sender and returns it.
    • It answers 404 while the flag is off for the team, or while the sandbox settings are empty.
    • It is idempotent: one row per project, integration_id = posthog-sandbox, created_by = null.
    • The display name is " via PostHog", rebuilt on every call so an organization rename reaches it.
    • It emits workflows sandbox sender provisioned on first creation only.
    • It never calls SES. A new workflows facade function, ensure_sandbox_email_sender(team_id), writes the row directly instead of using create_native_integration.
  • The row is marked by config.provider = "sandbox", not by a boolean beside provider = "ses".
    • Tenant migration, the tenant-state sweep and the post_delete identity cleanup select provider = "ses" rows only, so they skip the row without a code change.
    • A boolean would need every one of those paths, and any future one, to opt out by hand.
  • Users cannot change the row: verify, edit and delete answer 400 "The sandbox sender is managed by PostHog."
    • The edit guard runs before body validation, so sending the row's own config back also gets this message.
  • Teams cannot add an own sender on the sandbox domain or any subdomain of it.
  • The Reputation tab's ISP breakdown no longer lists the sandbox domain.
  • New Django settings in posthog/settings/ses.py: WORKFLOWS_SANDBOX_SENDER_DOMAIN, WORKFLOWS_SANDBOX_SENDER_FROM_ADDRESS, SES_SANDBOX_TENANT_NAME. All default to empty.
  • Mechanical: the regenerated integrations API client and a disabled MCP tool entry for the new operation.

Note

Any project member can call the endpoint from the app. An API key, OAuth token or MCP client needs integration:write, because the call writes a row. The row grants nothing beyond the sandbox sending rules, which later slices enforce in the Node send path.

Warning

Keep workflows-sandbox-sender off until the Node send-path PR lands. The email worker has no sandbox provider case yet, so a send through the row would log success without sending.

  • Turning the flag off leaves existing sandbox rows in place, still read-only. A later UI slice hides them while the flag is off.

How did you test this code?

Test seams: Django API tests through the integrations endpoints (APIBaseTest, flag patched, boto3 SES client mocked and asserted never called), task and command tests for the lifecycle exclusions, and one Reputation endpoint test.

Test rationale:

  • products/workflows/backend/tests/api/test_sandbox_sender.py catches a second row per project, a stale display name, a row created while the flag is off or settings are empty, a verify, edit or delete that reaches the row, and an own sender on the sandbox domain. No existing integration test covers a PostHog-managed row.
  • products/workflows/backend/test/test_sandbox_sender_lifecycle.py catches tenant migration, the tenant-state sweep or the post_delete cleanup touching the sandbox row or SES.
    • These pass on the base tree with a directly inserted row, because the existing paths only touch provider = "ses" rows.
    • They guard the choice of sandbox as the provider value: switching to a flag on an ses row fails them.
  • A scope case in test_sandbox_sender.py catches a read-only API key provisioning the row. It fails with the action in the read list.
  • A new case in test_email_reputation.py catches the sandbox domain showing up as a withheld domain in the ISP breakdown. It fails on the base tree.
  • On the base tree, all new API cases fail for the right reasons: no endpoint, the sandbox domain accepted, and the domain listed in the breakdown.

One scripted pass through the real URL stack against the local dev database, with a throwaway organization named "Acme Rockets", removed afterwards. SES and the flag client were mocked:

Live request output
flag off: 404 The sandbox sender is not available for this project.
settings unset: 404
ensure x2: 200 200 same id: True
config: {'name': 'Acme Rockets via PostHog', 'email': 'hello@sandbox.example.com', 'domain': 'sandbox.example.com', 'provider': 'sandbox', 'verified': True} created_by: None
own sender on sandbox subdomain: 400 Email domain mail.sandbox.example.com belongs to the PostHog sandbox sender. Please use a domain you own.
rows: 1
provisioned events: 1
SES client calls: 0

Not checked:

  • The verify, edit and delete guards in that live pass. The dev user hit the existing management permission (403) first, so the API tests carry those cases.
  • The endpoint against a real SES account.
  • The frontend, which has no consumer yet.
  • Backend and other fork CI workflows await maintainer approval. The local gate supplies proof while those runs remain unchecked.
  • The optional client-library review workflow had a startup failure. Its later run on the same head skipped the fork, as configured.
Recovered red, green and local gate proof

Tests, mypy and strict preflight all ran on the final commit 72ded8d9180.
Repo-wide mypy and the strict pre-push gate ran on 72ded8d9180.

Base API and Reputation cases: 14 failed, 26 passed in 15.78s
Base lifecycle cases with a directly inserted sandbox row: 3 passed in 6.98s
Green API, Reputation, lifecycle and existing SES suites on `72ded8d9180`: 75 passed in 18.03s
Repo-wide mypy: Success: no issues found in 22547 source files
mypy exit=0
Strict pre-push preflight: failures=0, mode=strict
push exit=0
OpenAPI generation: openapi exit=0

The lifecycle baseline already passed because those tools only select SES rows. It is not red proof for changed behavior.
The unavailable endpoint, reserved-domain rejection and Reputation exclusion provide the failing baseline.

To repeat the green test command in the prepared development environment:

flock /tmp/heavy-check.lock .codex/with-flox uv run pytest --reuse-db -q \
  products/workflows/backend/tests/api/test_sandbox_sender.py \
  products/workflows/backend/test/test_sandbox_sender_lifecycle.py \
  products/workflows/backend/tests/api/test_email_reputation.py \
  products/workflows/backend/test/test_email_integration_ses.py \
  products/workflows/backend/test/test_ses_tenant_state.py

Expected result: 75 passed. The recorded run used an isolated test database.

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Release status

  • No feature flag controls this change
  • This change is behind a feature flag and is not available to users
  • This change makes a previously flagged feature available to everyone

Automatic notifications

  • Publish to changelog?

Docs update

None. The feature is flagged off and no existing doc covers it.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, claude-opus-5-5

GPT-6.1 Sol recovered the proof and completed the handoff. It made no code changes.

  • Built test-first from the ticket's seams. Skills invoked: /tdd, /writing-tests, /improving-drf-endpoints, /writing-code-comments, /reviewing-with-coderabbit, /writing-pr-descriptions.
  • CodeRabbit CLI pass: skipped, because the CLI is signed out on this host and no person was available to sign in. The bot review on this PR covers it.
  • QA swarm: four rounds with a GPT-6.1 Sol panel (Correctness, Security, Production, Design, Tests, Adversary) and an Opus 5.5 Fresh eyes lane. Fable 5.1 was rate limited, so Opus 5.5 ran that lane. Last clean round on 72ded8d.
    • Fixed: the edit guard order, the read-scope POST (also raised by CodeRabbit), the MCP tools entry, unused re-exports, the event-on-creation assertion, and test annotations.
    • Calls: the endpoint is a write action for tokens rather than a read action, and the facade contract carries the core Integration row, documented like WorkflowBatchJob.
    • Rejected: removing SES_SANDBOX_TENANT_NAME (ticket Closes #228 update docker compose file #230 asks for it here), moving the name and cascade cases to lower-level tests (private helper and different setup), and the CodeRabbit outside-diff note to exclude ordinary rows on the sandbox domain (no customer row can be verified on that domain, and creating one is now refused).
  • No duplicate: a search for "sandbox sender" over open PRs found nothing related.
  • The workflows sandbox sender provisioned event is sent from the view, not the facade, because it needs the requesting user.
  • Public artifact: no session material beyond the public tickets. Fixture addresses use example.com.

🤖 Generated with Claude Code

Silthus and others added 4 commits October 3, 2026 09:48
…lusions

Unfinished: the red/green proof runs and the OpenAPI regeneration were
interrupted by a devbox rebalance. Tests are written but not yet run green;
generated OpenAPI output is not regenerated. Continue test-first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ender endpoint

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@trunk-io

trunk-io Bot commented Oct 3, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@coderabbitai

coderabbitai Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
  • Review profile: QUIET
  • Plan: Enterprise
  • Run ID: c5961ddd-1295-47ee-b961-8e2d219cf4c7
📥 Commits

Reviewing files that changed from the base of the PR and between 1465e4c and 72ded8d.

📒 Files selected for processing (1)
  • posthog/api/integration.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 3 remain after this review.


📝 Walkthrough

Walkthrough

The changes add configuration and management rules for a managed workflow sandbox email sender. A project-level POST endpoint creates or refreshes its integration when configuration and the team feature flag are available. The changes exclude the sender from email reputation results and selected SES lifecycle operations.

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 72ded

The sandbox sender remains feature-gated, and no actionable issue is established for this change. It is mergeable after normal checks; keep the feature flag off until the send path is ready.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 72ded

The new provisioning capability is project-scoped, write-authorized and feature-gated. Users cannot directly modify the managed sender, and provisioning does not call SES. Remaining uncertainty concerns downstream consumers and recovery when worker notification fails after a configuration update.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The newly reachable mutation is bounded to the authorized project's managed integration row and its worker notification. Its sender domain and address are platform-configured values potentially shared across projects, but the checked provisioning path does not grant callers SES identity-management authority.

Security Findings and Attack Paths

  • inferred — The checked provisioning and mutation routes did not establish a cross-project or managed-sender takeover path: provisioning uses the current team and fixed configuration, while verify, edit and delete reject sandbox ownership. This conclusion does not establish downstream email-worker enforcement or complete security coverage.

Trust Boundaries and Controls

  • observed — The provisioning action is explicitly write-scoped and receives authentication, API-scope, access-control and team-membership checks. Project membership permits this narrow provisioning action without granting arbitrary sender configuration.
  • observed — Native email creation rejects the configured sandbox domain and its subdomains before provider provisioning. Both deployment settings and user-provided native email addresses are lowercased, so mixed-case environment configuration does not support the suspected casing bypass.

Resilience and Maintainability Implications

  • inferred — Configuration persistence and worker notification are separate steps. If Redis publication fails after save, an identical repeated ensure request skips notification because the database configuration already matches. Recovery therefore depends on worker reconciliation outside the checked service; no current security impact was established.

Hardening Proposals

  • proposed — Before enabling sandbox email delivery, establish retryable configuration reconciliation and verify that downstream workers enforce managed-provider ownership and rollout gating. Also validate that the configured from-address belongs to the reserved sandbox domain.
🚥 Pre-merge checks | ✅ 1
✅ Passed checks (1 passed)
Check name Status Explanation
Description check ✅ Passed The description covers the problem, user-visible changes, testing and rationale, release status, docs, and agent context. It also reports key limitations. The required sections are mostly complete.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
  • Review profile: QUIET
  • Plan: Enterprise
  • Run ID: 1c474561-211a-4b6c-8935-70b4be0c4d59
📥 Commits

Reviewing files that changed from the base of the PR and between 1c99b5f and 9c0ee89.

⛔ Files ignored due to path filters (1)
  • products/integrations/frontend/generated/api.ts is excluded by !**/generated/**
📒 Files selected for processing (11)
  • posthog/api/integration.py
  • posthog/models/integration/__init__.py
  • posthog/models/integration/email.py
  • posthog/settings/ses.py
  • products/workflows/backend/facade/api.py
  • products/workflows/backend/facade/contracts.py
  • products/workflows/backend/presentation/views/hog_flow.py
  • products/workflows/backend/services/sandbox_sender.py
  • products/workflows/backend/test/test_sandbox_sender_lifecycle.py
  • products/workflows/backend/tests/api/test_email_reputation.py
  • products/workflows/backend/tests/api/test_sandbox_sender.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 6 remain after this review.

Comment thread posthog/api/integration.py Resolved
…equire write scope

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

🟡 Minor · Exclude the sandbox domain, not only sandbox-provider rows. · hog_flow.py:2252-2260

products/workflows/backend/presentation/views/hog_flow.py:2252-2260
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Exclude the sandbox domain, not only sandbox-provider rows.

If a pre-existing verified ordinary integration uses the configured sandbox domain, the provider-only filter keeps that domain in _isp_domains. The endpoint can then include it in ISP metrics and shared or withheld domain results. Filter projected domains with is_sandbox_sender_domain, which also recognizes subdomains.

Suggested fix
 from posthog.models.integration import SANDBOX_EMAIL_PROVIDER, Integration
+from posthog.models.integration.email import is_sandbox_sender_domain

 ...
-            if domain
+            if domain and not is_sandbox_sender_domain(domain)

ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
  • Review profile: QUIET
  • Plan: Enterprise
  • Run ID: e174666e-d7b5-4464-b61d-d2ab86b801cc
📥 Commits

Reviewing files that changed from the base of the PR and between 9c0ee89 and 85c2d19.

📒 Files selected for processing (6)
  • posthog/api/integration.py
  • posthog/models/integration/__init__.py
  • posthog/models/integration/email.py
  • products/integrations/mcp/tools.yaml
  • products/workflows/backend/facade/contracts.py
  • products/workflows/backend/tests/api/test_sandbox_sender.py
💤 Files with no reviewable changes (2)
  • posthog/models/integration/init.py
  • posthog/models/integration/email.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 0 remain after this review.

Silthus and others added 2 commits October 3, 2026 18:09
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…quest_access comment

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@scheduled-actions-posthog

Copy link
Copy Markdown
Contributor

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the stale label – otherwise this will be closed in another week. If you want to permanently keep it open, use the waiting label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant