Skip to content

feat(workflows): complete quick start after any delivered test email - #112299

Closed
Silthus wants to merge 9 commits into
PostHog:masterfrom
Silthus:feat/workflows-first-run-test-send-task
Closed

Silthus wants to merge 9 commits into
PostHog:masterfrom
Silthus:feat/workflows-first-run-test-send-task

Conversation

@Silthus

@Silthus Silthus commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Workflows users can receive a test email from a template, broadcast, or workflow email step while "Send yourself a test email" stays incomplete.

Stacked on #112068. Tracks Silthus/posthog#290.

Changes

Storybook, before and after the delivered test-send result, with fixture data only.

Before send, 520px After delivered result, 520px
Incomplete quick-start task at 520px Completed quick-start task at 520px
1280px screenshots

Before send at 1280px

After delivered result at 1280px

Before:

flowchart LR
    A[First-run delivered email] --> B[Quick-start task complete]
    C[Other delivered test emails] --> D[Task stays incomplete]
    classDef phBlue fill:#1d4aff,stroke:#1d4aff,color:#fff;
    classDef phYellow fill:#f9bd2b,stroke:#f9bd2b,color:#000;
    class A,C phBlue;
    class B,D phYellow;
Loading

After:

flowchart LR
    A[Any real delivered test email] --> B[Shared completion check] --> C[Quick-start task complete]
    D[Skipped, failed, or mocked send] --> E[Task stays incomplete]
    classDef phBlue fill:#1d4aff,stroke:#1d4aff,color:#fff;
    classDef phYellow fill:#f9bd2b,stroke:#f9bd2b,color:#000;
    class A,B,D phBlue;
    class C,E phYellow;
Loading

How did you test this code?

Red/green and local gate evidence. The touched Jest suites, full frontend typecheck, lint, and format check pass. The full pre-push hook passes.

Test rationale: The HTTP-mocked send logics verify delivered-only completion and persisted onboarding tasks for each send path. Existing first-run tests verify helper reuse. Existing WorkflowsFirstRunTab tests cover gallery/list selection. The gallery highlight stays intact.

Storybook screenshots exercise the template send logic with an HTTP-mocked success response. No live email provider delivery was exercised.

Release status

  • No feature flag controls this change
  • This change is behind a feature flag and is not available to users
  • This change makes a previously flagged feature available to everyone

Automatic notifications

  • Publish to changelog?

Docs update

No existing document covers this first-run setup task. This description records the behavior.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: T3 Code, Codex harness, GPT-6.1 Sol (gpt-6.1-sol, high reasoning effort).

Skills: tdd, writing-tests, writing-kea-logics, writing-ui-components, adopting-generated-api-types, placing-product-frontend-code, writing-user-facing-copy, setting-feature-flags-in-storybook, hogli, writing-pr-descriptions, running-ci-preflight, merge-risk, share-preview, and unslop.

The duplicate search found #112068, which completes only the first-run send. This increment wires the other existing send paths. All test addresses use example.com fixtures. CodeRabbit CLI is signed out, so its local pass was skipped. Merge risk is low; no sender or recipient rules changed. The read-only Claude Opus 5.5 review found one defect: lost gallery guidance. Commit dd8707653eb19 restores it; the same review session confirmed the fix. No findings remain.

The fork-master staleness advisory concerns fork-only agent docs. The specified stack parent stays unchanged; inherited OpenAPI/Python advisories do not originate in this frontend increment.

Assignment to the verified handle Silthus failed because the upstream API denied the assignee update. The PostHog asset uploader also denied access; screenshots use the authorized fork asset branch.

@trunk-io

trunk-io Bot commented Oct 6, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@coderabbitai

coderabbitai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Important

Review skipped

We couldn't safely recover the incremental review. No full review was started, and the last reviewed checkpoint was preserved. Retry later, or explicitly request a full review by commenting @coderabbitai full review.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The pull request adds a managed sandbox email sender with feature-flagged provisioning, organization-member recipient checks, sandbox-specific SES delivery, and workflow UI support. It also adds a feature-flagged workflow first-run gallery with template start metadata, email editing, test sending, workflow creation, and setup-task completion. Tests and Storybook stories cover the new flows.

Priority: ➖ Normal

Merge Risk: 🔵 Low · up to dd870

The sandbox sender and first-run onboarding are behind feature flags and are mostly well guarded. Two issues remain. A partially configured sandbox sender produces failed sends rather than a clean skip. One gallery template describes a start condition that differs from its actual trigger. Both are bounded and simple to fix.

Security Architecture Review

Security architecture risk: 🟡 Moderate · up to dd870

Server-side recipient checks and fixed sender identity substantially constrain sandbox email use. The main risk is enabling the shared sender before independent volume and pause controls are available: one project's permitted traffic could affect other projects. Sending is disabled by default, which limits immediate exposure.

Retained concerns

  • Medium · security · inferred: Enabling the new sandbox implementation exempts its traffic from project sending budgets while routing projects through one configured sandbox SES tenant. Independent sandbox caps and shared-tenant pause enforcement remain documented rollout prerequisites rather than controls demonstrated by the inspected send path. A project repeatedly sending to permitted member addresses could consume common quota or damage shared sandbox reputation, affecting other sandbox projects. The default-off switch and recipient, suspension and suppression checks reduce exposure; this is an activation risk, not evidence of an active deployed exploit.
Security review details

Security Blast Radius

  • inferred — Recipient exposure is constrained to eligible addresses in the invocation team's organization, but operational containment is broader: enabled sandbox projects share one configured sender and SES tenant. Excess permitted traffic can therefore affect shared sandbox quota or reputation without bypassing recipient authorization.

Security Findings and Attack Paths

  • inferred — The retained architecture concern is conditional on activation: an authorized project caller can repeatedly request real sends to eligible member addresses using the new managed identity, while sandbox sends bypass the ordinary project budget. Test sends also execute outside the email queue. The inspected implementation does not demonstrate the documented independent sandbox containment gates; external deployment controls remain unverified.

Trust Boundaries and Controls

  • observed — The frontend recipient picker is not the security boundary. The API checks project access, the worker binds integration ownership to the invocation team, and membership enforcement applies to test and normal sends. Managed sender mutation is rejected, while sandbox delivery overrides custom sender identity and removes Reply-To. These controls counter the supplied frontend authorization-bypass candidate.

Resilience and Maintainability Implications

  • observed — Sandbox SES throttling preserves the original email payload for retry and does not emit a premature successful-send result. Re-entry into EmailService repeats recipient authorization. Project suspension and recipient suppression also precede submission, providing fail-closed controls distinct from the unresolved sandbox volume-containment prerequisite.

Hardening Proposals

  • proposed — Keep sandbox delivery disabled until independent recipient-counted abuse budgets and shared-sandbox pause enforcement are demonstrated for both queued and in-process test sends. Make activation contingent on those controls and validated SES configuration, rather than relying solely on a deployment instruction.
🚥 Pre-merge checks | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is well structured and explains the test-email completion change, but it does not match the broader changes in the supplied file summaries. Those summaries include sandbox sender provi… Clarify which changes belong to this PR versus its stacked parent. If the sandbox sender and first-run features are part of this PR, describe their user-visible behavior and testing in Problem, Changes, and How did you test this code?, and …
Full details: Description check

Explanation

The description is well structured and explains the test-email completion change, but it does not match the broader changes in the supplied file summaries. Those summaries include sandbox sender provisioning and delivery restrictions, plus first-run workflow setup and gallery features. The statement that sender or recipient rules did not change conflicts with those summaries. The release-status section also keeps all three checkbox options instead of using one plain-bullet line.

Resolution

Clarify which changes belong to this PR versus its stacked parent. If the sandbox sender and first-run features are part of this PR, describe their user-visible behavior and testing in Problem, Changes, and How did you test this code?, and correct the claim that sender or recipient rules did not change. If they are inherited from the parent, state the exact scope of this PR. Replace the three checkbox lines under Release status with exactly one plain-bullet line.

✨ Finishing Touches 💡 1
⚔️ Resolve merge conflicts 💡
  • Resolve merge conflict in branch feat/workflows-first-run-test-send-task
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments (3)
products/workflows/backend/templates/onboarding_started_but_not_completed_template.json-4-8 (1)

4-8: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Trigger URL filter and starts_on detail disagree.

starts_on.detail says on /onboarding. The trigger filter at lines 35-43 matches $current_url with icontains on messaging/campaigns. The gallery will show a start condition that the workflow does not use.

Align the two. Either change detail to describe the actual filter, or change the filter value to /onboarding.

products/workflows/frontend/firstRun/WorkflowsFirstRunMakeItYours.stories.tsx-83-88 (1)

83-88: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Restore the global access level after the ViewerAccess story.

The ViewerAccess render function changes the shared window.POSTHOG_APP_CONTEXT object. It sets resource_access_control.hog_flow to 'viewer' and never sets it back. If another story renders after this one in the same runtime, that story can show viewer-disabled buttons in its snapshot. Put this state in a small wrapper that restores the previous value on unmount. As an alternative, use a story parameter or decorator that sets the app context for this story only.

Proposed fix
 export const ViewerAccess: Story = {
     render: function ViewerAccess(): JSX.Element {
-        ;(window as any).POSTHOG_APP_CONTEXT.resource_access_control.hog_flow = 'viewer'
+        const accessControl = (window as any).POSTHOG_APP_CONTEXT.resource_access_control
+        const previous = accessControl.hog_flow
+        accessControl.hog_flow = 'viewer'
+        useEffect(() => () => {
+            accessControl.hog_flow = previous
+        }, [])
         return <App />
     },
 }
nodejs/src/cdp/cdp-services.ts-464-472 (1)

464-472: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Turn the sandbox sender off when its routing config is incomplete.

enabled comes only from WORKFLOWS_SANDBOX_SENDER_ENABLED. The tenant name, configuration set and From address all default to ''. If the flag is on and any of these is unset, EmailService passes the gate at isSandbox && !this.sandboxSender?.config.enabled:

  • resolveFromSender returns email: ''.
  • sesRouteFor returns an empty TenantName or ConfigurationSetName.

Each sandbox send then reaches SES and fails as email_failed. It is not the clean switch_off skip.

The Node From address (SES_SANDBOX_FROM_ADDRESS) is also separate from the Django WORKFLOWS_SANDBOX_SENDER_FROM_ADDRESS stored in the integration row. If the two drift, the UI shows one address and the worker sends from another.

Derive enabled from all four values. Also log the misconfiguration at startup.

Proposed fix
         new SandboxEmailSender(
             {
-                enabled: config.WORKFLOWS_SANDBOX_SENDER_ENABLED,
+                enabled:
+                    config.WORKFLOWS_SANDBOX_SENDER_ENABLED &&
+                    !!config.SES_SANDBOX_TENANT_NAME &&
+                    !!config.SES_SANDBOX_CONFIGURATION_SET &&
+                    !!config.SES_SANDBOX_FROM_ADDRESS,
                 tenantName: config.SES_SANDBOX_TENANT_NAME,
🧹 Nitpick comments (4)
frontend/src/lib/components/ProductSetup/productSetupLogic.test.ts (1)

180-180: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Story and test guidelines forbid featureFlagLogic.actions.setFeatureFlags in stories only. Flag state leaks across tests here.

The test sets WORKFLOWS_FIRST_RUN through featureFlagLogic.actions.setFeatureFlags. The featureFlags reducer is persisted (persist: true). The test never resets it. The flag value from the last it.each case (false) stays in the reducer for later tests in the file. It also persists to local storage in jsdom. Reset the flags in the finally block.

Proposed fix
         } finally {
+            featureFlagLogic.actions.setFeatureFlags([], {})
             workflowsLogic.unmount()
         }
products/workflows/package.json (1)

22-22: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Pin @testing-library/dom to a version range instead of *.

* accepts any version. The root frontend/package.json declares >=9.3.4. Use the same range to keep the workspace resolution stable.

Proposed fix
-        "@testing-library/dom": "*",
+        "@testing-library/dom": ">=9.3.4",
products/workflows/backend/tests/api/test_hog_flow_template.py (2)

22-63: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Parameterize the per-template checks.

The subTest loop hides which template failed from the runner. The path instructions prefer parameterised tests over near-duplicate test functions. Use parameterized.expand over template names, or keep the loop if you want one DB-free test.

Source: Path instructions


54-61: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

The trigger check is vacuous for templates with empty trigger.filters.

Templates such as celebrate-milestone.json and trial_started_upgrade_nudge_template.json have "filters": {} on the trigger. For them, the loop asserts nothing. Their declared events (milestone_reached, trial_started) are never compared with a trigger. Assert that, for kind == "event", the trigger either has no event filters or has a first event that matches starts_on["events"][0]. Then confirm these templates are meant to trigger on any event.


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
  • Review profile: QUIET
  • Plan: Enterprise
  • Run ID: b3f0b0dd-8f4a-41f4-b932-04c427ab40c8
📥 Commits

Reviewing files that changed from the base of the PR and between d68cb14 and dd87076.

⛔ Files ignored due to path filters (4)
  • frontend/src/generated/core/api.schemas.ts is excluded by !**/generated/**
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
  • products/integrations/frontend/generated/api.ts is excluded by !**/generated/**
  • products/workflows/frontend/generated/api.schemas.ts is excluded by !**/generated/**
📒 Files selected for processing (112)
  • docs/internal/workflows-email-sending-tiers.md
  • frontend/src/lib/components/CyclotronJob/CyclotronJobInputs.tsx
  • frontend/src/lib/components/ProductSetup/ProductSetupPopover.tsx
  • frontend/src/lib/components/ProductSetup/productSetupLogic.test.ts
  • frontend/src/lib/components/ProductSetup/productSetupLogic.ts
  • frontend/src/lib/components/ProductSetup/productSetupRegistry.ts
  • frontend/src/lib/components/ProductSetup/types.ts
  • frontend/src/lib/constants.tsx
  • frontend/src/lib/integrations/IntegrationsList.tsx
  • frontend/src/lib/integrations/integrationsLogic.test.ts
  • frontend/src/lib/integrations/integrationsLogic.ts
  • frontend/src/lib/integrations/utils.ts
  • frontend/src/scenes/hog-functions/email-templater/EmailTemplater.test.tsx
  • frontend/src/scenes/hog-functions/email-templater/EmailTemplater.tsx
  • frontend/src/scenes/hog-functions/email-templater/NativeEmailIntegrationChoice.stories.tsx
  • frontend/src/scenes/hog-functions/email-templater/emailTemplaterLogic.test.ts
  • frontend/src/scenes/hog-functions/email-templater/emailTemplaterLogic.tsx
  • frontend/src/scenes/hog-functions/email-templater/selectSenders.test.ts
  • frontend/src/scenes/hog-functions/email-templater/selectSenders.ts
  • nodejs/package.json
  • nodejs/src/cdp/cdp-services.ts
  • nodejs/src/cdp/config.ts
  • nodejs/src/cdp/services/managers/organization-members.service.ts
  • nodejs/src/cdp/services/messaging/email.service.test.ts
  • nodejs/src/cdp/services/messaging/email.service.ts
  • nodejs/src/cdp/services/messaging/sandbox-email-sender.ts
  • nodejs/src/cdp/types.ts
  • nodejs/src/cdp/utils.ts
  • posthog/api/integration.py
  • posthog/cdp/test/test_validation.py
  • posthog/cdp/validation.py
  • posthog/models/integration/__init__.py
  • posthog/models/integration/email.py
  • posthog/models/team/setup_tasks.py
  • posthog/settings/ses.py
  • products/integrations/mcp/tools.yaml
  • products/workflows/backend/facade/api.py
  • products/workflows/backend/facade/contracts.py
  • products/workflows/backend/presentation/views/hog_flow.py
  • products/workflows/backend/presentation/views/hog_flow_template.py
  • products/workflows/backend/services/sandbox_sender.py
  • products/workflows/backend/templates/announce_a_new_feature_template.json
  • products/workflows/backend/templates/celebrate-milestone.json
  • products/workflows/backend/templates/feature-adoption-tips.json
  • products/workflows/backend/templates/heavy-usage-detected.json
  • products/workflows/backend/templates/high-intent-notification.json
  • products/workflows/backend/templates/negative-survey-response.json
  • products/workflows/backend/templates/onboarding_started_but_not_completed_template.json
  • products/workflows/backend/templates/re-engagement-workflow.json
  • products/workflows/backend/templates/repeated-failure-alert.json
  • products/workflows/backend/templates/support-ticket-notification.json
  • products/workflows/backend/templates/ticket-unresolved-alert.json
  • products/workflows/backend/templates/trial-ending-reminder.json
  • products/workflows/backend/templates/trial_started_upgrade_nudge_template.json
  • products/workflows/backend/templates/unlocking_advanced_features.json
  • products/workflows/backend/templates/unused-features-education.json
  • products/workflows/backend/templates/user-stuck-onboarding.json
  • products/workflows/backend/templates/user-upgrade-webhook.json
  • products/workflows/backend/templates/welcome_email_sequence_template.json
  • products/workflows/backend/test/test_sandbox_sender_lifecycle.py
  • products/workflows/backend/tests/api/test_email_reputation.py
  • products/workflows/backend/tests/api/test_hog_flow_template.py
  • products/workflows/backend/tests/api/test_sandbox_sender.py
  • products/workflows/backend/tests/api/test_sandbox_sender_validation.py
  • products/workflows/frontend/Broadcasts/broadcastTestSendLogic.ts
  • products/workflows/frontend/Broadcasts/steps/BroadcastContentStep.tsx
  • products/workflows/frontend/Channels/EmailSetup/emailSetupModalLogic.test.ts
  • products/workflows/frontend/Channels/EmailSetup/emailSetupModalLogic.ts
  • products/workflows/frontend/Channels/MessageChannels.stories.tsx
  • products/workflows/frontend/Channels/MessageChannels.tsx
  • products/workflows/frontend/Channels/SandboxEmailSenderRow.tsx
  • products/workflows/frontend/Channels/ownSenderQuickStartLogic.test.ts
  • products/workflows/frontend/Channels/ownSenderQuickStartLogic.ts
  • products/workflows/frontend/TemplateLibrary/SendTestEmailModal.stories.tsx
  • products/workflows/frontend/TemplateLibrary/SendTestEmailModal.test.tsx
  • products/workflows/frontend/TemplateLibrary/SendTestEmailModal.tsx
  • products/workflows/frontend/TemplateLibrary/messageTemplateTestSendLogic.test.ts
  • products/workflows/frontend/TemplateLibrary/messageTemplateTestSendLogic.ts
  • products/workflows/frontend/Workflows/hogflows/findTestSendSkipReason.test.ts
  • products/workflows/frontend/Workflows/hogflows/findTestSendSkipReason.ts
  • products/workflows/frontend/Workflows/hogflows/panel/testing/hogFlowEditorNotificationTestLogic.ts
  • products/workflows/frontend/Workflows/hogflows/steps/components/HogFlowFunctionConfiguration.tsx
  • products/workflows/frontend/Workflows/templates/WorkflowTemplateCard.tsx
  • products/workflows/frontend/WorkflowsScene.tsx
  • products/workflows/frontend/completeTestEmailSetupTask.ts
  • products/workflows/frontend/emptyState/workflowsEmptyState.tsx
  • products/workflows/frontend/firstRun/GalleryTemplateCard.tsx
  • products/workflows/frontend/firstRun/MakeItYoursActions.tsx
  • products/workflows/frontend/firstRun/MakeItYoursEmailList.tsx
  • products/workflows/frontend/firstRun/TemplateStartsOnLine.tsx
  • products/workflows/frontend/firstRun/WorkflowsFirstRun.stories.tsx
  • products/workflows/frontend/firstRun/WorkflowsFirstRunGallery.tsx
  • products/workflows/frontend/firstRun/WorkflowsFirstRunMakeItYours.stories.tsx
  • products/workflows/frontend/firstRun/WorkflowsFirstRunMakeItYours.tsx
  • products/workflows/frontend/firstRun/WorkflowsFirstRunTab.test.tsx
  • products/workflows/frontend/firstRun/WorkflowsFirstRunTab.tsx
  • products/workflows/frontend/firstRun/buildFirstRunTestSend.ts
  • products/workflows/frontend/firstRun/buildFirstRunWorkflow.ts
  • products/workflows/frontend/firstRun/firstRunGalleryLogic.test.ts
  • products/workflows/frontend/firstRun/firstRunGalleryLogic.ts
  • products/workflows/frontend/firstRun/firstRunMakeItYoursLogic.test.ts
  • products/workflows/frontend/firstRun/firstRunMakeItYoursLogic.ts
  • products/workflows/frontend/firstRun/firstRunSenderLogic.test.ts
  • products/workflows/frontend/firstRun/firstRunSenderLogic.ts
  • products/workflows/frontend/firstRun/firstRunStoryFixtures.ts
  • products/workflows/frontend/firstRun/firstRunWorkflowStorage.ts
  • products/workflows/frontend/firstRun/templateEmails.ts
  • products/workflows/frontend/firstRun/withFirstRunSender.test.ts
  • products/workflows/frontend/firstRun/withFirstRunSender.ts
  • products/workflows/frontend/testEmailSetupTask.test.ts
  • products/workflows/package.json
  • services/mcp/src/api/generated.ts

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 6 remain after this review.

@Silthus

Silthus commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Note

🤖 GPT-6.1 Sol responding on behalf of Michael

Replaced by the draft fork PR #334 in its native fork stack. @meikelmosby will start migrating the stack from the fork into PostHog/posthog for upstream review and CI. Closing this master-targeted copy to keep the handoff clear.

@Silthus Silthus closed this Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant