Skip to content

feat(workflows): capture missing activation funnel steps - #357

Closed
Silthus wants to merge 1 commit into
masterfrom
feat/workflows-activation-events
Closed

Silthus wants to merge 1 commit into
masterfrom
feat/workflows-activation-events

Conversation

@Silthus

@Silthus Silthus commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

Problem

Feature flags: None control this increment. The behavior applies regardless of rollout flags.

  • The Workflows team cannot see where teams drop off between creating a workflow and delivering a first real message.
  • Several funnel steps fire no product analytics event: a delivered message, a send blocked by quota, and a send failing on an unverified domain or a recipient without an email address.
  • Product intent records only "New workflow" clicks. Workflows created from a cohort, Support, PostHog AI, MCP or the API go uncounted.

Warning

Merge risk: medium (qa-swarm danger rubric)

  • Every CDP consumer and the SES delivery path get a new reporter, and each call is fire-and-forget inside try/catch.
  • The server-side intent moves the first funnel step away from its history.

Fork handoff for source #111267. Modernized head a35f68f183863e4cf2fc9b3ae8766f216c657b42. This draft prepares review and recreation in the upstream repository.

Changes

Kind Files Lines
Production 16 +172 −34
Tests 11 +279 −20
Total 27 +451 −54
  • Every created workflow now registers the workflows product intent on the server, whatever channel created it.
    • HogFlowViewSet.perform_create calls ProductIntent.register, which emits user showed product intent with intent_context: workflow_created.
    • The "New workflow" button drops its client-side intent call, so a button-created workflow counts once.

Note

The intent now fires when a workflow is saved, not when someone clicks "New workflow". A click that never leads to a saved workflow no longer counts, so the first funnel step reads lower than its history.

  • The CDP send path now reports three new events to PostHog's own analytics, one per team and reason per hour per service instance.
    • A funnel needs only the first occurrence. One event per send would flood the stream.
    • Test sends from the editor never report.
    • Only workflow sends report. A hog function destination that sends email reports nothing.
    • A failed team lookup, a missing team, or a failed capture does not use up the hour, so the next send reports again.
    • A delivery still reports after its workflow is deleted, because the signed tracking code carries the workflow version.
  • A quota block reports only when the email quota stops a workflow with an email step.
    • A webhook-only workflow blocked by the destination quota does not count as a blocked message send.
    • Rejected alternative: report every block with a quota property. The funnel step is about messages, so a destination block would add noise.
  • Email domain verification is not in this PR. The open domain wizard stack reports email domain verified from #110516 onwards.
Event Fires when Properties
user showed product intent (existing) A workflow is created through any channel product_key: workflows, intent_context: workflow_created, is_first_intent_for_product
workflows message delivered SES reports an email delivered for a workflow workflow_id, channel: email
workflows send blocked The email quota drops a run of a workflow with an email step workflow_id, channel: email, reason: quota_limited
workflows send failed A workflow email send fails on an unverified sending domain, or the recipient has no email address workflow_id, channel: email, reason: unverified_domain or missing_recipient
  • The CDP events use captureTeamEvent: the distinct ID is the team UUID, with project and organization groups, so a funnel can run per team.
  • The rest of the Node change is mechanical: one WorkflowsActivationReporter wired through cdp-services, the consumers and CdpApi, as a required dependency of each reporting service.
  • Nothing a user sees changes. The button looks and behaves the same, so there are no screenshots.

How did you test this code?

Built test-first. Test seams:

  • HogFlow create API: test_create_hog_flow_registers_workflows_product_intent fails if creation stops registering the intent.
  • WorkflowsActivationReporter.report: catches a per-send event flood, dedupe that leaks across teams, and a crash on an unknown team. It also catches a team silenced for an hour after a failed lookup, a missing team, or a failed capture.
  • shouldBlockHogFlowDueToQuota: the existing block test now runs for an email-quota block, which reports, and a destination-quota block on a webhook, which does not.
  • EmailService.executeSendEmail: the existing unverified-domain test now runs for a workflow send, a workflow test send, and a hog function send. Only the first reports.
  • RecipientPreferencesService.shouldSkipAction: a missing recipient reports for marketing and transactional sends, not test sends.
    • The transactional case matters: those sends skip the opt-out lookup that throws.
  • SES webhook through CdpApi: a Delivery notification reports workflows message delivered for a workflow and for a deleted workflow, and nothing for a hog function.
  • HogFunctionHandler.execute: the existing recipient-preferences test now asserts that the test-send flag reaches the check.

Test rationale: each new assertion extends the nearest existing test, except the reporter, which is a new unit, and the delivery case, which no existing webhook test covered.

Verify the reporting end to end through the SES webhook, from nodejs/:

pnpm jest src/cdp/services/messaging/email-tracking.service.test.ts -t "activation step"

Expect 3 passing tests: a workflow delivery reports, a delivery after the workflow is deleted reports, and a hog function delivery does not.

Run locally: the touched Jest suites, the Python test above, the nodejs typecheck, ESLint and oxlint on the touched files, hogli ci:preflight --strict, and semgrep on the diff (0 findings).

Not run: the full nodejs ESLint pass and repo-wide mypy, which timed out on the shared devbox. No live CDP send or capture in a running stack.

Known gap, not fixed here: a test send whose SES notification lacks the tracking header falls back to a short tag with no test flag, so it counts as a real delivery. The bug predates this PR and affects the Metrics tab the same way.

Modernization on 5bd2fa405dfee: Preserves upstream cohort parsing and dead-letter reporting. Focused Node activation, invocation-pipeline, and email tests pass; workflow-create API cases and strict preflight pass. Fresh browser rendering remains unchecked. Historical proof above describes the earlier head. Fresh CI remains unverified.

Combined handoff proof

The 14-layer stack now descends from pinned upstream b75567aca6cce9f0499e882463fc7e71d65a064c. Fresh checks passed: 136 frontend behavior tests across 12 suites, 333 Node delivery/engagement tests across 6 suites with 7 snapshots, the backend committed-creation intent test, and 15 project-config/permission overlap tests. Frontend tsgo and normal Node tsc passed; all 14 strict increment gates passed. Node Jest required --forceExit for existing lingering handles. Upstream workflow step edits and MCP opt-in policy remain intact. The new replay consumer now receives its required activation reporter from the bottom layer. The final backend audience/template/sender/quota/unsubscribe and upstream-proposal overlap run passed 225 cases. Historical proof and screenshots above belong to their recorded heads. Fresh upstream CI, a new full mypy run, OpenAPI regeneration, a fresh Claude review, and combined browser rendering remain unchecked.

Release status

Fork handoff only. This task does not activate rollout flags. Upstream CI and the unavailable Opus review remain handoff gaps.

  • No feature flag controls this change

  • This change is behind a feature flag and is not available to users

  • This change makes a previously flagged feature available to everyone

  • Publish to changelog?

Docs update

None. These are internal analytics events.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5 (1M context)

  • Directed by Michael from the Workflows growth map, quick win Silthus/posthog#220.
  • Skills: /tdd, /writing-tests, /reviewing-with-coderabbit, /writing-pr-descriptions, /qa-swarm.
  • QA round 1 (a cross-model reviewer panel plus the CodeRabbit review) led to the second commit: workflow-only reporting, dedupe release on failure, the email-only quota report, and a required reporter dependency.
  • QA round 2 led to the third commit: deliveries from deleted workflows, event properties typed per event, and wider recovery tests.
  • QA rounds 3 and 4 changed tests and one comment position only. The Claude lane ran on Opus 5.5 in rounds 3 and 4, because Fable hit rate limits.
  • CodeRabbit CLI: skipped, the CLI was signed out and no person was available to sign in.
  • Duplicate search found no open PR for this.
  • No session material beyond this repository went into the diff or this description.

🤖 Generated with Claude Code

Modernization: T3 Code, Codex, gpt-6.1-sol. Skills: modernize-pr, pr, writing-pr-descriptions, writing-voice, unslop, running-ci-preflight. A fresh Claude Opus 5.5 review attempt hit an API rate limit; the modernization review remains incomplete.

Fork handoff: T3 Code, Codex, GPT-6.1 Sol (gpt-6.1-sol). Skills: pr, writing-pr-descriptions, writing-voice, unslop, stacking-prs, running-ci-preflight. A fresh Claude Opus 5.5 review remains unavailable. No rollout flags or preview labels were activated for this fork draft.

@Silthus Silthus self-assigned this Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant