Map: AI decisions in workflows, a decision step backed by Jev #240

Destination

Redrawn 2026-10-05 by Michael: one minimal draft PR. Revive PostHog/posthog#107832 (Harley Alexander's "Classify with Jev") on current master as a flag-gated function step, "AI decision (Jeeeeeeeeev)". It asks the PostHog-hosted Jev model on the AI gateway to pick one of the author's options and writes the answer and its confidence to workflow variables. The author branches on them with the regular conditional branch. Everything beyond that is a later improvement.

Notes

• Parked (2026-11-14). Off the focus of the Workflows sprint of 16 to 27 Nov (Sprint - November 16 to November 27 PostHog/posthog#133091). Dispatch no new ticket; running lanes finish and settle. Tripwire: the sprint's focus items done, or 2026-11-25.
  Execution. This map carries execution: resolved specs publish implement tickets here, and shipped means a draft PR that qa-swarm handed off as ready for review. No merge authority: Michael reads, marks ready, and merges.

Grilling (Michael, 2026-10-05). Open decisions go to Michael through grilling tickets, per the Wayfinder default. The earlier autonomy waiver is withdrawn.

Public repo. These tickets are public. Slack thread contents, customer names, usage numbers, and internal metrics never go here. Slack context lives in a private note in Silthus/personal (workflows-jev-decisions.md), and tickets refer to it without quoting it.

PRs. Draft PRs on PostHog/posthog from branches on the Silthus/posthog fork, based on upstream/master (the fork's master is far behind). Fork CI is held as action_required, so the local gate stands in (see the fork-pr-ci-and-semgrep-gap memory).

Domain and prior art. Read before any ticket:

• products/workflows/CONTRIBUTING.md and products/workflows/skills/building-workflows/references/graph-schema.md; the executor lives in nodejs/src/cdp/services/hogflows/.
• products/ml_inference/backend/facade/ is the team-billed decisions facade (decide, decide_when_available, 16 options, 32 questions, is_ai_data_processing_approved, flag ml-inference-decisions). posthog/llm/system_one.py holds the wire types.
• The HogQL decide() / jev() function (posthog/hogql/transforms/prompt_jev.py) shows how a batch caller budgets, dedupes, and maps a gateway 402 to "out of AI credits".
• PostHog/posthog#107832 ("Classify with Jev" workflow step, closed by its author unmerged): a function template that calls a Django route through a scoped service JWT and writes jev_category / jev_confidence variables. Its reviews are useful evidence.
• Map LLM action for workflows #1 and spec Spec: Generate text — an LLM action node for workflows #22 ("Generate text", an LLM node for workflows): settled decisions on templating, billing, failure modes, and mocked test walks that a decision step should stay consistent with.

Model. PostHog-hosted models through the AI gateway only. TypeSafe (third party) never receives customer data, so it is not a path for this step.

Casting. Choose worker models per the model selection policy: GPT-6.1 Sol through codex:codex-rescue for research and bounded implementation; Opus 5.5 for routine UI; Fable 5.1 for UI design with real ambiguity. Provider bridges are retired casts.

Skills. /tdd, /writing-user-facing-copy, /writing-ui-components, /improving-drf-endpoints, /implementing-mcp-tools, /writing-tests, /writing-pr-descriptions, qa-swarm.

Decisions so far

• How a workflow step reaches a hosted System One model: Node CDP calls a workflows-owned Django route that calls the ml_inference facade (ai_product="workflows", privacy_mode), through a new decide queue parameter that reschedules on 429/503 rather than an inline call. Django owns flag, consent, credits, and token-bucket admission. A new scoped JWT purpose. No direct gateway call, no cross-person batching in v1.
• Which decisions lifecycle teams want, and how other tools offer them: v1 serves five acceptance examples (onboarding track, use-case label, pick one of three templates, yes/no fit gate with a threshold, reply triage). Competitors mostly write a value and split afterwards; only n8n asks and routes in one node. No vendor offers a confidence threshold or a per-person why, which Jev's probabilities make possible.
• What a decision sees, and how it stays safe: the author picks a Context list of named, templated fields sent as one JSON state, capped at 8 KiB (the hosted model's 8,192-token window) and failing loudly when over. The question stays author text only; a function template enforces that only in the UI, which favors a native step or a server-side read. 2 to 16 options, yes/no threshold default 50%, optional Unsure branch off by default, privacy_mode always on, AI approval checked per run.
• Decision step in the builder: shape and name: shape A, a native branching action type whose outputs are its answers (Yes/No, or one per option), an optional Unsure path, and an "If the decision fails" path; labeling via Output variables. Named "AI Decision (Jeeeeeeeeev)" by Michael, shown as "AI decision (Jeeeeeeeeev)" under Audience split.
• Workflows team asks and why the Classify with Jev step closed: the closure reason is unknown. Build on top of PostHog/posthog#107832 (branch workflows/jev-classify-step), reshaped to the native branching step, through the AI gateway's hosted Jev.
• Spec: decision step for workflows: published Spec doc: AI decision step for workflows, a native ai_decision step with a workflows-owned Django route and scoped JWT over ml_inference (privacy mode, credit pre-check, 8 KiB cap, token-bucket admission), reschedule on 429/503 inside the Node handler. Three implementation tickets: the Django base reviving feat(workflows): add a classify with jev workflow action PostHog/posthog#107832, then the Node runtime and the builder UI in parallel. Test with a person spends one decision; the mocked walk spends none.
• AI decision step: decide route and action type in Django, reviving #107832: shipped as draft PostHog/posthog#111612, qa-swarm clean on e6ecec8 after 20 rounds. feat(workflows): add a classify with jev workflow action PostHog/posthog#107832's commits carried with Harley Alexander's authorship.
• Minimal v1 redraw (Michael, 2026-10-05): the native-step stack grew to about 10,000 added lines across three PRs. v1 is now one PR in Harley's shape plus the regular conditional branch, sized like feat(workflows): add a classify with jev workflow action PostHog/posthog#107832. No cleverer path exists: every CDP-to-Django call needs its own scoped JWT, and calling the gateway from Node would put its relay key in the CDP.

Not yet specified

• Follow-ups from the AI decision step: decide route and action type in Django, reviving #107832 #248 build. A combined token budget for question plus options (at the spec's limits they can approach the model's 8,192-token window). A decision step whose edge loops back to itself bills one decision per pass. The shared token bucket warns on every Redis error. ml_inference opens a new HTTP client per decision. Template grandfathering ignores the action type for other non-function steps (pre-existing).
• Rollout checklist. Keep workflows-ai-decision off until the Node runtime deploys; provision WORKFLOW_AI_DECISION_JWT_SECRET for web and the CDP worker; confirm the gateway bills hosted System One calls before the credits copy ships.
• AI approval copy. The organization AI data processing switch speaks of third-party AI services, yet every Jev caller uses it as the gate. Whether a PostHog-hosted model needs different copy or a different gate.
• Gateway billing and limits. Whether the AI gateway bills hosted System One calls above zero cost and applies its own per-team rate limits lives in PostHog/ai-gateway, which agents here cannot read. The spec treats it as an assumption to confirm.
• Prompt management. Whether authors can version a decision's wording, as signals does with managed System One prompts.
• Observability. How authors see decisions after a run: run logs, per-branch counts, LLM analytics traces, a confidence histogram.
• Batch scale. What happens when a batch workflow sends thousands of people through the step at once: gateway rate limits, credit checks, and backpressure.
• Score questions. Whether a 1-to-N score is a third mode or waits for a caller.

Out of scope

• Free-text generation. Writing copy with an LLM is the "Generate text" node (spec Spec: Generate text — an LLM action node for workflows #22), a separate effort.
• TypeSafe. The third-party Jev API is approved for experiments without customer data only.
• Self-hosted PostHog. Without the AI gateway, the step stays hidden there.
• Training or fine-tuning models. This effort uses the models the gateway already serves.
• Native branching step (v2). Node runtime, builder UI, batch admission, and the Django base PostHog/posthog#111612 with its stacked drafts feat(workflows): run ai decision steps in the cdp PostHog/posthog#111685 and feat(workflows): build, configure and test ai decision steps in the builder PostHog/posthog#111686, all closed on 2026-10-05. Their branches feat/workflows-ai-decision-{route,runtime,builder} stay on the fork as the starting point for v2.