Repository navigation
Description
Part of #240
Spec: #247 (Spec doc: AI decision step for workflows). Sections: "Answer selection", "The result and variables", "The Node runtime", "Test runs and the mocked walk", "Docs and MCP".
What to build
The CDP runs an ai_decision step end to end: it renders the Context, asks the decide route, picks the output from the probabilities and the author's thresholds, and reschedules the person when the route says "later". After this ticket, a decision step built through the API or MCP runs in a live workflow.
ai_decisionin the Node action schema (nodejs/src/cdp/schema/hogflow.ts), and anAiDecisionHandlerin the executor registry (hogflow-executor.service.ts).- A pure answer-selection function: yes or no with
yes_threshold, the Unsure band withno_threshold, pick one with the top probability andmin_pick_probability, ties to the earlier option. - A decision service that owns the call: render
inputs.contextwithbuildInputsWithGlobals, failstate_too_largeover 8,192 bytes of compact JSON without a call, mint theWORKFLOW_AI_DECISIONJWT (claimsteam_id,hog_flow_id), andPOSTto/api/projects/:team_id/workflow_ai_decisions/with a 10 s timeout. NocallInternalApi, no inline retry loop. - Map responses by the spec's status contract. On 429 and 503, return
scheduledAtwith the spec's reschedule bounds, carried in the action's state: 429 waitsRetry-Aftercapped at 60 s plus jitter, up to 30 minutes, thenthrottled; 503, a timeout, or a connection error waitsRetry-Afteror 2 to 32 s, up to 5 times, thengateway_unavailable. - A reschedule writes no "Resuming..." / "Workflow will pause until..." pair into the run log.
- Success returns
nextActionon the answer's branch edge and the result{answer, probability, probabilities, model}foroutput_variable. The run log names the answer, all probabilities, and the model, never the state. A failure returns an error, soon_errorsends the person down thecontinueedge ("If the decision fails") or ends the run. - Test runs: with
mock_async_functions, no call; validate and render like a live run, answer withmock_answeror the first answer, say it was mocked in the run log, and return the rendered Context and its size. Without mocks, one real call; 429 or 503 ends the test with a busy message instead of rescheduling. - Docs and MCP, per the spec: the
ai_decisionsection and worked example inproducts/workflows/skills/building-workflows/references/graph-schema.md, the node type and the credit clause plusmock_answerinproducts/workflows/mcp/tools.yaml(regenerate the MCP schemas), and a short section on the native handler inproducts/workflows/CONTRIBUTING.md. - Where feat(workflows): add a classify with jev workflow action PostHog/posthog#107832's
classify.tscode fits (token minting, input parsing, the mock and live parity cases), carry it and addCo-authored-by: Harley Alexander <harley.a@posthog.com>to that commit.
Acceptance criteria
- A live
ai_decisionstep whose stubbed route answersP(yes) = 0.62withyes_threshold50 follows the Yes edge, and writesanswer: "yes"to a mapped variable. - With Unsure on,
P(yes)betweenno_thresholdandyes_thresholdfollows the Unsure edge; a pick one whose top probability is undermin_pick_probabilityfollows the Unsure edge. - A terminal failure envelope follows the
continueedge withon_error: continue, and ends the run withon_error: abort; the run log names the code. - A 429 reschedules at
Retry-Afterand gives up withthrottledafter 30 minutes; a 503 reschedules at most 5 times, then fails withgateway_unavailable. Neither writes the resume log pair. - A Context over 8,192 bytes fails with
state_too_largeand makes no HTTP call. - The mocked and live paths reject the same configs the Django serializer rejects, from one shared cases table.
- A mocked test run with
mock_answerset to the second option follows that option's edge and spends nothing. graph-schema.mdandtools.yamldescribe the step, its edges, and its credit use; MCP schemas are regenerated.
Test seam
Spec seams 3, 4, and 5: the answer-selection function (it.each at threshold boundaries), the handler in the existing hogflow executor test harness with the route stubbed, and mock and live parity over one cases table.
Base branch
Stacks on #248's branch feat/workflows-ai-decision-route. New branch feat/workflows-ai-decision-runtime. Independent of #250.
Cast
GPT-6.1 Sol through codex:codex-rescue (bounded Node work) when Codex answers, else Opus 5.5.
Mandatory skills
/tdd, /writing-tests, /implementing-mcp-tools, /writing-user-facing-copy (run-log messages), /writing-code-comments, /writing-pr-descriptions, then qa-swarm.
Activity
Silthus commentedon Oct 4, 2026
Scope note from the conductor, from the #248 build (see its progress comment):
ml_inferencefacade's 5 s timeout applies to each httpx phase, not as a total deadline. Size the worker's 10 s timeout against that.Retry-After; the Node backoff owns that wait. A 429 carries a jitteredRetry-After.throttledin large batches) is its own ticket, AI decision step: let large batches through admission without throttling people #251, stacked after this one. Do not solve it here.Silthus commentedon Oct 5, 2026
Claimed by background implementation worker (Claude Code, Opus 5.5). CodeRabbit slot reserved. Wall-time cap 240 minutes.
Silthus commentedon Oct 5, 2026
Progress: draft PR PostHog#111685 open on branch
feat/workflows-ai-decision-runtime(head6ef6edd, stacked on #248'se6ecec8). Red then green proven; local gate green (semgrep 0, preflight 0 failures, repo-wide mypy clean). Codex (GPT-6.1 Sol) answered and wrote the first implementation from a brief; reviewed and refactored before commit. qa-swarm round 1 is running (six GPT lenses plus Claude fresh eyes). Ledger at/tmp/jev-249/qa/ledger.md.Silthus commentedon Oct 5, 2026
Superseded: Michael redrew v1 as one minimal PR (see the map's Destination). This ticket moves to v2. Its branch, if any, stays as the starting point.