[Silthus]

Problem

A workflow author cannot make a workflow write text that adapts to the person it runs for. Personalizing beyond variable substitution means calling an LLM from their own backend and writing the result back, which most teams never do.

This is the bottom layer of a 4-PR stack implementing the Generate text spec. It ships the server side: the generation endpoint the CDP calls, and the Celery task that runs the model calls and signals completion.

Changes

• Adds POST/GET /api/projects/:team_id/workflow_llm_generations/, authenticated with the team secret token the CDP already holds, behind the workflows-llm-action flag (fail-closed, organization-aggregated).
• Status codes are retry instructions: 202 while pending, 200 for both success and terminal failure, 404 feature_unavailable when the flag is off, 429 with Retry-After from the 60/minute per-project submit throttle.
• Retrieve skips the flag check, so in-flight generations survive a rollback.
• Submit accepts an optional wake_token minted by the caller. Every terminal write (success, failure, exhausted retries, revoked task) produces a $workflows_llm_generation_finished internal event echoing it, so the parked CDP job wakes without polling.
• The event is wake-only: it carries no generation output, and a produce failure only means the caller's backup polls collect the result.
• Failure and pending record TTLs rise to 5 and 10 minutes. A record must outlive the caller's backup ladder (marks at 1, 3 and 5 minutes), or a poll after expiry reads as feature_unavailable.
• A Celery task on the long_running queue runs two passes: the author's model generates text, then a pinned extraction model (gpt-5.6-luna) fills declared output fields under a strict JSON schema. Zero declared fields skip the second pass.
• One retry, only for model-fixable failures (oversized, malformed, refused). Transport is never retried.
• Generation records live in Redis and double as the dedupe store, keyed by invocation_id plus a payload hash claimed with an atomic SET NX. No Django model, no migration.
• Both passes bill as PostHog AI credits ($ai_billable, ai_product: workflows) on one trace per workflow run. Submit gates on the org AI credit budget and answers over-quota as a 200 terminal failure without enqueuing.
• Concurrency caps: 30 global, 5 per team, slot TTL 120s so a dead worker cannot free a live slot before the 90s hard time limit.
• The six picker models are a pinned subset of a new validation superset built from the OpenAI and Anthropic registries, so a registry refresh cannot break a saved workflow.

Note

Two decisions to sanity-check in review: tach.toml gains a products.workflows -> products.ai_observability edge for the model registry import (ai_observability is not isolated and has no facade; a facade would be a larger cross-product change), and byte-identical duplicate Generate text steps in one run intentionally share a generation (the payload hash is the separator, per spec).

How did you test this code?

• Endpoint contract tests (DRF client, Celery mocked): 202/200/404/429 semantics, idempotency enqueues once, cross-team token rejection, throttle bucketed per team, quota and model gates.
• Task tests (gateway stubbed): envelope assembly, null-to-empty coercion, zero-field skip, the single retry, the shared wall-clock budget covering the retry, billing tags on both passes.
• New wake-event tests: every terminal write emits exactly one event echoing the stored token and no generation text; a record without a token emits nothing; a task discarded before running still emits. Each guards the regression where a dropped emit silently demotes every generation to the slow backup ladder.
• Concurrency limiter tests against isolated fakeredis.
• A registry pin test makes the picker-subset relationship a fact instead of an intention.
• Not run: the three live-gateway probes the spec names (strict response_format pass-through, minimal reasoning acceptance, route model coverage). No gateway credentials exist in this environment. The reasoning fallback (minimal, degrade to low) is implemented and tested; the probes need a real gateway before rollout.

Automatic notifications

• Publish to changelog?

Docs update

Docs, skills, and MCP updates ship in the top layer of this stack.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Orchestrated multi-agent implementation: Opus 5 implemented under strict red-green TDD, three Fable 5 reviewers (spec compliance, adversarial correctness, cleanliness/test quality) raised 17 findings, an Opus fix pass resolved them, and a Fable confirm pass verified each fix by reading the code and re-running the tests. Skills invoked: /tdd, /implement, /improving-drf-endpoints (via review lens), /writing-tests, /writing-pr-descriptions, /stacking-prs. Notable session decisions: rejected a scouted Django-model design in favor of the spec's Redis-only records; kept gpt-5.6-luna a workflows-local constant instead of adding it to the shared model registry to avoid a side effect on the AI observability picker. The review pass converted a read-then-write dedupe into an atomic claim and added the revoked-task handler that prevents forever-pending records.

v2 revision (Fable 5, single session, human-directed): after review, the transport moved from constant polling to park-and-wake. This layer gained the wake_token field, the terminal-write wake event, and the raised TTLs. Skills invoked: /improving-drf-endpoints, /writing-tests, /writing-user-facing-copy, /stacking-prs, /writing-pr-descriptions. The spec deltas are recorded on issue #22 and in the updated blueprint artifact.