Generate text node and its polling transport #48

Parent

Spec: #22 — Generate text, an LLM action node for workflows.

What to build

The runtime half of the node: a workflow containing a Generate text step runs, waits for a slow generation without holding a worker, and puts the result where downstream steps can read it.

The template. A hidden destination template, id template-workflow-llm, living beside its async function in the Node CDP template registry. Hidden rather than beta, because a beta status would list the node in the CDP destinations catalog for everyone and bypass the feature flag entirely. Its Hog shell makes one async call and returns the envelope. Three inputs in panel order: a required liquid-templated prompt, an output_fields dictionary keyed by workflow variable name with the instruction as the value, and a required searchable model choice defaulting to gpt-5-mini with the six ids grouped by provider, cheapest first. No reasoning-effort input — an author who wants more reasoning picks a stronger model. Default on_error is abort, not the usual continue, because a failure means the team was already charged and downstream steps would otherwise act on empty variables.

The mock. The registered async function declares a mock that powers the product's Test panel. It returns the real envelope shape with placeholder text and one obviously-fake value per declared field, plus a log line echoing the resolved prompt. Nothing in it may read like a real answer — an author testing flow logic must be able to tell at a glance that no credits were spent.

The transport. A new llmGenerate queue-parameter type, executed inline with self-reschedule, following the push-notification pattern rather than the email one. No new queue, no consumer, no capability, no pod, no chart change — jobs stay on the existing workflow queue. Each execution is one short HTTP call, a submit or a single retrieve. On a pending response the service returns unfinished with the same queue parameters and reschedules about two seconds out; a reschedule does not resume the VM, so it consumes no async step. A success pushes the envelope onto the VM stack. A terminal generation failure surfaces its error code and message into the run log and lets the node's error handling take over. A missing-feature response is terminal and must never be polled. A throttle response reschedules per its retry hint, up to a bounded number of bounces, then gives up as terminal. The service owns its own timeouts explicitly rather than inheriting the shared fetch defaults, which are far too short for this. There is a poll-count backstop carried in queue metadata and a wall-clock give-up, with the wall-clock as the primary check.

Two traps this ticket has to get right, both of which have bitten this code before. The result constructor clears queue parameters, so a reschedule must restore them explicitly or the next dequeue resumes with an empty stack and silently drops the generation. And the dispatch site gates on a hardcoded list of async types separately from its switch, so the new type has to be added in both places or it falls through to the resume path.

Preview. When the invocation is a test, the service loops inline in process to completion or the preview deadline instead of rescheduling. Preview therefore never exercises the reschedule transport, which is why it cannot serve as the rollout smoke test.

Metrics. Counters for generations started, finished and failed, and a duration histogram, all owned by the service.

The picker pin. A test asserting the six picker ids are members of the endpoint's validation superset. The picker is a deliberate subset so that refreshing it can never break a saved workflow; this test is what makes that a fact rather than an intention.

Acceptance criteria

• A workflow containing the node runs end to end against the generation endpoint and the envelope reaches the VM stack.
• A pending generation reschedules on the same queue at roughly the stated interval, and the reschedule consumes no async step.
• Queue parameters survive a reschedule — a multi-hop generation completes rather than resuming with an empty stack.
• The new async type is dispatched, not swallowed by the fall-through path, with the hardcoded type list updated alongside the branch.
• A terminal generation failure surfaces its error code and message into the run log, and the node's on_error applies.
• A missing-feature response is terminal and is never polled.
• A throttle response reschedules per its retry hint and becomes terminal after the bounded number of bounces.
• The wall-clock give-up and the poll-count backstop both terminate a generation that never completes.
• Submit and retrieve timeouts are set explicitly by the service rather than inherited from the shared fetch defaults.
• A test invocation loops inline to completion or the preview deadline and never reschedules.
• The mock returns the envelope shape with placeholder text and one obviously-fake value per declared field, and cannot be mistaken for a real answer.
• The template is hidden, so it does not appear in the CDP destinations catalog.
• The node's default error handling is abort.
• A test pins the six picker ids as members of the endpoint's validation superset.
• Started, finished, failed and duration metrics are emitted by the service.
• Covered by the existing workflow executor test harness, extending the template test helper if the new service has to be wired into it.

Blocked by

• Probe the gateway: strict output, extraction reasoning, and the six picker models #41 — the gateway probes. The picker cannot ship models the route rejects.
• Stop the run-log flood from same-queue self-reschedules #43 — the run-log flood fix. This ticket is what makes the flood happen.
• Generation endpoint, task, and first pass #45 — the generation endpoint this transport calls.