Skip to content

Probe the gateway: strict output, extraction reasoning, and the six picker models #41

Open

Description

@Silthus

Parent

Spec: #22 — Generate text, an LLM action node for workflows.

What to build

Three facts the spec depends on but nobody has confirmed against the live gateway. Each is one request. None of them is a design decision — they choose between an already-decided primary and an already-decided fallback, or they confirm a claim the spec rests on.

P1 — strict response_format survives the gateway's OpenAI route. The field-extraction contract rests entirely on strict structured output: strict: true, additionalProperties: false, every property required. Send a strict schema through the gateway route the generation endpoint will use and confirm the schema reaches the provider unaltered and the response obeys it. If it does not, the extraction pass has no contract and the spec needs revisiting before any extraction work starts.

P2 — gpt-5.6-luna accepts minimal reasoning. The extraction pass is hardcoded to this model. The spec's primary is minimal; the fallback is low. Send one request at minimal and record which value the implementation should use.

P3 — the route permits all six picker models. gpt-4.1-mini, gpt-5-mini, gpt-5.4, claude-haiku-4-5, claude-sonnet-4-6, claude-opus-4-8. Ownership of this claim was previously nobody's. Send one request per model through the same route the endpoint uses. A model that does not answer cannot ship in the picker.

Record the answers as a comment on this issue so the tickets that depend on them read the result rather than re-probing.

Acceptance criteria

  • P1 answered: a strict-schema request through the gateway's OpenAI route is confirmed to reach the provider unaltered, and the response obeys the schema.
  • P2 answered: the reasoning value the extraction pass should use is recorded as either minimal or low.
  • P3 answered: each of the six picker models is confirmed to answer through the route, or named as failing.
  • Results are posted as a comment on this issue, so no dependent ticket has to re-probe.
  • If P1 fails, that is escalated rather than worked around — the extraction contract does not survive it.

Blocked by

None — can start immediately.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified, ready for an AFK agent

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions