Feature: first-class cross-platform payload transport for script steps

[e-s-gh]

Summary

Script steps currently pass structured upstream data as rendered command-line arguments (for example, --evaluations <huge JSON>). This is fragile cross-platform, especially on Windows where command-line length limits are lower.

We validated this in real workflow runs where the same logical step worked only after rewriting large inline JSON args to temp-file based input (--evaluations-file).

Why this is platform-level (not workflow-specific)

Any workflow that feeds large structured payloads into script steps can hit the same failure mode. Workflow authors should not have to implement custom argument-size workarounds.

Evidence

From recent logs:

• docs-eval-run-10-final.log: validator/analyst invoked with inline --evaluations <large-json> payload.
• docs-eval-run-10-hardened.log: same run succeeds after rewriting to --evaluations-file with temp files.

Requested feature

Introduce first-class, generic script payload transport in Conductor, e.g. one or more of:

1. stdin_json (rendered object written to process stdin)
2. managed temp artifact handoff (engine-owned temp file path injection)
3. automatic large-arg spillover that is argument-name agnostic (not tied to a specific flag name)

Acceptance criteria

• Works uniformly on Windows/macOS/Linux.
• No dependency on argument names like --evaluations.
• Clear precedence/behavior when both inline args and managed transport are configured.
• Temp artifact lifecycle is deterministic and cleaned up.
• Add tests covering large payloads and Windows path/length behavior.

Notes

Current local mitigation is executor-level rewrite for --evaluations, which is pragmatic but workflow-shaped. This issue tracks the generic long-term design.

[jrob5756]

Triage check (2026-06-16, against main @ v0.1.18): still valid, not addressed.

Confirmed this hasn't been solved by any intervening work. Recording the current state so the next triager doesn't have to re-derive it.

Script steps still pass structured data only as rendered command-line args — the exact fragile mechanism this issue calls out. src/conductor/executor/script.py renders command / args / working_dir via Jinja2 and hands them to asyncio.create_subprocess_exec; stdin is inherited, nothing is piped in. The only change to that file since this issue was filed is the UTF-8 output fix (#90), which is unrelated.

None of the three proposed transport mechanisms exist. The type: script schema fields are just command, args, env, working_dir, timeout (config/schema.py:579-588):

• stdin_json → not built. Stdin piping was an explicit Non-Goal (NG1) in the original script-step design (docs/projects/usability-features/script-execution-steps.plan.md).
• managed temp-artifact handoff → none.
• argument-name-agnostic spillover → none.

The env: field is not a viable workaround: env values are passed to the subprocess as-is and are not Jinja2-rendered (script.py:98-99; documented at docs/workflow-syntax.md:335), so you can't inject an upstream agent's output through an env var. The Windows command-line-length failure mode described above is still reachable.

Note on a likely conflation: EPIC #234 fixed a different "large payload" problem — parsing agent JSON output containing triple-backticks (greedy fence regex in executor/output.py). That's about reading model output, not passing payloads into scripts. Distinct from this issue.

Conclusion: keeping open. A stdin: / stdin_json: field on type: script (render an object, write it to the child's stdin) is the clean, well-scoped fix and would resolve the cross-platform arg-length fragility without the argument-name-specific hacks.