Skip to content

feat(cli): stream long-form speech with say --stream - #6

Merged
luke-speechify merged 5 commits into
mainfrom
feat/audio-stream
Aug 28, 2026
Merged

feat(cli): stream long-form speech with say --stream#6
luke-speechify merged 5 commits into
mainfrom
feat/audio-stream

Conversation

@ayushkcs

Copy link
Copy Markdown
Member

feat(cli): stream long-form speech with say --stream

Adds say --stream for long-form speech using POST /v1/audio/stream. Audio is written to disk as it arrives, without holding the full clip in memory.

API Reference

Stream Speech - POST https://api.speechify.ai/v1/audio/stream

1. One Command, Two Routes

The existing say behaviour stays unchanged. --stream adds a second route for long-form text.

say "…" say --stream "…"
Endpoint /v1/audio/speech /v1/audio/stream
Text limit 2,000 chars 20,000 chars
Response One JSON response Raw audio chunks
Memory Full clip One chunk at a time
Formats wav, mp3, ogg, aac, pcm mp3, ogg, aac, pcm, ulaw
Billing Reported Not reported by this route
Existing output Replaced Kept unless --force is used

Also adds the stream_text_to_speech MCP tool for generating long-form audio directly to a file.

2. Mechanism - Where the bytes actually go

The one thing worth understanding: audio is never held in memory and never written straight to the file you asked for. It lands in a hidden scratch file, which is renamed into place only after the last chunk.

image

Audio is appended to a hidden .part file in the destination folder and renamed only after the final chunk, so the file you asked for is either complete or absent.

Every failure exit — a rejected flag, a stalled server, a dropped connection, Ctrl-C — removes the scratch file on the way out.

3. Flag Behaviour

All invalid combinations are rejected before making a request.

YOU TYPE RESULT WHY
say --stream "…" MP3 Default. Writes ./speech.mp3
--stream --format ogg OGG Container choice: mp3, ogg, aac, pcm
--stream --format wav EXIT 65 Only the non-streaming route makes wav; the message says so
--stream --output-format pcm_16000 PCM, 16 KHZ Exact codec, sample rate and bitrate. Writes ./speech.pcm
--stream --output-format ulaw_8000 TELEPHONY What Twilio and SIP expect. Writes ./speech.ulaw
--stream --format mp3 --output-format … EXIT 65 The server would silently ignore one of them, so we refuse both
--output-format … (no --stream) EXIT 65 It only applies to the streaming route
--force (no --stream) EXIT 65 Nothing to force: without --stream the file is always replaced
--stream (speech.mp3 exists) EXIT 65 We chose that filename, so we will not overwrite it. Use --out or --force
--stream --out mine.mp3 (exists) REPLACED You named the file, so replacing it is what you asked for
--stream --out - (into a pipe) RAW AUDIO Pipe it to a player for the fastest possible start
--stream --out - (to a terminal) EXIT 65 Refuses to fill your terminal with binary
--stream --play (pcm or ulaw) SKIPPED No player can guess the rate; the exact ffplay command is printed instead
--stream (20,001 characters) EXIT 65 Over the route's ceiling, caught before the request

4. Failure Handling

SITUATION YOUR FILE EXIT WHAT YOU ARE TOLD
Server answers, then goes quiet NOTHING WRITTEN 69 Stream stalled after 30s, and how to raise the budget
Connection drops mid-download NOTHING WRITTEN 69 Stream ended unexpectedly
You press Ctrl-C NOTHING WRITTEN 130 Nothing, as expected of an interrupt
Server sends zero bytes NOTHING WRITTEN 69 The stream ended without sending any audio
Reader closes the pipe (| head) NORMAL END 0 How much was streamed, on stderr
Out of quota, rate limited NOTHING WRITTEN 75 The server's own message, plus its request id

The final output file is only created after the complete stream succeeds, so a failed stream never leaves a truncated audio file behind.

5. Verification

pnpm typecheck && pnpm lint && pnpm test

215 unit tests are passing, including 63 new tests.

image

Verification Script:

pnpm build
SKIP_LIVE=1 ./verify/stream.sh
./verify/stream.sh

verify/stream.sh contains 31 checks:

  • 10 offline: flag validation, character limits, emoji counting, and cleanup
  • 8 local: stalled streams, dropped connections, Ctrl-C, and cleanup
  • 13 live: real long-form streaming, formats, overwrite behaviour, piping, and terminal protection
image

6. Main Changes

File State What it does
core/speech.ts EDITED Asks for audio as a flow instead of one lump; fixes the character count
core/stream.ts NEW Reads the chunks, gives up clearly if the server goes silent
audio/sink.ts NEW Saves to disk safely, or to a pipe; cleans up after itself
commands/say.ts EDITED The --stream, --output-format and --force options
mcp/server.ts EDITED The long-form tool for assistants
3 test files NEW 63 new tests: streaming, saving, failing, every option combination
2 test files EDITED Cover the new behaviour in the existing suites
verify/stream.sh NEW 31 checks against a fake server and the real API
verify/README.md NEW How to run the checks, and the format a new script follows
pnpm-workspace.yaml NEW Lets esbuild run its install script, so build and test work
biome.json EDITED Keeps local editor state out of lint and format

pnpm 10 no longer reads the "pnpm" field in package.json, so dependency
build scripts were blocked and both `pnpm build` and `pnpm test` exited 1
with ERR_PNPM_IGNORED_BUILDS. esbuild needs its install script to fetch and
verify the platform binary tsup builds with.

Also exclude the local agent tooling directory from biome. It holds untracked
editor state, not source, and one settings file in it is enough to fail
`pnpm lint`.
Adds streamSpeech() for POST /v1/audio/stream, which returns as soon as the
response headers land so the caller can write bytes as they arrive, plus the
two pieces that move those bytes somewhere.

- core/stream.ts reads the body chunk by chunk with a stall budget measured
  between chunks. The SDK and fetchWithTimeout only bound time to first
  response, so a server that answers and then goes quiet would otherwise
  hang the CLI forever. Empty keep-alive chunks are skipped and the reader
  is cancelled if the consumer stops early.
- audio/sink.ts writes to a hidden scratch file in the destination directory
  and renames it into place only after the last chunk, removing it on any
  failure including SIGINT and SIGTERM. The file you asked for is therefore
  either complete or absent, never truncated and looking finished. The stdout
  path honours backpressure and treats a closed reader as a clean stop.
- Format selection has one source of truth: output_format overrides the
  Accept header server side, so passing both is refused rather than silently
  ignoring one. The codec, and so the file name, comes from the request and
  never from a response field. wav is not offered because it needs a length
  in its header.
- The input length check now counts code points. 1,500 emoji is 3,000 UTF-16
  units, which the old check rejected even though the server accepts it. Past
  the 2,000 limit the error now points at --stream.
- Adds the simba-3.2 model, which is streaming native with a lower TTFB.
say --stream synthesizes up to 20,000 characters through
POST /v1/audio/stream and writes the audio as it arrives, so the first byte
lands sooner and nothing is buffered in memory.

- --output-format takes an exact codec, sample rate and bitrate, for example
  pcm_16000 or mp3_24000_64. --format keeps naming the container.
- The default output file is chosen by us, not the user, so an existing
  speech.<ext> is refused with --force named as the way past it. A path given
  with --out is replaced as before.
- --out - into a terminal is refused, with the redirect, the pipe and the
  drop-the-flag alternatives spelled out.
- Every flag combination is checked before text is read, a token is refreshed
  or a request is sent, so an invalid run costs nothing.
- pcm and ulaw carry no header, so --play is skipped for them and the output
  gives the ffplay command that works.
- Flag validation moves into one assertSayFlags() helper and the streaming
  run into runStream(), keeping the action callback readable.
Long text can now be synthesized straight to a file instead of being pulled
through the conversation. outputPath is required and the tool returns the
path and byte count, never the audio, so a 20,000 character narration cannot
flood an agent's context.

The tool description tells a caller to prefer it over text_to_speech past
2,000 characters.
verify/stream.sh proves the feature against the real API, the layer pnpm test
cannot reach. Three groups by what they cost: 10 offline flag checks with no
network, 8 checks against a deliberately broken server on 127.0.0.1 for the
stall, drop and Ctrl-C paths, and 13 live checks behind SKIP_LIVE.

It asserts exit codes and messages rather than just that something happened,
works in a mktemp dir so it never writes into the repo, and checks what was
left behind after each failure.

verify/README.md explains how to run them and the format a new script follows.

@luke-speechify luke-speechify left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

appreciate the flag-combination work in here, assertSayFlags rejecting everything before the request goes out is the right instinct. but SAY_INPUTS didn't move with it, and that's the one thing I want fixed before this lands.

say.ts:56 still lists text, voice, format and out. --stream, --output-format and --force are all missing from it, so say --stream with no text under an agent gets back a spec that doesn't mention the flag it was just given, and still offers --format wav, which that route rejects with a 65. that spec is the only thing an agent sees on the needs-input path, so a flag that isn't in it may as well not exist.

there's a rule behind this that was never written down anywhere, which is on me. global flags (--api-key, --workspace, --json, --no-input) never change whether a command can prompt. any command flag or positional argument does: pass one and the command is binary, it either runs or it stops and says what else it needs, and when it stops it lists the whole input set rather than just the missing bit. I've written it up in #8, worth a read before you pick this back up.

two smaller things fall out of it. speech.mp3 already exists, pass --out or --force (audio/sink.ts:110) and the --out - to a terminal refusal (sink.ts:101) are both 65s carrying the alternatives in prose. the fix in each case is another flag, so those are needs-input outcomes, exit 2 with the structured list.

and --stream is a mode flag: it narrows --format's enum and gates the other two. InputField[] is flat so the spec can't express that, and when the interactive walk lands it won't be able to branch on it either. I'd rather it became say stream with its own flat input set than we grow conditionals in InputField, but I'm open on which way round.

asks:

  • add --stream, --output-format and --force to SAY_INPUTS, with the route-dependent --format values noted
  • turn the two sink.ts refusals into NeedsInputError
  • tell me which way you want to go on --stream as a mode flag vs a subcommand, before more flags land on it

getOptionValueSource("format") === "cli" at say.ts:171 is exactly the primitive that rule needs, so half of it is already in your diff. rest of the PR is good, the .part rename and the interrupt cleanup especially. I've left the lighter version of the same note on #7.


Generated by Claude Code

@luke-speechify
luke-speechify merged commit ad1f436 into main Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants