Repository navigation
Description
Why
Podcast transcripts get proper nouns wrong: podcast names, host and guest names, product names. The transcription API's hint parameters exist for exactly this problem, and we already hold the relevant metadata (podcastName, episodeTitle, podcastAuthor) in the queue consumer at the moment we call transcribeAudio. Today we throw it away.
This is the follow-up carved out of #46 ("Not in scope: prompt, keywords[], languages[]"), kept separate so a behavior change can't hide inside the model swap. Blocked by #46: keywords[] is a gpt-transcribe-only parameter and would 400 against gpt-4o-mini-transcribe. prompt alone works on the current model, but shipping half the feature ahead of the swap means two behavior changes instead of one, so the whole thing lands after #46.
The open question #46 recorded (how array params encode in multipart form data) is answered in the docs: the speech-to-text guide's own curl example sends repeated keywords[] fields (-F 'keywords[]=premium plan' -F 'keywords[]=AC-42'), matching the timestamp_granularities[] convention. Still worth one live confirmation in pre-flight since we build FormData by hand.
API surface (verified against docs, 2026-08-02)
From the speech-to-text guide and the migration cookbook:
| Param | Type | Models | Semantics |
|---|---|---|---|
prompt |
string | gpt-transcribe, gpt-4o-transcribe(-mini) | "Unstructured context about the recording." Cookbook: describe the topic/setting, don't list terms there |
keywords |
string[] (multipart: repeated keywords[]) |
gpt-transcribe only | "Literal terms you expect to hear." Hints, not required output |
languages |
string[] (multipart: repeated languages[]) |
gpt-transcribe only | Expected input languages; replaces the legacy language field |
Validation rules that matter to us:
- Each keyword must be a single-line literal and must not contain
<,>, CR, or LF. The API rejects the request otherwise. - Never send both
languageandlanguages. - An over-long
promptis rejected. No numeric limit is documented, so keep it to a sentence.
whisper-1 supports none of this in the same form: its prompt is the old style/continuation hint (224-token cap, different semantics) and it takes a singular language. This matters for the fallback path below.
Current state (verified in code, 2026-08-02)
Line refs are current main; #46 will shift them slightly.
- The queue consumer calls
transcribeAudio(audioUrl, { apiKey, skipValidation })atsrc/queue/consumer.ts:531withmetadata.podcastName,metadata.episodeTitle, andmetadata.podcastAuthorall in scope. Both metadata paths populatepodcastAuthor: the RSS-sourced path frommonitored?.author(consumer.ts:444), the Apple-submit path viagetEpisodeMetadata. callWhisperApibuilds the request FormData atsrc/services/transcription.ts:474-478(file, model, response_format only). Chunked episodes go throughtranscribeWithChunking(:869), which callscallWhisperApionce per chunk (:962). Each chunk is an independent API call, so hints have to reach every call.- The whisper-1 fallback (
:581-584) builds its own bare FormData. It fires on 400s whose body matches the"corrupted or unsupported"/"input_too_large"substrings. A hint-validation 400 would NOT match those substrings, so a bad keyword would fail the chunk outright rather than falling back. That's why sanitization happens at hint-build time, not in the API call. TranscribeOptions(:682) currently carriesapiKey,provider(legacy),onProgress,skipValidation.- Tests already inspect FormData bodies from the fetch spy (
test/transcription.test.ts:464-469,:596), so asserting hint fields follows the existing pattern. - No language metadata exists anywhere in the pipeline. The RSS parser doesn't read the feed's
<language>tag, and neitherEpisodeMetadatanor the monitored-podcast KV records store one. (This is whylanguages[]is out of scope below.)
Design
Send per-episode hints on the primary (gpt-transcribe) call only:
prompt: one sentence of context, per the cookbook's "describe the recording" guidance:An episode of the podcast "<podcastName>", hosted by <podcastAuthor>, titled "<episodeTitle>".with segments dropped when a field is missing or a placeholder. The episode title's proper nouns (usually the guest's name) reach the model here.keywords[]:[podcastName, podcastAuthor]. These are literal terms actually spoken in nearly every episode intro and outro. The episode title is deliberately not a keyword: keywords are for "literal terms you expect to hear," and a full editorial title isn't one.- Same hints on every chunk of a chunked episode.
- whisper-1 fallback: completely unchanged. It exists to rescue chunks the primary rejected, so it stays the most minimal request we know works.
keywords[]would 400 there anyway. - No hints when metadata is placeholder junk (
"Unknown Podcast"/"Untitled Episode"from the RSS-path defaults). Send nothing rather than teach the model garbage.
Cost: a sentence of prompt plus two short keywords is tens of text-input tokens per call, a rounding error at admin-curated volume.
Caveat on measurement: the A/B harness covers summary models only; there is no transcription eval. Acceptance is the pre-flight spot check on known-bad proper nouns, not a metric.
Work
1. Add the hint builder (src/services/transcription.ts), exported for the consumer and tests:
export interface TranscriptionHints {
prompt?: string;
keywords?: string[];
}
/** Strip chars the API rejects in keywords (<, >, CR, LF), collapse whitespace. */
function sanitizeHintText(value: string): string {
return value.replace(/[<>\r\n]/g, " ").replace(/\s+/g, " ").trim();
}
export function buildTranscriptionHints(meta: {
podcastName?: string;
episodeTitle?: string;
podcastAuthor?: string;
}): TranscriptionHints | undefinedRules:
- Sanitize all three fields; treat
"Unknown Podcast","Untitled Episode", and empty strings as absent. - Cap the title's contribution (~200 chars) so a pathological RSS title can't produce an over-long prompt.
- keywords = sanitized
[podcastName, podcastAuthor], deduped case-insensitively (author sometimes equals the podcast name), empties dropped. - Return
undefinedwhen nothing usable remains, so callers can pass the result straight through.
2. Thread hints to the API call. TranscribeOptions gains hints?: TranscriptionHints. transcribeAudio passes opts.hints to both the single-shot call (:837) and transcribeWithChunking (new parameter, :869-876), which passes them to every per-chunk callWhisperApi (:962). In callWhisperApi, after the existing appends at :474-478:
if (hints?.prompt) formData.append("prompt", hints.prompt);
for (const keyword of hints?.keywords ?? []) {
formData.append("keywords[]", keyword);
}The fallback FormData (:581-584) gets no hint fields. Add a comment saying that's deliberate so nobody "fixes" it later.
3. Pass metadata in the consumer (src/queue/consumer.ts:531):
const transcriptionResult = await transcribeAudio(audioUrl, {
apiKey: env.OPENAI_API_KEY,
skipValidation,
hints: buildTranscriptionHints(metadata),
});4. Debug route support (src/index.ts:186). Optional podcast, title, author query params on GET /debug/transcribe, run through buildTranscriptionHints, so pre-flight can exercise hints in dev without queueing a job.
5. Observability. Add hasPrompt: boolean and keywordCount: number to the whisper_api_call_start log event. Hint content is recoverable from episode metadata, so counts are enough to see whether hints were in play when eyeballing a bad transcript in wrangler tail.
6. Tests (test/transcription.test.ts, TDD per repo norm):
buildTranscriptionHints: strips<>/CR/LF, collapses whitespace, drops placeholders, dedupes name/author, truncates long titles, returnsundefinedon junk-only input.callWhisperApiFormData containspromptand repeatedkeywords[]entries when hints are passed (assert viaformData.getAll("keywords[]")on the fetch spy), and neither field when hints are absent.- The fallback call's FormData contains no
promptorkeywords[]. - Chunked path: every chunk's call carries the same hint fields.
7. Docs. One line in AGENTS.md (Request Flow step 2 and the services/transcription.ts structure entry) plus the transcribeAudio doc-comment example.
Pre-flight check (dev, before merging)
Using GET /debug/transcribe (requires ENVIRONMENT=development):
- One small episode with hints: confirm HTTP 200 and a normal body. This is the live confirmation of the
keywords[]multipart encoding that Migrate transcription from gpt-4o-mini-transcribe + whisper-1 to gpt-transcribe #46 asked for. - Find an episode whose existing prod transcript misspells the podcast, host, or guest name (browse recent
transcript:{episodeId}records for candidates). Re-run it through the branch with hints and diff the proper nouns. This is the acceptance check. - One chunked (>25MB) episode: confirm hints ride along on every chunk (log events) and the stitched output looks normal.
Verify (after merge)
npm testgreen.- Deploy is manual (
npm run deploy); merging does not deploy. npx wrangler tailon the next cron tick:whisper_api_call_startshowskeywordCount: 2(or 1), andwhisper_fallback_startdoesn't spike. A spike would mean the hinted request 400s in some new way; the logged error body will show the wording.- Spot-check the next few transcripts for podcast/host name spelling.
Not in scope
languages[]— nothing in the pipeline knows the episode language (no RSS<language>capture, no stored field), so there's no honest value to send. Hardcoding["en"]bakes an assumption into every future podcast for zero observed benefit, and auto-detect plus the detected-languagesresponse field already cover it. If a non-English podcast joins the archive, first capture<language>at ingest, then send it per call (and never alongside legacylanguage).- Guest names as keywords — the title's proper nouns already reach the model via
prompt; promoting them to keywords means extracting names from editorial titles (heuristics or an LLM pass). Revisit only if pre-flight shows title-cased names still coming out wrong. - Continuation prompting across chunks (previous chunk's tail as prompt) — that's whisper-1's prompt semantics, not gpt-transcribe's context prompt, and it would interact with the overlap-dedup stitching.
prompton the whisper-1 fallback — different parameter semantics (style/continuation, 224-token cap); the fallback stays minimal on purpose.- Retry-without-hints on 400 — sanitization makes hint rejection unreachable for expected inputs; if a rejection shows up anyway,
whisper_api_errorlogs the full body and we handle it then.
References
- Speech-to-text guide — hint params, keyword validation rules, the
keywords[]curl example - Cookbook: Migrating from Whisper to GPT-Transcribe — prompt-vs-keywords guidance,
languagesmigration - Migrate transcription from gpt-4o-mini-transcribe + whisper-1 to gpt-transcribe #46 — the model migration this depends on
Activity