irodori_tts: accept Speaker Inversion embeddings as the speaker condition

[KKTTSPJ]

Irodori-TTS can learn a reusable voice with Speaker Inversion: a few speaker tokens are trained while the base model stays frozen, and saved as *.speaker.safetensors (one tensor, speaker_embedding, [tokens, speaker_dim]). The Python runtime uses such a file in place of reference audio with infer.py --ref-embed. audio.cpp's irodori_tts could not take one, so an embedding-based voice had to go back to a reference clip.

This adds it to the irodori_tts session only. No framework, server or CLI change.

How a request selects an embedding

Following PocketTTS's layout for precomputed voices:

• By path: request option speaker_embedding_path (--request-option speaker_embedding_path=..., or "options" in a server request).
• By name: <model dir>/embeddings/<name>.safetensors, selected with the voice id (--voice-id <name>, server "voice": "<name>"). The server's GET /v1/audio/voices already lists that directory for any family, so before this PR it offered such names for irodori_tts too, but selecting one silently gave no-reference speech. A voice id with no such file is ignored, as before, so clients that always send an OpenAI voice name are unaffected.

audiocpp_cli --task tts --family irodori_tts \
  --model models/Irodori-TTS-v4-Small-GGUF/irodori-tts-v4-small-q8_0.gguf \
  --backend cuda --language ja --text "こんにちは、これは学習した話者埋め込みを使った推論です。" \
  --request-option speaker_embedding_path=path/to/name.speaker.safetensors --out out.wav

Behaviour

• The tokens become the speaker condition as they are — no speaker encoder, normalization or prepended mean token, all positions unmasked — which is what InferenceRuntime._load_speaker_embedding_condition / speaker_inversion_batch_tensors do. The duration predictor reads the first token, as _speaker_vec does. The reference encode and the reference cache are skipped.
• One speaker source per request, as in Python: an embedding together with reference audio or an explicit no_ref=true is an error, and so is a checkpoint without speaker conditioning.
• F32, F16 and BF16 load; [1, tokens, dim] is accepted like normalize_speaker_embedding_tensor. A wrong tensor name, dim or data size names the file.
• No embedding → the existing no-reference and reference-audio paths are unchanged (byte-identical outputs below).

Two implementation notes:

• IrodoriTTSAssets gains model_dir (the model path as given, or its directory). resources.model_root() cannot be used for embeddings/: for a standalone GGUF with embedded sidecars it points at the extracted sidecar directory.
• Published packages take the option too. The shipped GGUFs embed a contract written before speaker_embedding_path existed, so it is dropped from the validation copy when the contract does not declare it — the same pattern as instruction a few lines above and phonemes in kokoro_tts: accept a supplied phoneme stream, bypassing the built-in G2P #577. Unrelated unknown options still fail.

Validation

Windows 10, MSVC 14.51, CUDA 12.8, RTX 5060 Ti (sm_120). Built with -DAUDIOCPP_MODEL_SET=custom -DAUDIOCPP_MODELS=irodori_tts, CPU (AVX2) and CUDA. (The local CUDA build also needed a workaround for MSVC 14.51 being newer than CUDA 12.8 supports; it is not part of this PR.) Model: irodori-tts-v4-small-q8_0.gguf; embedding: a 16-token v4 Small Speaker Inversion checkpoint; seed 42, 48 steps. All requests ran through one long-lived audiocpp_server per configuration.

Parity with the Python runtime (infer.py, fp32, CUDA, same text/seed/steps; log-mel over the common length):

path	duration (audio.cpp / Python)	mel cosine	mel L1
embedding (--ref-embed)	6.64 s / 6.64 s	0.9975	0.159
no reference (existing path)	6.88 s / 6.88 s	0.9964	0.289
reference WAV (existing path)	7.08 s / 7.04 s	0.9724	0.911
embedding vs Python no-reference (control)	6.64 s / 6.88 s	0.7766	4.127

The predicted duration matches Python to the frame, and the embedding path tracks Python at least as closely as the existing paths do (Q8 weights here vs fp32 in Python).

Server / CLI (CUDA unless noted; "identical" = byte-identical WAV output):

case	result
speaker_embedding_path	200
same, long text (several chunks)	200
"voice": "mio" with embeddings/mio.safetensors	200, identical to the by-path output
F16 copy of the embedding	200 (close to, not identical with, F32 — expected)
CLI --request-option speaker_embedding_path / --voice-id mio	identical to the server output
CPU backend, by path / by name	200, identical to each other
GET /v1/audio/voices	{"voices":["mio"]}
reference WAV, no embedding	200, identical to v0.9.1 without this PR
no reference	200, identical to v0.9.1 without this PR
"voice": "alloy" (no such file)	200, identical to no reference, as before

Errors (HTTP 500 with these messages):

with reference audio   -> Irodori-TTS speaker embedding cannot be combined with reference audio
with no_ref=true       -> Irodori-TTS speaker embedding cannot be combined with no_ref=true
missing file           -> Irodori-TTS speaker_embedding_path does not exist: ...
wrong tensor name      -> Irodori-TTS speaker embedding has no speaker_embedding tensor: ...
wrong dim              -> Irodori-TTS speaker embedding must have shape [tokens, 768]: ...
token count / data size mismatch -> Irodori-TTS speaker embedding data does not match its header: ...

Both branches of the contract check ran, with identical outputs:

• with the PR's model_specs/, where the option is declared;
• from a directory without model_specs/, so the GGUF's embedded contract applied and the option is undeclared.

The embedding request is also faster than a reference clip (about 1.2 s vs 2.0 s for this text on CUDA), since there is no reference encode.

Known limitations

• An embedding is tied to the checkpoint it was trained on. One trained on another checkpoint with the same speaker_dim loads, but the voice may not match; the docs say so, as the upstream Irodori-TTS README does.
• GET /v1/audio/voices lists embeddings/ only when the server config's model path is the directory, not the .gguf file (existing server behaviour); lookup by name works either way.

🤖 Generated with Claude Code

[0xShug0]

@KKTTSPJ Thanks for adding this feature to Irodori-TTS! PR merged.