Note

A Speaker Label Is Not an Identity: Offline Transcription Arrives in llama.cpp v0.9.0

llama.cpp v0.9.0 adds offline Nemotron 3.5 Transcribe: the recording can stay local, but a speaker label is clustering, not identity, and accuracy still has gaps.

2026-11-09 · oss-ai, local-ai, llama-cpp, speech-to-text, transcription, text-to-speech, data-residency, sovereignty, civil-liberties

Listen · 10 min · AI voice

Dark CryptoDecentral hex mesh with a calm sound waveform flowing into a sealed teal hexagonal enclosure; inside, the wave splits into five bronze strands ending in small unnamed nodes; no text.

Open-source AI special · week of 9 Nov 2026

The most sensitive file most people own isn't a document. It's a recording: an interview, a doctor's appointment, a union meeting, a voice note from a source. For years the easy way to turn that audio into text has been to upload it to someone else's server. That default just got weaker. llama.cpp v0.9.0, published 4 Nov 2026, added support for NVIDIA's open-weight Nemotron 3.5 Transcribe, offline speech-to-text "in 40 languages with prompt context and speaker attribution," plus two local text-to-speech paths.

That's a real gain for data residency. The recording can now stay on the machine where it was made. But the same release is a good moment to be precise about what local transcription gives you. It keeps the audio. It doesn't make the transcript true, and it doesn't tell you who was speaking. A label like <spk:1> means "a different voice from <spk:0> in this file." It is not a name.

What actually shipped

The transcription work landed in pull request 31471, merged 3 Nov. It runs through llama.cpp's multimodal layer (mtmd), the same path that handles images and other audio models (multimodal docs). You load the language model plus a projector file, hand it a recording, and get text back from either llama-mtmd-cli or llama-server.

The model itself, per its model card, is a 34B-parameter mixture-of-experts with about 4.3B active, built from a Transformer speech encoder, a roughly 99M-parameter speaker encoder, and a Mamba-2–Transformer hybrid language backbone. It handles recordings up to an hour. It is published under OpenMDW-1.1, which is not one of the licences most people already know. Read it before you build on it, for the reasons in our note on open weights versus open source.

Three mechanisms in the PR matter more than the headline.

Long audio is cut into windows. The PR says a recording is "split into 40s core window with 4s of context on each side, one graph each (upto 48s)." Each window is processed separately, so something has to stitch speakers together across the cuts. That's a speaker cache and a first-in-first-out buffer carried from window to window, ported from NVIDIA's streaming diarization code. Without it, the author notes, "speaker numbers would reset every 40s."

Speaker numbers are clustering, not recognition. The speaker encoder groups stretches of audio that sound alike and gives each group an index. The model card says so directly: "Speaker labels are intended to distinguish speakers within a recording; they do not establish real-world identity." Speaker 0 in one file and speaker 0 in the next are unrelated. Two people with similar voices on a bad phone line can merge into one label. One person who moves away from the microphone can split into two.

One setting can make the output vanish. The checkpoint's chat template turns the model's "thinking" on by default. In llama-server, the PR warns, you need --reasoning off, or "the server returns only reasoning and no transcript." The CLI needs no flag. It's a small thing, but it shows how much of a local stack's behaviour lives in templates and flags rather than in the weights.

Word timestamps and live streaming are, in the PR's words, "beyond the scope of this PR." The model card describes timestamps through a different serving engine, vLLM. If your workflow depends on "who said what, when," llama.cpp doesn't do that part yet.

How good is "good"?

The contributor published their own measurements, which is better than most releases offer:

	Q8_0	IQ4_NL	BF16 (model card)
LibriSpeech test-clean word error rate	1.03	1.02	1.09
AMI meetings, speaker-aware word error rate	9.91	10.38	10.29

Two readings follow. First, on these tests the quantised files track the full-precision reference closely, so a smaller download didn't obviously cost accuracy. Second, look at the meeting row. LibriSpeech is one person reading a book clearly. AMI is real meetings, and there roughly one word in ten is wrong or credited to the wrong speaker. Your recording, with traffic, crosstalk and a cheap phone, is closer to the meeting row than the book row. These are one contributor's numbers on one GPU setup, not a guarantee.

The language list is a map of who gets served

The model card sorts its languages into three tiers. Thirteen are "Leading," including English, Hindi, Portuguese, Spanish and Russian. Fifteen are "Production-ready." The lowest tier, "Supported," is defined as "may need fine-tuning," and it includes Arabic, Mandarin, Bengali, Indonesian, Tamil, Telugu, Tagalog and Vietnamese. Together those are languages for well over a billion people. Swahili, Hausa, Yoruba, Amharic and Zulu don't appear at all.

That matters for a civil-liberty reading of "offline." The people with the strongest reason not to upload a recording, such as journalists working with sources, rights workers taking testimony, or anyone whose government can subpoena a cloud provider, are often speaking a language in that bottom tier or off the list. Local transcription removes the upload. It doesn't remove the accuracy gap, and a confident transcript in a poorly supported language can be worse than no transcript, because people quote it.

The other direction: a voice is cheaper than ever

The same release added speech generation. PR 30993 brings Chatterbox TTS, which its author describes as "quite complicated, still WIP." PR 31429 changes Kyutai's Pocket TTS, whose llama.cpp support had been "voice-cloning based," so it can now use a set of predefined voices from the pocket-tts-without-voice-cloning repository instead. In the same release, the server's GET /props lists predefined TTS voices and the TTS endpoint accepts them by name.

That's a thoughtful default. A fixed, published voice can't be mistaken for a real person's. The Pocket TTS project describes a roughly 100M-parameter model that runs on two CPU cores. Its model card asks users to agree that prohibited uses include "voice impersonation or cloning without explicit and lawful consent" and "presenting generated content as genuine recordings of real people." On your own machine, nothing enforces that. There's no provider in the loop to refuse a request or keep a log. Sovereignty cuts both ways. The machine that keeps your source's recording private can also make audio of a voice that never said those words.

The practical lesson runs back to transcription. A recording is weaker evidence of a voice than it used to be, and a speaker label was never evidence of a person.

What an operator can check

Keep the server on loopback. If you run llama-server for transcription, leave it bound to 127.0.0.1 unless another machine truly needs it. Audio in, text out, nowhere else.
Set --reasoning off for this model in llama-server, and confirm you get a transcript, not just reasoning text.
Treat <spk:N> as "voice N in this file." Never carry a label across recordings, and never attach a name without a human who knows the voices.
Spot-check against the audio. Before a transcript is quoted, shared or filed, listen to the passages that matter, above all in a "Supported"-tier language.
Delete what you don't need. Local doesn't mean safe from a seized laptop. Decide how long the recording and the transcript live, and where.
Prefer predefined voices for any generated speech, and label generated audio as generated.

Elsewhere this week

Two smaller llama.cpp v0.9.0 fixes fit our earlier notes on verifying what you load. The GGUF split-merge tool now rejects shards that don't belong together (PR 31579), and -hf downloads no longer match a quant tag inside a longer quant name (PR 31620). Both mean the file you asked for is more likely to be the file you got. The release also adds an FP8 quantisation type and Mistral Large 4 support.

Ollama shipped v0.40.14 and v0.40.15 on 3 and 4 Nov. The latter adds Xiaomi's 9B MiMo v2.6 coding model and lets recurrent and vision GGUF models serve more than one request at a time. It also adds "AutoTrust" models that return yes/no probabilities for questions you supply. A probability from a classifier is still a model's guess, not a decision anyone made.

What this is not

This note doesn't rank transcription models or tools, and the accuracy figures above are the contributor's and NVIDIA's, not ours. It isn't legal advice on recording people. Consent and recording laws differ widely across countries, so check yours. It doesn't claim local transcription is private by default; that depends on how you bind the server, store files and secure the device. It doesn't claim speaker attribution can identify anyone, and it isn't a guide to voice cloning. Nothing here is investment, token, custody, swap or trading advice, and nothing recommends buying any product or service.

Frame: Offline transcription moves the recording out of someone else's data centre. It doesn't move the burden of checking off you. The audio stays home, and so does the responsibility.