Who

Label who said what in audio and video.

On-device speaker labeling: per-person turns with timestamps.

• SDKs, install and examples: https://desertant.com/docs/who/
• Website: https://desertant.com/models/who/

Label the speakers in a recording and see exactly when each person starts and stops talking. A 50-minute interview takes 3s on a MacBook Pro, entirely on the device.

Use the timestamps to label a transcript by speaker, jump to the next time a guest talks, or cut every time the speaker changes. Who is a speaker diarization model made for podcasts, interviews, calls and meetings, and handles recordings of any length.

Who numbers the speakers in the order they first speak, and when two people talk at once, Who marks both of them. Who processes each five-minute section in one pass and matches voices across sections. Who runs on the Neural Engine on Apple platforms and through LiteRT on Linux, Windows and Android, and there's also an ONNX build. The Core ML download is 81MB as float16 or 42MB with 8-bit weights, and both builds have the same error rate on AMI meetings, 12.6%.

Platforms	iOS, macOS, tvOS, visionOS, Android, Linux, Windows, Browser, Node
Weights	v0.3.4

Install

Swift (requirements)

.package(url: "https://github.com/Desert-Ant-Labs/desert-ant-core.git", from: "4.2.1")

Then add the Who product to your target.

Kotlin (requirements)

implementation("ai.desertant:who:4.2.1")

JavaScript (requirements)

npm i @desert-ant-labs/who @litertjs/core   # browser
npm i @desert-ant-labs/who                  # Node, prebuilt native core

Files

Each folder holds the same five model graphs for its runtime, plus its own copy of the four supporting files; the LiteRT folder holds the five graphs as signatures of one file, named as below.

Directory	Runtime	Weights	Size
coreml-fp16/	Core ML, compiled	float16	81MB
coreml-int8/	Core ML, compiled	8-bit	42MB
litert-int8/	LiteRT	8-bit, the five graphs as signatures of one who.tflite	32MB
onnx-int8/	ONNX Runtime	8-bit matrix products, the rest float32	99MB

File	Contents
encoder_6000	From 60s of audio to voice frames
queries_6000	The speakers in those 60s and when each one speaks
encoder_30000	From 300s of audio to voice frames
queries_30000	The speakers in those 300s and when each one speaks
voice_3256	A voice print of a 30s stretch, used to match speakers across sections
who_meta.json	The front end's geometry, each graph's input and output names, the decision thresholds
who_mel_banks.f32	The mel filterbank the encoders read, 80 by 257, little-endian
who_voice_banks.f32	The mel filterbank the voice graph reads
who_embed.f16	The voice graph's last layer, as a table

Inputs and outputs

• In: 16kHz mono audio of any length, from a few seconds up. The SDK converts other sample rates for you.
• Out: each speaker, numbered in the order they first speak, and when they start and stop talking. Times are in seconds, to the nearest 10ms. When two people talk at once, both are marked. Who labels up to four speakers in each five-minute section, and more across a whole recording.

Languages

Who works from the sound of each voice and has no language setting. Every score on this card was measured on English recordings.

Results

Measured through the SDK on the files in this repo, on whole recordings.

Corpus	Samples	Diarization error rate	Missed speech	False alarm	Speaker confusion
AMI meetings, headset mix	16	0.126	0.079	0.026	0.021
VoxConverse, up to 4 speakers	86	0.084	0.027	0.040	0.017
VoxConverse, 2-21 speakers	232	0.175	0.026	0.073	0.076
NOTSOFAR-1 meetings, far-field, up to 4 speakers	65	0.162	0.060	0.052	0.050
NOTSOFAR-1 meetings, far-field, 3-8 speakers	129	0.243	0.087	0.057	0.098
Earnings-21 calls, 2-20 speakers	44	0.277	0.059	0.055	0.163
DiPCo dinner parties, far-field	10	0.411	0.207	0.064	0.140
CHiME-6 dinner parties, far-field	2	0.399	0.190	0.081	0.128

On VoxConverse, the error rate rises with the number of speakers in a recording.

Speakers	Samples	Diarization error rate
1 to 2	46	0.077
3	18	0.116
4	22	0.078
5 to 8	85	0.170
9 or more	61	0.247

Performance

Through the SDK.

Device	SDK	Workload	Result
MacBook Pro, M5 Pro, macOS 26.6	desert-ant-core 3.6.0, coreml	float16 build, 16 meetings, 544 minutes	950x faster than real time
MacBook Pro, M5 Pro, macOS 26.6	desert-ant-core 3.6.0, coreml	8-bit build, 16 meetings, 544 minutes	953x faster than real time
MacBook Pro, M5 Pro, macOS 26.6	desert-ant-core 3.6.0, coreml	float16 build, model loaded	275MB memory
MacBook Pro, M5 Pro, macOS 26.6	desert-ant-core 3.6.0, coreml	float16 build, an 8-minute recording	336MB peak memory
MacBook Pro, M5 Pro, macOS 26.6	desert-ant-core 3.6.0, coreml	float16 build, a 34-minute recording	437MB peak memory
MacBook Pro, M5 Pro, macOS 26.6	desert-ant-core 3.6.0, coreml	float16 build, a 2.65-hour recording	927MB peak memory

On CPU alone, the 8-bit LiteRT build runs the same 544 minutes in 87s on the M5 Pro and 191s on a 24-core AMD EPYC server. These two times were measured outside the SDK.

Limitations

• More than four speakers in five minutes. Who splits a recording into five-minute sections and can label up to four speakers per section. Who matches voices across sections, so it can identify more than four speakers in a longer recording. On VoxConverse, the error rate is 8% with one or two speakers, versus 17% with five to eight and 25% with nine or more.
• Far-field recordings. Who misses more speech when the speakers are far away from the microphone. In group conversations recorded across the room, Who misses 19-21% of the speech, versus 8% in meetings where everyone wears a headset. Who does best on close-miked audio, such as podcasts, interviews and calls. On room recordings, expect stretches of speech with no speaker label.
• Overlapping speech. Who misses the most speech when two people talk at once. On AMI meetings, Who misses 8% of all speech, and about 65% of that is overlapping talk. Expect more misses in lively discussions with lots of interruptions.
• Short stretches of speech. Who mixes up speakers more often when many people each speak for only a few seconds. On Earnings-21 calls, with up to 20 speakers, Who assigns 16% of the speech to the wrong speaker. Expect more mix-ups on calls with lots of short remarks.
• Laughter, applause and jingles. Who can give a laugh, applause or a jingle its own speaker number. To hide these, set the SDK to drop any speaker with less than a chosen share of the recording's speech.
• Who labels speakers as 1, 2, 3 and so on, not by name, and doesn't transcribe. For a transcript labeled by speaker, pair Who with Voz.

Credits

Open models and tools this work builds on, with their licenses:

• pyannote-community/speaker-diarization-community-1, CC BY 4.0

Dataset attributions are in THIRD_PARTY_NOTICES.md.

License

Desert Ant Labs Source-Available License. Free for most apps, and a commercial license is required at scale. Full terms are at the link. Licensing: licensing@desertant.com.

See THIRD_PARTY_NOTICES.md.

Citation

@software{who_2026,
  title  = {Who: On-device speaker labeling: per-person turns with timestamps},
  author = {Desert Ant Labs},
  year   = {2026},
  url    = {https://huggingface.co/desert-ant-labs/who},
}

---

© 2026 Desert Ant Labs · https://desertant.com