Arm/nemotron-3-diarization-litert-fp32

Nemotron 3 Diarization LiteRT FP32

This package contains a version of NVIDIA Nemotron 3 Diarization that runs with LiteRT and ExecuTorch.

The model identifies who speaks (up to eight speakers) and when.

The model is provided in FP32 without model-specific optimizations.

This implementation supports the following features:

• Offline speaker diarization.
• Streaming speaker diarization.

✨ Key Highlights

• Faster than real time — Using a 97.6-second audio sample on one Arm® CPU core:
  • 122.0× faster than real time on a MacBook Pro® computer with an M4 Pro chip.
  • 44.4× faster than real time on Vivo X300 Pro.

📦 Model Details

Model Description

Nemotron 3 Diarization assigns recording-local speaker labels to speech intervals, including overlapping speech, for up to eight speakers. The FP32 encoder uses LiteRT; the FP32 frontend uses ExecuTorch and the FP32 speaker cache uses LiteRT.

• Developed by: NVIDIA
• Base model: nvidia/Nemotron-3-Diarization
• License: OpenMDW-1.1, inherited from the upstream model
• Packaged variant: nemotron-3-diarization-litert-fp32
• Precision: FP32 throughout

Model Sources

• Pinned NVIDIA checkpoint
• Pinned Transformers implementation

🚀 Get Started with the Model

Run with Compute Flow

Follow these steps to run Nemotron 3 Diarization LiteRT FP32 on macOS® with Apple Silicon or on Linux® with an Arm®-based CPU, such as a Raspberry Pi® 5.

Step 1: Clone Compute Flow and set up the virtual environment

git clone https://github.com/ARM-software/ComputeFlow.git && cd ComputeFlow

uv sync && source .venv/bin/activate

Step 2: Install the brick and model

compute-flow update
compute-flow model install audio_detection:nemotron3-diarization nemotron-3-diarization-litert-fp32
compute-flow build native

Step 3: Run the Python CLI example

python -m cli_examples.audio_detection_cli \
  -m @manifest/nemotron3-diarization/nemotron-3-diarization-litert-fp32/nemotron3_diarization_manifest.json \
  -a @sample/diarization_example.wav \
  -t 1

The example identifies speakers in the bundled samples/diarization_example.wav recording. Replace -a with a path to your own WAV file.

📊 Quality evaluation

We have not conducted a formal diarization quality evaluation. In informal checks with the included sample, we did not observe significant speaker misclassification.

🎯 Performance evaluation

Performance was measured under the following conditions:

• Targets: MacBook Pro® computer with an M4 Pro chip and Vivo X300 Pro.
• CPU: One or two CPU threads.
• Runs: Five warmups and 30 measured runs per variant, target, and thread count, each in a fresh process.
• Run spacing: Five seconds between processes.
• Input: The complete 97.6-second, 16 kHz mono sample recording.
• Processing latency: Sum of native run() scopes from input processing through diarization output construction. It excludes model loading, setup, warmup, audio decoding, and CLI formatting.
• Memory: Average memory is mean sampled process resident set size (RSS); peak memory is maximum sampled RSS on macOS® and maximum sampled high-water-mark RSS on Android™. Sampling interval was 0.2 seconds.

Apple M4 Pro

CPU threads	P50 processing	P90 processing	P99 processing	Average RSS	Peak RSS
1	800.00 ms	823.24 ms	836.43 ms	801.41 MiB	1169.12 MiB
2	654.12 ms	659.16 ms	700.22 ms	740.38 MiB	1146.69 MiB

Vivo X300 Pro

CPU threads	P50 processing	P90 processing	P99 processing	Average RSS	Peak RSS
1	2198.99 ms	2235.55 ms	2262.93 ms	870.97 MiB	1054.39 MiB
2	2063.27 ms	2081.42 ms	2105.80 ms	873.82 MiB	1054.70 MiB

🛠️ Technical Specifications

Objective

Assign recording-local speaker IDs to speech intervals, including simultaneous speakers.

Runtime Architecture

Component	Framework / format	Precision
Audio frontend	ExecuTorch / PTE	FP32
Encoder	LiteRT / TFLite	FP32
Speaker cache	LiteRT / TFLite	FP32

Precision and Quantization

All model components use FP32. No quantization is applied.

Input Specification

Input	Description
Audio recording	Mono 16 kHz WAV audio.

Output Specification

The model returns recording-local speaker labels, speech intervals in seconds, and confidence scores. Overlapping speakers can have separate intervals.

Repository Contents

• README.md — this model card.
• metadata.yaml — package identity.
• nemotron3_diarization_manifest.json — model and asset contract.
• frontend.pte — FP32 audio frontend.
• encoder.tflite — FP32 encoder.
• speaker_cache.tflite — FP32 speaker cache.
• assets/class_names.json — recording-local speaker display names.
• samples/diarization_example.wav — 97.6-second benchmark sample.
• SHA256SUMS — checksums for the model bundle.

🗂️ Model and Asset Origin

The model exports derive from the pinned NVIDIA checkpoint and Transformers implementation. Model materials use OpenMDW-1.1; the Transformers implementation uses Apache-2.0.

The included sample WAV was decoded to mono 16 kHz from diarization_example.mp3 in hf-internal-testing/dummy-audio-samples.

🔐 Checksums

SHA256SUMS covers every regular file in the model bundle, including subdirectories, except the generated root SHA256SUMS and paths containing a dotfile component. After retrieving Git LFS objects, verify the package from the repository root:

shasum -a 256 -c SHA256SUMS