realhidden/whistle-finetuned-hu

Whistle fine-tuned for Hungarian

The full story is in the blog post Whistle Hungarian on the stormregion blog.

This is a fine-tune of Cactus-Compute/whistle that teaches it Hungarian. The original model knows seven languages (English, German, French, Spanish, Italian, Dutch and Polish) and has no Hungarian. It is still a single on-device file (whistle.cact, 17.9 MB, the same 2-bit / 4-bit layout as the original) that runs on the original CPU engine.

It was made for a microcontroller port of Whistle on the M5Stack StickS3 (ESP32-S3, 8 MB flash), so it was trained for the small configuration that fits there: the 2-layer decoder rung (--audio-depth 2), without the engram tables.

This is a Hungarian-only model. Training used Hungarian data only, and the other seven languages no longer work (English FLEURS WER goes from 13% to 98%). If you need those, use the original model.

Results

All scores below come from the shipped whistle.cact, run on the official needle runner (macOS arm64, cactus-needle 3.2.6) with --audio-depth 2, greedy decoding. WER and CER are computed after the Whisper basic normalizer (English: the Whisper English normalizer). Lower is better. Each set is 100 clips, sampled evenly from the test split.

Test set	Original Whistle	This model
Common Voice 19 hu test, WER / CER	139.5% / 71.8% (no Hungarian)	21.4% / 6.8%
FLEURS hu_hu test, WER / CER	149.2% / 87.8% (no Hungarian)	36.4% / 14.4%
FLEURS en_us test, WER	13.4%	98.3%

The original model was run at full depth with language detection, since it has no hu tag; it labelled the Hungarian clips as Italian, German, English and the others. This model detected Hungarian on 200 of 200 clips.

Larger evaluation from the training side (JAX, the same 2-bit / 4-bit numerics, depth-2 rung, engrams off, greedy, up to 1,000 clips per set):

Test set	Clips	WER	CER
Common Voice 19 hu test	1,000	27.8%	8.6%
VoxPopuli hu test	1,000	19.1%	8.1%
FLEURS hu_hu test	898 (all)	40.1%	19.0%
Held-out university lecture	214	28.3%	13.2%

The lecture references are Whisper pseudo-labels, not human transcripts, so that row is only a rough guide.

Usage

The model must run at decoder depth 2. At the default full depth (8 layers) the output is unusable (95% WER on the sets above). The needle runner exposes the depth; the Python needle.Whistle wrapper and the C header in cactus-needle 3.2.6 do not, so use the runner:

pip install cactus-needle
needle download macos-arm64 --out .        # or your platform folder; puts the `needle` runner in ./macos-arm64
hf download realhidden/whistle-finetuned-hu whistle.cact --local-dir .

./macos-arm64/needle --model whistle.cact --audio clip.wav --audio-depth 2 --audio-language hu

{"text":"Ennek ellenére tartsa be a hatóságok tanácsait, kövesse a táblák utasításait és fordítson nagy figyelmet a biztonsági figyelmeztetésekre.","language":"hu","ttft_ms":14.3,"decode_tps":1306.4}

The runner accepts any WAV and resamples it. Without --audio-language hu the language is detected, which picked Hungarian on every test clip.

Training

• Method: LoRA (rank 256, alpha 512) on the attention projections (q, k, v, gate, out) of the encoder and the two kept decoder layers, and on the encoder's conv-module pointwise projections. The base weights stay frozen. Training ran on the depth-2 decoder rung (layers 0 and 7) with the engram tables off, as on the device, with fake quantisation at the published archive's bit widths (2-bit kernels, 4-bit embedding and mHC). An adapter trained at 4-bit only collapsed to 106% WER once exported at 2-bit.

• Vocabulary: the original tokenizer splits Hungarian into 4.9 tokens per word. 3,354 Hungarian BPE pieces (from a 4,000-piece model trained on the transcripts) were added with merge priority over the base vocabulary, which cut this to 2.4 tokens per word. A <|hu|> language tag was added. Vocabulary: 8,199 → 11,554 tokens.

• Data: about 1,300 hours of unique audio, 1,609 hours per epoch with the repeats below.

  Source	License	Hours	Clips	Weight
  WorldSpeech hu_hu (Hungarian parliament), clips with the dataset's CER score < 0.15	CC BY-NC 4.0	1,136.5	266,772	1×
  Common Voice 19 hu train	CC0	70.7	48,855	3×
  VoxPopuli hu train	CC0	51.3	17,592	3×
  University lecture recordings from the author's own course (youtube.com/@thisisdone), pseudo-labels from Whisper large-v3-turbo and YouTube captions	—	21.5	5,714	3×
  FLEURS hu_hu train + dev	CC BY 4.0	10.5	3,468	3×
  CSS10 Hungarian (single speaker)	CC0	10.0	4,513	1×

  • Validation: 789 clips from Common Voice dev and VoxPopuli validation.
  • No test split was trained on: not Common Voice test, VoxPopuli test, FLEURS test or the held-out lecture.
  • No data from the original seven languages was mixed in, which is why they no longer work.

• Schedule: three stages, each AdamW (no weight decay) with warmup and cosine decay to 5% of the peak learning rate, gradient clipping at 1.0, and length-bucketed batches of 120–200 s of padded audio. The new vocabulary rows train together with the LoRA weights at the same learning rate.

  1. Common Voice + VoxPopuli (122 h), learning rate 3e-4, 9,000 steps.
  2. + WorldSpeech and lectures, learning rate 2e-4, 2 epochs (104,076 steps).
  3. + FLEURS train/dev, CSS10 and more lectures, learning rate 1e-4, 1.5 epochs (80,917 steps). This release is the end of stage 3.

• Hardware: a single NVIDIA RTX 3060 (12 GB), about 8.5 hours of training in total.

Limitations

• Decoder depth: only depth 2 works (see Usage). Full depth is not supported.
• Other languages: English, German, French, Spanish, Italian, Dutch and Polish are broken. English input comes out as Hungarian-sounding text.
• Domain: most of the data is parliamentary speech (WorldSpeech, VoxPopuli), so it does best on read or formal speech. FLEURS (read Wikipedia sentences with many names and numbers) is the hardest set.
• Proper nouns: foreign names are often spelled phonetically ("Mitchell Gourley" → "Michael Gurley", "Szuper-G" → "szuperdgi").
• Long words: Hungarian compounds are sometimes split ("szigetvilágban" → "sziget világban"), which counts as two word errors.
• Loops: on rare clips the decoder repeats a fragment or emits broken characters until the token limit.

License and attribution

• Model: a derivative of Cactus-Compute/whistle by Cactus Compute, Inc., licensed under the Apache License 2.0 (LICENSE). Because WorldSpeech is CC BY-NC 4.0, this fine-tune is released for non-commercial use under CC BY-NC 4.0.
• Training data: WorldSpeech (CC BY-NC 4.0), Common Voice (CC0), VoxPopuli (CC0), FLEURS (CC BY 4.0), CSS10 (CC0).

If you use Whistle, please cite the original work:

@misc{whistle_2026,
  title        = {Whistle: Speech Recognition for Tiny Devices},
  author       = {Mroz, Jakub and Ndubuaku, Henry and Mosoyan, Karen and Cylich, Noah and
                  Kumar, Satyajit and Sandhu, Parkirat and Shemet, Roman and Lee, Justin H.},
  year         = {2026},
  organization = {Cactus Compute, Inc.},
  howpublished = {\url{https://github.com/cactus-compute/needle}}
}