DJLougen/LFM2.5-VL-3B-DSpark-MLX-8bit

LFM2.5-VL-3B-DSpark MLX drafter (8-bit)

What

An MLX-native conversion of Liquid AI's four-layer LFM2.5-VL-3B-DSpark DFlash drafter. Draft transformer linears (four draft layers and the fc projection) use affine 8-bit quantization with group size 64; the Markov and confidence heads and normalization weights remain BF16. This is a speculative-decoding drafter, not a standalone model. Model file: model.safetensors (358,408,820 bytes), SHA-256 1c99bd3cf35eb0b651c61ce44596e28086cfa489499ea7eae3cafc0f992aeca4.

Upstream source: LiquidAI/LFM2.5-VL-3B-DSpark, pinned at revision af77e9306a26e8625fde74d2a3051ab6d21bd955. Conversion and benchmark code: DJLougen/lfm25-vl-3b.

Why

The original DSpark checkpoint uses an SGLang-oriented layout. This conversion provides MLX-native tensors and config metadata for the MLX-VLM DFlash runtime, with a smaller draft model than the original BF16 draft.

Why it matters

A DFlash drafter proposes multiple tokens for the target to verify. Greedy target verification preserves the target model's output; the practical speedup depends on how many proposals are accepted and on the target's own speed. This 8-bit drafter was tested against both BF16 and 8-bit MLX targets, and the results differ substantially.

Results

Single-machine measurement on an Apple M3 Max with 36 GB unified memory, Python 3.12, mlx 0.32.2, mlx-vlm 0.7.3, and transformers 5.17.0. Short-prompt screen: three instructions against the same image, greedy generation, two repetitions per prompt/configuration. The run with the lowest end-to-end wall time is retained per prompt/configuration, then throughput and ratios are averaged across prompts; best-of-two selection may be optimistic. results/sweep.json contains the selected rows only, not both repetitions.

Target precision	Draft block	Target-only decode (mean)	With this drafter (mean)	Decode speed ratio	End-to-end ratio	Accepted tokens/round
BF16	8	43.6 tok/s	131.9 tok/s	3.02x	2.44x	4.30
8-bit	4	74.9 tok/s	82.2 tok/s	1.11x	1.09x	2.99

The selected draft output for each prompt matched its target-only text. These are local measurements, not a vendor benchmark; the small 8-bit-target gain may not hold on another workload or device. With the 8-bit target, the tested block-8 setting was slower than target-only. Benchmark your own inputs before enabling speculation.

How to use

Install compatible Apple-Silicon MLX and MLX-VLM builds, then download this drafter and the target checkpoint. For the requested 8-bit target, block 4 was the best tested draft size:

python -m mlx_vlm.generate \
  --model LiquidAI/LFM2.5-VL-3B-MLX-8bit \
  --draft-model DJLougen/LFM2.5-VL-3B-DSpark-MLX-8bit \
  --draft-block-size 4 --temperature 0 \
  --image image.jpg --prompt "Describe this image." --max-tokens 128

The drafter requires mlx-vlm >= 0.7.2; results above used mlx-vlm==0.7.3. The tested DFlash path is greedy-only, so use temperature zero. To compare against target-only, remove the --draft-model and --draft-block-size arguments.

Known issues

• This is an MLX conversion for the LFM2.5-VL-3B target family; it is not a general-purpose Qwen3 drafter.
• The measured 8-bit-target improvement was small (about 9% end-to-end over a three-prompt screen); do not treat it as a guaranteed gain.
• Block 8 was slower than target-only with the 8-bit target in the same screen.
• The long-context sweep used the BF16 drafter, not this quantized drafter, and showed speculation slower than target-only at 32k and 64k book-context tokens. Long-context performance for this 8-bit draft is unestablished.
• No SGLang runtime or upstream benchmark result is claimed.

License

The draft weights and upstream-derived model artifacts are distributed under Liquid AI's LFM Open License v1.0, following the upstream DSpark repository. The linked source-code repository has a separate Apache-2.0 code license; it does not apply to these model weights.