Instructions to use patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE
# Run inference directly in the terminal:
llama cli -hf patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE
# Run inference directly in the terminal:
llama cli -hf patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE
# Run inference directly in the terminal:
./llama-cli -hf patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE
# Run inference directly in the terminal:
./build/bin/llama-cli -hf patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE

Use Docker

docker model run hf.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE

Llama new
LM Studio
Jan

vLLM

How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE

Ollama
How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF with Ollama:
```
ollama run hf.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE
```
Unsloth Desktop

How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF with Docker Model Runner:
```
docker model run hf.co/patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE
```

Lemonade

How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE

Run and chat with the model

lemonade run user.MiMo-V2.6-Flash-REAP50-GGUF-MXFP4_MOE

List all available models

lemonade list

Hermes Agent

How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF:MXFP4_MOE" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MiMo-V2.6-Flash — REAP-50 (GGUF `MXFP4_MOE` / `Q3_K_S` / `Q2_K` + `mmproj` + `mtp`)

GGUF quantizations (MXFP4_MOE 86.7 GiB, Q3_K_S 77.7 GiB, Q2_K 64.8 GiB) + multimodal projector (mmproj) + 3-layer MTP draft head of 50%-pruned XiaomiMiMo/MiMo-V2.6-Flash (315B → 164B total, 11B active).

Sibling Artifact	Format	Size	Description
`MiMo-V2.6-Flash-REAP50`	Native MXFP4 + FP8 + BF16 (`safetensors`)	85.8 GiB	Bit-exact 128-expert master checkpoint (with vision/video/audio encoders + 3 MTP layers)
`MiMo-V2.6-Flash-REAP50-GGUF` (this repo)	GGUF (`MXFP4_MOE`, `Q3_K_S`, `Q2_K`, `mmproj`, `mtp`)	64.8–86.7 GiB	Single-file GGUFs + multimodal projector + 3-layer MTP draft head

GGUF quantizations of patrickbdevaney/MiMo-V2.6-Flash-REAP50 — a 50% REAP expert-pruned derivative of XiaomiMiMo/MiMo-V2.6-Flash (315B → 164B total parameters, 11B active parameters per token, 256 → 128 routed experts per MoE layer across all 59 MoE layers).

✨ Highlights

Lossless Native MXFP4_MOE GGUF (86.7 GiB): Packs the upstream OCP MXFP4 (E2M1 nibbles + E8M0 scales) expert weights directly into GGML_TYPE_MXFP4 blocks with zero re-quantization loss on any of the 128 surviving experts per layer, paired with Q8_0 hybrid MLA attention (full + SWA) and Q8_0 Layer-0 dense MLP.
Compact Q3_K_S (77.7 GiB) & Q2_K (64.8 GiB) Variants: Dequantized from MXFP4 and re-quantized into GGML_TYPE_Q3_K (3.4375 bpw) and GGML_TYPE_Q2_K (2.625 bpw) while keeping all attention, embeddings, output head, and Layer-0 dense MLP at Q8_0 (and all norms/routing biases at F32).
Complete Companion Artifacts Included (mmproj, mtp, imatrix): Ships with MiMo-V2.6-Flash-REAP50-mmproj-BF16.gguf (4.22 GiB vision/video/audio encoders), MiMo-V2.6-Flash-REAP50-mtp-Q8_0.gguf (1.64 GiB 3-layer MTP speculative decoding head), and MiMo-V2.6-Flash-REAP50.imatrix (75.8 MiB).
Transparent Architectural & Evaluation Status Disclosures: Full documentation of the 6-bucket group-balanced REAP-50 keep-set (n_group=8, 16 experts/group) and the custom mimo2 GGUF tensor requirements for llama.cpp.

📋 Model Summary

Property	Value
Base Model	`XiaomiMiMo/MiMo-V2.6-Flash` (315B total, 11B active, Apache-2.0)
Master Safetensors Repo	`patrickbdevaney/MiMo-V2.6-Flash-REAP50` (85.8 GiB)
Architecture (`general.architecture`)	`mimo2` (60 layers: Layer 0 dense + Layers 1–59 MoE; hybrid full MLA + 512-token SWA MLA)
MoE Configuration	128 routed experts/layer (`n_group = 8`, `topk_group = 4`, 16 experts/group, `top-8` active, `routed_scaling_factor = 2.827`)
Dense & Attention Path	`Q8_0` on all 60 MLA layers, Layer 0 dense MLP, `token_embd`, and `output`; `F32` on all norms, `attn_scale`, and `exp_probs_b`
Main GGUF Variants	`MXFP4_MOE` (86.7 GiB), `Q3_K_S` (77.7 GiB), `Q2_K` (64.8 GiB)
Companion Files	`mmproj-BF16.gguf` (4.22 GiB), `mtp-Q8_0.gguf` (1.64 GiB), `MiMo-V2.6-Flash-REAP50.imatrix` (75.8 MiB)
License	Apache-2.0

Corrections & evaluation status (2026-10-24)

Q3_K_S and Q2_K were quantised with the custom Python/CUDA converter (scripts/quantize_gguf_variants.py), not llama-quantize --imatrix.
- Because stock llama.cpp does not yet support the mimo2 hybrid-MLA architecture (see Runtime Compatibility below), llama-quantize could not be used to produce Q3_K_S or Q2_K.
- Instead, scripts/quantize_gguf_variants.py dequantised the MXFP4_MOE expert tensors (blk.*.ffn_{gate,up,down}_exps.weight) and packed them directly into GGML_TYPE_Q3_K (3.4375 bpw) and GGML_TYPE_Q2_K (2.625 bpw) using standard min-max / absmax block scales on GPU.
- The file MiMo-V2.6-Flash-REAP50.imatrix (75.8 MiB, 357 entries) is published in this repository for downstream use once llama.cpp lands mimo2 support, and contains real activation $\mathbb{E}[x^2]$ statistics collected during the 610,304-token REAP calibration pass — however, those importance weights were NOT used when computing the scales inside the currently uploaded Q3_K_S.gguf and Q2_K.gguf files. An earlier bullet on this card described MiMo-V2.6-Flash-REAP50.imatrix as "used for Q3_K/Q2_K quantization"; that was inaccurate and has been corrected here.
No post-pruning router or weight "healing" was applied.
- Every surviving expert tensor in MiMo-V2.6-Flash-REAP50-MXFP4_MOE.gguf is a lossless repack of the upstream XiaomiMiMo/MiMo-V2.6-Flash OCP MXFP4 nibbles and E8M0 scales, and every router ffn_gate_inp.weight ([128, 4096]) and exp_probs_b.bias ([128]) is an exact unmodified slice of the 128 kept experts per layer.
Held-out teacher-vs-student KL / NLL benchmark status: not yet evaluated.
- Unlike GLM-5.3-Flash-REAP50-GGUF, we have not yet run a held-out teacher-vs-student KL/NLL evaluation suite on MiMo-V2.6-Flash-REAP50 (neither on the safetensors checkpoint nor on these GGUF files), because MiMo-V2.6-Flash's custom mimo_v2_flash architecture is not yet supported by llama.cpp or our standalone GGUF evaluation harness. Do not assume GLM-5.3-Flash's KL numbers transfer to MiMo-V2.6-Flash; we will publish a  table here once a verified teacher-vs-student evaluation has been run.

Files in This Repository

File	Size	Routed Experts (`ffn_*_exps`)	Attention & Layer 0 Dense MLP	Recommended Hardware & Notes
`MiMo-V2.6-Flash-REAP50-MXFP4_MOE.gguf`	86.1 GiB (`88,123 MiB`)	Lossless `MXFP4` (4.25 bpw, bit-exact repack of upstream OCP MXFP4)	`Q8_0` (`F32` norms & scales)	128 GB (Jetson AGX Thor, DGX Spark, Mac Studio, Strix Halo)
`MiMo-V2.6-Flash-REAP50-Q2_K.gguf`	61.6 GiB (`63,122 MiB`)	`Q2_K` (2.63 bpw)	`Q8_0` (`F32` norms & scales)	80–96 GB (fits 96 GB Mac / RTX PRO 6000 96GB / 128 GB edge)
`mmproj-MiMo-V2.6-Flash-REAP50-BF16.gguf`	2.56 GiB (`2,621 MiB`)	—	`BF16` / `F32`	Vision (`vision_tower`), Video (`video_tower`), Audio (`audio_tower`) + Multimodal Aligners (BF16)
`mmproj-MiMo-V2.6-Flash-REAP50-Q8_0.gguf`	1.46 GiB (`1,492 MiB`)	—	`Q8_0` / `F32`	Quantized `Q8_0` Multimodal Projector (`vision` / `video` / `audio`)
`mtp-MiMo-V2.6-Flash-REAP50-BF16.gguf`	4.17 GiB (`4,274 MiB`)	—	`BF16` / `F32`	3-layer Multi-Token Prediction (`mtp.layers.0..2`) speculative decoding draft head (BF16)
`mtp-MiMo-V2.6-Flash-REAP50-Q8_0.gguf`	2.22 GiB (`2,273 MiB`)	—	`Q8_0` / `F32`	3-layer Multi-Token Prediction (`mtp.layers.0..2`) speculative decoding draft head (`Q8_0`)

How These GGUFs Were Built

Domain-Balanced REAP-50 Pruning (256 → 128 experts per layer, 16 per group across 8 groups):
- Calibrated over 610,304 tokens (298 × 2048) across 6 buckets (general, code, math, zh, tool, and synthetic multimodal embeddings) on an NVIDIA Jetson AGX Thor (128 GB) using scripts/mimo_saliency.py and scripts/reap_select.py.
- Within each of the 59 MoE layers, the union of the top-16 experts per domain is locked as protected domain specialists (35–75 specialists locked per layer), and the remaining slots in each of the 8 routing groups (32 → 16 experts/group) are filled by cross-domain normalized REAP score ($\text{REAP}j = \frac{1}{|T_j|} \sum{t \in T_j} g_{t,j} |\text{FFN}_j(x_t)|_2$).
Lossless MXFP4_MOE Conversion (MiMo-V2.6-Flash-REAP50-MXFP4_MOE.gguf):
- Upstream XiaomiMiMo/MiMo-V2.6-Flash stores routed experts in OCP MXFP4 (weight_packed [out, in/2] U8 + weight_scale [out, in/32] F8_E8M0), where consecutive nibbles (low=col 2i, high=col 2i+1) sit in each byte.
- GGML_TYPE_MXFP4 uses 17-byte blocks (1 byte E8M0 scale + 16 bytes data for 32 elements) where low nibble = element j (0..15) and high nibble = element j+16 (16..31).
- Our converter transposes the nibble layout inside each 32-element block losslessly — zero re-quantization error on any of the 128 surviving experts per layer.
- All 60 hybrid MLA attention layers and Layer 0's dense MLP are dequantized from upstream FP8 (F8_E4M3 × F32 128×128 block scale) and quantized to Q8_0, preserving high attention fidelity across all variants.
High-Attention-Fidelity Q2_K Variant (MiMo-V2.6-Flash-REAP50-Q2_K.gguf):
- Only the 3D routed expert tensors (blk.*.ffn_{gate,up,down}_exps.weight) are quantized to Q2_K using per-block/super-block absmax/min-max scaling.
- All attention layers (attn_q_a, attn_q_b, attn_kv_a_mqa, attn_kv_b, attn_output), Layer 0 dense FFN, token_embd, and output remain at Q8_0 (and all norms, attn_scale, and exp_probs_b remain at F32).

Runtime Compatibility Note (`mimo2` Hybrid MLA Architecture)

XiaomiMiMo/MiMo-V2.6-Flash uses the new MiMoV2FlashForCausalLM (mimo_v2_flash) architecture introduced in April 2026 (surveyed in research/MIMO_V26_FLASH_SURVEY_2026-09-23.md), which combines:

Full Multi-head Latent Attention (MLA) on layers [0, 5, 11, 17, 23, 29, 35, 41, 47, 53, 59] (kv_lora_rank=512, qk_nope_head_dim=192, qk_rope_head_dim=64, v_head_dim=256, 64 heads)
Sliding-Window MLA (SWA, window=512) on the remaining 49 layers (swa_kv_lora_rank=256, swa_qk_nope_head_dim=128, swa_qk_rope_head_dim=64, swa_v_head_dim=128, 48 heads, with per-head learned attention_scale)
Value Residual (v_first + v_mix_logit) across SWA layers and Q/K LayerNorm (q_norm, k_norm) inside MLA

These GGUFs use general.architecture = "mimo2" with canonical llama.cpp DeepSeek2/MLA tensor naming (blk.{i}.attn_q_a, attn_q_b, attn_kv_a_mqa, attn_kv_b, attn_q_norm, attn_k_norm, attn_scale, attn_v_mix, layer_scale, ffn_gate_inp, exp_probs_b, ffn_{gate,up,down}_exps) plus full per-layer hyperparameter arrays in GGUF metadata so that llama.cpp PRs adding mimo2 (hybrid MLA + SWA + v_mix) can load them directly without re-conversion.

For immediate inference with vLLM, SGLang, or Transformers, use the companion safetensors repository patrickbdevaney/MiMo-V2.6-Flash-REAP50.

🔎 Provenance & Verification Ledger

Every file, architectural parameter, and pruning step on this card can be audited directly against the files in this repository and the companion open-source pipeline:

Claim / Artifact	Primary Verification Source
Upstream Base Model (`315B` MXFP4+FP8, Apache-2.0)	`XiaomiMiMo/MiMo-V2.6-Flash`
Bit-Exact REAP-50 Master Checkpoint (`85.8 GiB`)	`patrickbdevaney/MiMo-V2.6-Flash-REAP50`
REAP Pruning Formulation	Lasby et al., 2025 — REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression (`arXiv:2510.13999`)
Main GGUF Model Files (`MXFP4_MOE`, `Q2_K`)	`./MiMo-V2.6-Flash-REAP50-MXFP4_MOE.gguf` · `./MiMo-V2.6-Flash-REAP50-Q2_K.gguf`
Multimodal Projector & MTP Draft Head GGUFs	`./mmproj-MiMo-V2.6-Flash-REAP50-BF16.gguf` · `./mmproj-MiMo-V2.6-Flash-REAP50-Q8_0.gguf` · `./mtp-MiMo-V2.6-Flash-REAP50-BF16.gguf` · `./mtp-MiMo-V2.6-Flash-REAP50-Q8_0.gguf`
MiMo-V2.6-Flash REAP-50 Pipeline Repository	`patrickbdevaney/xiaomi-2.6-flash-REAP` (`main` @ `329123f`) · `wiki/README.md` · `research/MIMO_V26_FLASH_SURVEY_2026-09-23.md`
Saliency Streaming, Selection & Mask Application	`scripts/mimo_saliency.py` · `scripts/reap_select.py` · `scripts/apply_mask.py` · `scripts/mimo_shards.py`

Citation & License

License: Apache-2.0 (inherited from XiaomiMiMo/MiMo-V2.6-Flash)
Pruning Method: REAP: Router-weighted Expert Activation Pruning (arXiv:2510.13999)
REAP-50 Pipeline: patrickbdevaney/xiaomi-2.6-flash-REAP

Downloads last month: 7,944

GGUF

Model size

157B params

Architecture

mimo2

Hardware compatibility

2-bit

4-bit

8-bit

16-bit

Paper for patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF

REAP the Experts: Why Pruning Prevails for One-Shot MoE compression

Paper • 2510.13999 • Published Oct 15, 2025 • 23

MiMo-V2.6-Flash — REAP-50 (GGUF MXFP4_MOE / Q3_K_S / Q2_K + mmproj + mtp)

✨ Highlights

📋 Model Summary

Corrections & evaluation status (2026-10-24)

Files in This Repository

How These GGUFs Were Built

Runtime Compatibility Note (mimo2 Hybrid MLA Architecture)

🔎 Provenance & Verification Ledger

Citation & License

Paper for patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF

MiMo-V2.6-Flash — REAP-50 (GGUF `MXFP4_MOE` / `Q3_K_S` / `Q2_K` + `mmproj` + `mtp`)

Runtime Compatibility Note (`mimo2` Hybrid MLA Architecture)