Instructions to use barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit
MTPLX 3-bit conversion of barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MTP โ the REAP-pruned (205/256 experts) Qwen3.6-35B-A3B MoE merged with an Opus-reasoning-distilled LoRA, plus its native MTP layer โ for speculative decoding on Apple Silicon via mtplx.
Sibling formats: the bf16 safetensors source above and the GGUF collection for llama.cpp/Ollama/LM Studio.
Converted straight from the bf16 safetensors weights (not from any GGUF quant), so there is no dequantize-requantize drift in the chain.
Why this release exists
MTPLX grafts multi-token prediction (MTP) onto a standard checkpoint and exposes it through MLX on Apple Silicon: the draft head proposes 1โ3 tokens per step, the base model verifies them, and accepted tokens are emitted in a single forward pass. For this MoE (29B total, ~3B active per token) the 3-bit trunk cuts the on-disk footprint to ~13.8 GB and keeps the MTP sidecar in bf16, unlocking speculative speedups without a separate draft model.
This repo is the 3-bit affine point of that family. If you can afford the VRAM/unified-memory budget, the sibling 4-bit MTPLX build (when published) will be closer to lossless; for tight budgets this 3-bit file is the entry point.
Quality benchmarks (of the source safetensors model)
Measured with lm-evaluation-harness (HF backend, bfloat16, chat template disabled) on the underlying safetensors checkpoint. Same base chain as the GGUF card: unpruned Qwen3.6-35B-A3B โ REAP 205/256 (no LoRA, RangerX/Qwen3.6-35B-REAP-Pruned-ratio-0.2) โ this checkpoint.
| Task | Axis | Qwen3.6-35B-A3B | RangerX REAP-20% | This model |
|---|---|---|---|---|
| wikitext perplexity (lower=better) | sanity | 7.85 | 10.06 | 10.06 |
| MMLU | out-of-calibration | 0.8409 | 0.8152 | 0.8257 |
| HellaSwag (0-shot, acc_norm) | out-of-calibration | 0.7420 | 0.7440 | 0.7340 |
| ARC-Challenge (0-shot, acc_norm) | out-of-calibration | 0.5320 | 0.5340 | 0.6160 |
Key finding: the reasoning LoRA pushes ARC-Challenge above the unpruned 256-expert base (0.616 vs 0.532) โ a genuine skill transfer from the Opus chain-of-thought data. See the GGUF card and the safetensors card for full methodology.
Available MTPLX conversions
| Repo | Bits | Size (on disk) | Use case |
|---|---|---|---|
| this repo (3-bit) | 3 | ~13.8 GB (12.8 GiB) | tight unified memory, entry point |
| barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MTP | bf16 | ~56 GB | reference / re-quant source |
No separate mmproj/vision file โ this MoE is text-only.
Which one to pick:
- Best quality โ bf16 source
- Apple Silicon with โค24 GB unified memory โ this 3-bit MTPLX
- llama.cpp / Ollama / LM Studio workflows โ use the GGUF repo instead (IQ/MTP-GGUF quants)
Conversion notes
- Source:
barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MTP(commit9fa24b5, 16 shards +model-mtp.safetensors), forge-local viamtplx==2.9.1 - Recipe:
body_bits=3,body_group_size=64,body_mode=affine,mtp_policy=keep_bf16,contract_calibration_depth=1/limit=8/windows=4,concat_orders=embedding_hidden, MoE-aware (qwen3-next-mtp,mtp_sidecar=bf16-qwen-moe) - MTP contract:
base_hidden_variant=pre_norm,hidden_variant=post_norm,concat_order=embedding_hidden,mtp_position_mode=local,mtp_quant_group_size=64,mtp_quant_mode=affine - Output: 3 safetensors shards +
mtp.safetensors(1.3 GB, bf16 sidecar) +model.safetensors.index.json; architectureQwen3_5MoeForCausalLM(40 layers, hidden 2048, 205/256 experts, hybrid Gated-DeltaNet / full-attention) - Forged at: 2026-08-23T18:26:09+03:00 on Apple M5 Pro (18 CPU / 20 GPU, 24 GB unified memory) โ tuned for 24 GB Macs ยท macOS 27.0 arm64,
mtplx_runtime.jsonships in-repo as provenance
Vision
Not applicable โ text-only MoE. No model-vision.safetensors, no mmproj.
MTPLX usage
# install
pip install -U mtplx
# single-turn generation (auto-selects the verified depth if you omit --depth)
mtplx run --model barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit \
--prompt "Explain the Monty Hall problem briefly." \
--depth 1 --max-tokens 512
# OpenAI-compatible local server
mtplx serve --model barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit \
--depth 1 --port 8080
# then: curl http://localhost:8080/v1/chat/completions ...
# text-only chat (temperature 0.6, top_p 0.95, top_k 20 are the verified defaults)
mtplx run --model barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit \
--prompt "Write a short story about a robot learning to paint." \
--depth 1
Depth is the number of draft tokens proposed per step. Higher depth is not always faster โ use the verified optimum below.
Verification results
Verified locally with mtplx forge verify (MLX backend, greedy + quality gate, 512-token budget). Depth 0 is plain autoregressive (no MTP); depth N proposes N draft tokens.
| Depth | tok/s | vs AR | Acceptance by position | Quality |
|---|---|---|---|---|
| 0 (AR) | 91.5 | 1.00x | โ | pass |
| 1 | 101.5 | 1.08x | 58.2% | pass |
| 2 | 84.7 | 0.93x | 52.6% / 15.5% | pass |
| 3 | 71.8 | 0.78x | 50.3% / 17.4% / 2.4% | pass |
- Recommended depth: 1 (
mtp_depth_winsat D1; D2/D3 lose throughput on this MoE+3-bit combo despite passing quality) - Verdict:
mtp_depth_wins;quality_rejected=[],failure_reasons=[] - Hardware: Apple M5 Pro (18 CPU / 20 GPU, 24 GB) ยท macOS 27.0 arm64 (Apple Silicon, MLX), mtplx 2.9.1, artifact
sha256:6af5c44e... - All depths passed the quality gate; the gate is strict (rejects on drift), so any
quality_passed=truerow is safe to serve. Throughput is the tiebreaker.
Note: This MoE's head is unusually sensitive to quantization at 3-bit (single-position acceptance 58% vs 83โ98% on the Qwen3.8 dense family). Expect the best speedup at D1 only; the dense VLM siblings in this collection sustain D2/D3 at >2x.
Training details (source safetensors model)
- Base: Qwen/Qwen3.6-35B-A3B โ MoE, REAP-pruned to 205/256 experts
- Method: LoRA r=64, alpha=64, dropout 0.05, merged into base weights
- Data:
barozp/opus-reasoning-distill-train(Opus reasoning traces) - Vision + MTP: vision N/A; MTP layer carried over from base and kept in bf16 for MTPLX
Source chain
Qwen/Qwen3.6-35B-A3B (base) โ RangerX/Qwen3.6-35B-REAP-Pruned-ratio-0.2 (REAP 20% pruning) โ barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill (LoRA finetune) โ barozp/Qwen3.6-29B-REAP-Opus-Reasoning-Distill-MTP (MTP head) โ this repo (MTPLX 3-bit conversion)
- Downloads last month
- 68
3-bit
Model tree for barozp/Qwen3.6-29B-REAP-Opus-Distill-MTPLX-3bit
Base model
Qwen/Qwen3.6-35B-A3B