Instructions to use steampunque/GLM-Z1-32B-0414-MP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use steampunque/GLM-Z1-32B-0414-MP-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf steampunque/GLM-Z1-32B-0414-MP-GGUF # Run inference directly in the terminal: llama cli -hf steampunque/GLM-Z1-32B-0414-MP-GGUF
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf steampunque/GLM-Z1-32B-0414-MP-GGUF # Run inference directly in the terminal: llama cli -hf steampunque/GLM-Z1-32B-0414-MP-GGUF
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf steampunque/GLM-Z1-32B-0414-MP-GGUF # Run inference directly in the terminal: ./build/bin/llama cli -hf steampunque/GLM-Z1-32B-0414-MP-GGUF
Use Docker
docker model run hf.co/steampunque/GLM-Z1-32B-0414-MP-GGUF
- Llama new
- LM Studio
- Jan
- Ollama
How to use steampunque/GLM-Z1-32B-0414-MP-GGUF with Ollama:
ollama run hf.co/steampunque/GLM-Z1-32B-0414-MP-GGUF
- Unsloth Desktop
- Pi
How to use steampunque/GLM-Z1-32B-0414-MP-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf steampunque/GLM-Z1-32B-0414-MP-GGUF
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "steampunque/GLM-Z1-32B-0414-MP-GGUF" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use steampunque/GLM-Z1-32B-0414-MP-GGUF with Docker Model Runner:
docker model run hf.co/steampunque/GLM-Z1-32B-0414-MP-GGUF
- Lemonade
How to use steampunque/GLM-Z1-32B-0414-MP-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull steampunque/GLM-Z1-32B-0414-MP-GGUF
Run and chat with the model
lemonade run user.GLM-Z1-32B-0414-MP-GGUF-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use steampunque/GLM-Z1-32B-0414-MP-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf steampunque/GLM-Z1-32B-0414-MP-GGUF
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default steampunque/GLM-Z1-32B-0414-MP-GGUF
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use steampunque/GLM-Z1-32B-0414-MP-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf steampunque/GLM-Z1-32B-0414-MP-GGUF
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "steampunque/GLM-Z1-32B-0414-MP-GGUF" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Mixed Precision GGUF layer quantization of GLM-Z1-32B-0414 by zai-org
Original model: https://huggingface.co/zai-org/GLM-Z1-32B-0414
The hybrid quant employs different quantization levels on a per layer basis to enable both high performance and small file size at the same time. An extended layer definition E quant Q4_E_H for the model is defined as follows (updated 10/12/2026):
LAYER_TYPES='[
["A","attn","Q","attn_q","K","attn_k","V","attn_v","O","attn_o","S","ssm","F","ffn","G","ffn_g","U","ffn_u","D","ffn_d"],
["MAP","VOD","0","QN_K","2","Q2_K","3","Q3_K","4","Q4_K","5","Q5_K","6","Q6_K","8","Q8_0","h","F16","f","F32"],
[0 ,"Q5_K_886"],[1 ,"Q5_K_665"],[2 ,"Q4_K_655"],[3 ,"Q4_K_655"],
[4 ,"Q4_K_644"],[5 ,"Q4_K_644"],[6 ,"Q4_K_644"],[7 ,"Q4_K_644"],
[8 ,"Q4_K_544"],[9 ,"Q4_K_544"],[10,"Q4_K_544"],[11,"Q4_K_544"],
[12,"Q4_K_544"],[13,"Q4_K_544"],[14,"Q4_K_544"],[15,"Q4_K_544"],
[16,"Q4_K_544"],[17,"Q4_K_544"],[18,"Q4_K_544"],[19,"Q4_K_544"],
[20,"Q4_K_544"],[21,"Q4_K_544"],[22,"Q4_K_544"],[23,"Q4_K_544"],
[24,"Q4_K_544"],[25,"Q4_K_544"],[26,"Q4_K_544"],[27,"Q4_K_544"],
[28,"Q4_K_544"],[29,"Q4_K_544"],[30,"Q4_K_544"],[31,"Q4_K_544"],
[32,"Q4_K_544"],[33,"Q4_K_544"],[34,"Q4_K_544"],[35,"Q4_K_544"],
[36,"Q4_K_544"],[37,"Q4_K_544"],[38,"Q4_K_544"],[39,"Q4_K_544"],
[40,"Q4_K_544"],[41,"Q4_K_544"],[42,"Q4_K_544"],[43,"Q4_K_544"],
[44,"Q4_K_544"],[45,"Q4_K_544"],[46,"Q4_K_544"],[47,"Q4_K_544"],
[48,"Q4_K_544"],[49,"Q4_K_544"],[50,"Q4_K_544"],[51,"Q4_K_544"],
[52,"Q4_K_555"],[53,"Q4_K_445"],[54,"Q4_K_655"],[55,"Q4_K_655"],
[56,"Q4_K_655"],[57,"Q4_K_666"],[58,"Q4_K_666"],[59,"Q5_K_668"],
[60,"Q6_K_886"]
]'
FLAGS="--token-embedding-type Q4_K --output-tensor-type Q6_K --layer-types-high"
The layer quants were optimized with smallest quant across layers at Q4_K to minimize entropy loss and provide extremely strong performance across a set of reasoning prompts using greedy sample. The final bpw results in ~0.5G smaller file size than Q4_K_M.
Comparison:
| Quant | size | PPL | Comment |
|---|---|---|---|
| Q4_K_M | 19.7e9 | 7.7 | default embedding/output |
| Q4_E_H | 19.3e9 | 7.7 | Hybrid E quant with Q4_K embedding Q6_K output |
Usage:
This is a RL trained thinking model. The layer quants for this model were evaluated on a set of test/eval prompts using greedy sampling. The quant/model show excellent performance in providing both minimal overthinking and/or getting stuck in infinite generations.
The model can be speculated using Qwen3 0.6B if the inference engine can support dynamic vocab translation between draft and target models. Approx performance using a downstream speculator with llama.cpp on two 4070s (12G VRAM) with layers and context fully in GPU and connected via RPC:
| Q | QKV | ND | NKV | gen tps | Comment |
|---|---|---|---|---|---|
| Q4_E_H | F16 | 0 | 32k | 22 | No draft |
| Q4_E_H | F16 | 4 | 16k | 42 | Spec 4 |
| Q4_E_H | Q8_0 | 4 | 18k | 41 | Spec 4 |
Benchmarks:
Math benchmarks for the model are given here: https://huggingface.co/spaces/steampunque/benchlm
Download the file from below:
| Link | Type | Size/e9 B | Notes |
|---|---|---|---|
| GLM-Z1-32B-0414.Q4_E_H.gguf | Q4_E_H | 19.3e9 B | ~0.5B smaller than Q4_K_M |
A discussion thread about the hybrid layer quant approach can be found here on the llama.cpp git repository:
- Downloads last month
- 28
We're not able to determine the quantization variants.
Model tree for steampunque/GLM-Z1-32B-0414-MP-GGUF
Base model
zai-org/GLM-Z1-32B-0414