0xtb/ZGCM-1-7B-GGUF

ZGCM-1-7B - GGUF for llama.cpp (Apple Silicon)

These are GGUF conversions and quantizations of zgcagi/ZGCM-1-7B, a 7.39B-parameter model for math reasoning and agentic search from the Zhongguancun Academy and the Zhongguancun Institute of Artificial Intelligence.

The file called the "full-precision reference" below is an unquantized mixed F16/BF16 GGUF, not a byte-identical copy of the original BF16 checkpoint. Its 32 FFN down-projections remain BF16.

What is here:

• a numerically validated full-precision reference file;
• three quantization levels (Q8_0, Q6_K, Q5_K_M), each measured against that reference with evaluations designed and fixed before the results existed, from 129 tokens up to the model's full 262,144-token context;
• the importance matrix used for Q6_K and Q5_K_M, so the quantization can be reproduced.

Requirements.

• ZGCM needs a llama.cpp build with support for its architecture. It has hybrid sliding-window and global attention with an attention gate. As of 2026-09-28, this release requires the linked fork; compatibility with other builds is not established. Use the fork linked below; see "Quick start".
• Upstream status. This is a community fork, not an upstream llama.cpp release. No upstream support issue or pull request is claimed here.
• Everything was tested on Apple Silicon (M2 Max, Metal backend), plus a short CPU-backend check of the full-precision file. Other backends are untested; see "Known issues".

This is an independent community conversion. It is not affiliated with or endorsed by the model's authors.

---

Which file should I use?

File	Size	Bits/weight	Closeness to full precision	Recommended for
ZGCM-1-7B-BF16-downall32-F16.gguf	14.8 GB	16.0	reference	Maximum fidelity; 32 GB+ Macs
ZGCM-1-7B-BF16-down28-31-Q8_0.gguf	8.04 GB	8.7	near-lossless (top-1 agreement 98.2-99.6%)	16 GB+ Macs wanting near-reference quality
ZGCM-1-7B-BF16-down28-31-IMATRIX-Q6_K.gguf	6.29 GB	6.8	very close (top-1 94.8-98.9%)	Default choice
ZGCM-1-7B-BF16-down28-31-IMATRIX-Q5_K_M.gguf	5.50 GB	5.9	close, visibly less so than Q6_K (top-1 91.0-98.1%)	When memory is tight

"Top-1 agreement" is the share of tokens at which the quantized model's most likely next token matches the full-precision model's, measured on the full-precision model's own answers. The ranges run across five task areas; the lowest value is always Chinese chat.

Why no Q4_K_M? We built and measured one. It exceeded the quality-warning threshold we had fixed in advance: its worst task area had a KL upper bound of 0.24 nats/token against a limit of 0.10. In Chinese chat it disagreed with full precision on about 1 token in 6.

It also triggered a separate, pre-set disclosure rule: it may collapse into "!" more often than full precision (5.7% against 2.8%; borderline, p = 0.058).

With Q5_K_M only 0.75 GB larger, we do not publish Q4_K_M.

Why "BF16-down28-31"? In each quantized file, the FFN down-projection weights of layers 28-31 are kept in BF16 instead of being quantized. On Apple GPUs this protects against an activation overflow (see "Numerical safety"). It adds about 170-215 MB per file.

Filename clarification

The principal format is the final label in each filename: F16, Q8_0, Q6_K or Q5_K_M. The earlier BF16 label describes the protected FFN down-projections; these are mixed-precision files. Hugging Face uses the last recognized label when grouping GGUF downloads.

On 2026-09-28, the four model files were renamed to make those sidebar labels accurate. Only filenames changed: model bytes, sizes, checksums and evaluation results are identical. Existing downloads still work under their original names and need no replacement. The importance matrix is unchanged and is calibration data, not another model variant.

Original filename	Current filename
ZGCM-1-7B-F16-BF16-downall32.gguf	ZGCM-1-7B-BF16-downall32-F16.gguf
ZGCM-1-7B-Q8_0-BF16-down28-31.gguf	ZGCM-1-7B-BF16-down28-31-Q8_0.gguf
ZGCM-1-7B-Q6_K-IMATRIX-BF16-down28-31.gguf	ZGCM-1-7B-BF16-down28-31-IMATRIX-Q6_K.gguf
ZGCM-1-7B-Q5_K_M-IMATRIX-BF16-down28-31.gguf	ZGCM-1-7B-BF16-down28-31-IMATRIX-Q5_K_M.gguf

For historical commands using the original filenames, add --revision 0cf73a60999d5d606fe2c03da03a9bf12f29811a to hf download.

Evidence at a glance

Measure	Q8_0	Q6_K	Q5_K_M	Section
KL from full precision, mean over 5 task areas (nats/token)	0.0008	0.0071	0.023	Quality
Top-1 agreement, worst task area (Chinese chat)	98.2%	94.8%	91.0%	Quality
"!" collapse rate on its own answers (full precision: 2.8%)	1.4%	3.2%	2.1%	Known issues
Needle retrieval, 4k to 262k tokens, 15 cells	15/15	15/15	15/15	Long context
Lowest probability on a correct answer token (full precision: 0.91)	0.94	0.94	0.64	Long context
Top-1 agreement on short real text (129 / 200 / 1024 tokens)	99.2 / 98.0 / 99.2%	93.0 / 93.0 / 97.5%	93.0 / 88.5 / 96.3%	Long context
Activation headroom of unprotected layers, chat prompts	2.35x	2.35x	2.37x	Numerical safety
Same, arbitrary input up to 262k tokens	11.7x	11.4x	11.9x	Numerical safety
Generation speed at 4k context, M2 Max (tokens/s; full precision 26)	44.8	52.9	50.5	Speed
Prompt processing at 4k context, M2 Max (tokens/s; full precision 760)	735	639	588	Speed
Peak GPU memory at 4k / 256k context	8.5 / 14.9 GB	6.8 / 13.1 GB	6.0 / 12.3 GB	Memory

---

Quick start

1. Build the llama.cpp fork with ZGCM support

Source: ghthbl/llama-zgcm, zgcm-v0.1.0, commit e70b67f5ef242ceb29308642953177a2d6edd14c. The clean checkout built all four release tools and matched the evaluated build on five tokenizer samples and one deterministic chat response. See release validation.

git clone --branch feature/zgcm https://github.com/ghthbl/llama-zgcm.git
cd llama-zgcm
git checkout e70b67f5ef242ceb29308642953177a2d6edd14c
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON
cmake --build build --config Release -j 8 --target llama-server llama-cli llama-quantize llama-imatrix

The fork is llama.cpp at commit 41abbfd59 plus the ZGCM port: a model graph, conversion module, reference verifier and small edits to existing files. All results on this page come from that build.

Compatibility note. These files use the GGUF architecture name zgcm and the metadata keys defined in the fork. If official llama.cpp later adds ZGCM support with different names, these files may need re-conversion. Compatibility will need to be checked against that implementation before this page can recommend it.

2. Download a file

hf download 0xtb/ZGCM-1-7B-GGUF ZGCM-1-7B-BF16-down28-31-IMATRIX-Q6_K.gguf --local-dir .

3. Run

Use a local model path. Context 131072 is a starting point for agent clients that require at least 64k; increase to 262144 for the full supported context.

# OpenAI-compatible server with a web chat UI at http://127.0.0.1:8080
./build/bin/llama-server -m ZGCM-1-7B-BF16-down28-31-IMATRIX-Q6_K.gguf \
    -c 131072 -ngl 99 -fa on -np 1 --jinja --host 127.0.0.1 --port 8080

# interactive chat in the terminal
./build/bin/llama-cli -m ZGCM-1-7B-BF16-down28-31-IMATRIX-Q6_K.gguf -c 131072 -ngl 99 -fa on --jinja

• Always set -c. Without it, llama.cpp may use the model's full 262,144-token context, which costs about 5.5 GB of KV cache plus a compute buffer about 1 GB larger.
• -np 1 gives one conversation slot; raise it for several concurrent users, at the cost of more memory.
• The other files work the same way; replace the file name.

Prompt format.

• Always use the chat template (--jinja). It prefixes every conversation with [gMASK]<sop>.
• The model has a thinking mode (the default template emits <think>) and a direct-answer mode.
• Do not use these files for raw, untemplated text completion; see "Known issues".

Stop tokens.

• The end-of-generation token is <|endoftext|> (id 154820). llama.cpp stops on it automatically.
• In our evaluation, every answer that ended by itself ended with <|endoftext|>: over 1,000 answers, across English and Chinese chat, math, code and tool-use prompts. None ended with <|user|> or <|observation|>.
• Tool calling through llama.cpp's OpenAI-compatible API (parsing the model's <tool_call> format) has not been validated for reliable agent tool use. Local Hermes and direct-API text generation were tested separately; that does not establish tool-calling correctness.

Sampling.

• The model authors recommend temperature 1.0 and top-p 1.0; their benchmarks use those settings.
• At exactly those settings we observed the "!" behaviour described under "Known issues".
• Whether llama.cpp's default sampler or a lower temperature reduces it has not been established by a controlled sampler comparison.

---

Memory and speed on Apple Silicon

Memory

The KV cache is small for a 256k-context model:

• 20 KiB per token for the 5 global-attention layers (F16 cache);
• plus a constant 81 MiB for the 27 sliding-window layers (window 128).

At 256k tokens that is 5.45 GB, about one sixth of what the same model with all-global attention would need.

Peak GPU (Metal) memory measured during our runs, in GB (F16 KV cache, flash attention on):

File	4k	16k	64k	128k	256k
Q8_0	8.5	8.8	9.8	11.5	14.9
Q6_K	6.8	7.0	8.1	9.8	13.1
Q5_K_M	6.0	6.2	7.3	9.0	12.3

For the full-precision file we have only process memory (RSS), which is not directly comparable with the table: 15.0-15.2 GB at 4k, and 21.4 GB at 262k.

Choosing for your Mac. macOS lets the GPU use only part of the RAM (roughly two thirds to three quarters on most machines), and your other applications need memory too. So pick a file and context whose number above fits comfortably below that share. These are estimates, not tested on those machines:

• An 8 GB Mac is probably too small even for Q5_K_M.
• A 16 GB Mac should run Q5_K_M or Q6_K to about 64k.
• For the full 262k context: a 24 GB Mac for Q5_K_M or Q6_K, 32 GB for Q8_0. Full precision is marginal on 32 GB and comfortable on 48 GB or more.

Speed

Measured on an M2 Max with stock llama-bench, flash attention on, F16 KV cache, batch 512. There are two passes, each case in a fresh process; the cells show the range over the two passes.

File	Prompt processing, 4k	Generation at 4k context	Prompt processing, 64k	Generation at 64k context
Full precision	754-765	25.9-26.0	555-594	23.4
Q8_0	730-740	44.8	548-553	37.6-37.7
Q6_K	635-643	52.8-53.0	492-493	43.2-43.3
Q5_K_M	584-592	50.2-50.8	471	42.5-42.7

All values are tokens/s.

• Generation is mostly limited by memory bandwidth, so the smaller files are 1.6-2.0x faster than full precision.
• Q5_K_M generates slightly slower than Q6_K on this chip, although it is 13% smaller; its kernels are likely more expensive.
• Prompt processing is mostly limited by compute. There, the K-quants are 11-23% slower than full precision, and Q8_0 is 0-8% slower.
• One full-precision number varied 7% between passes, so treat prompt-processing ratios at 64k as +/- 7%.

---

Quality: how close is each quant to full precision?

Method in one sentence. We had the full-precision model answer 282 prompts across five task areas, then measured how differently each quantized model would have predicted every token of those answers.

We measure this with the Kullback-Leibler divergence (KL) per token:

• 0 means identical predictions;
• it grows as the quantized model's probabilities drift from the reference.

It measures divergence from the original model, not task accuracy, and it also counts harmless rephrasings.

Forward KL per token (nats; lower is better; 95% bootstrap intervals where given):

Level	English chat	Chinese chat	Math (thinking)	Code	Tool use	Mean of 5
Q8_0	0.0009 [0.0007, 0.0011]	0.0021 [0.0019, 0.0023]	0.0002	0.0003	0.0007	0.0008
Q6_K	0.0076 [0.0061, 0.0093]	0.018 [0.016, 0.020]	0.0018	0.0022	0.0057	0.0071
Q5_K_M	0.026 [0.021, 0.032]	0.060 [0.054, 0.067]	0.0059	0.0070	0.018	0.023

Top-1 agreement with full precision:

Level	English chat	Chinese chat	Math	Code	Tool use
Q8_0	99.0%	98.2%	99.6%	99.6%	99.3%
Q6_K	97.2%	94.8%	98.9%	98.6%	97.9%
Q5_K_M	94.9%	91.0%	98.1%	97.6%	96.3%

What the numbers show.

• Going from Q8_0 to Q6_K multiplies the divergence by about 9; going to Q5_K_M multiplies it by about 3 more.

• Math and code are the most robust.

• Chinese chat is the most sensitive at every level, about 2-2.5x English chat, including Q8_0, which uses no importance matrix.

• With llama.cpp's default sampler (temperature 0.8, top-k 40, top-p 0.95, min-p 0.05), the average per-token difference between the sampling distributions (total variation) is:

  Level	Range across areas	Chinese chat
  Q8_0	0.3-1.6%	1.6%
  Q6_K	1.0-4.7%	4.7%
  Q5_K_M	1.8-8.4%	8.4%

Perplexity can mislead here. On reference texts, a more aggressive quantization (Q4_K_M) sometimes showed lower perplexity than full precision (0.93x on grade-school math solutions), while its KL on the same texts was large. Flatter, smoothed probabilities can score better on text the model did not write. Raw-text perplexity is also dominated by the "!" effect below. Treat perplexity-only comparisons of these files with caution.

---

Long context: what was tested up to 262,144 tokens

Every published quant ran the same pre-planned long-context battery on the M2 Max. The full-precision reference ran the same retrieval grid and short-text inputs at an earlier stage; the reference values below come from those runs.

Needle retrieval

• The task. A 16-character hexadecimal code (the same code in every cell) is hidden in a haystack of filler text at 10%, 50% or 90% depth, in contexts of 4k, 16k, 64k, 128k and 256k tokens (15 cells). The model must return exactly that code, with greedy decoding.
• Result. Every file, full precision included, returned the exact code in all 15 cells.

Because every file scores 15/15, exact match cannot rank the files. So we also recorded how confident each model was on every decision in all 15 cells: 12 per cell (11 answer tokens plus the stop), 180 per file.

File	Lowest probability on a correct answer token	Decisions whose margin over the runner-up is below 0.8
Full precision	0.908	0
Q8_0	0.942	0
Q6_K	0.940	0
Q5_K_M	0.639	3

The lowest confidence of Q8_0 and Q6_K is higher than full precision's. Q5_K_M is always right, but in one 128k cell its probability on one answer token fell to 0.64, where full precision had 0.94; and 3 of its 180 decisions had a margin below 0.8. That is a small, real loss of long-context robustness, measured on one synthetic task.

Divergence along the way. We also compared each file's predictions with full precision at the end of every 512-token chunk of the haystack. The mean KL per cell was:

• about 0.00001-0.004 nats for Q8_0;
• up to 0.002 for Q6_K;
• up to 0.03 for Q5_K_M.

These ranges exclude one cell dominated by a single position (below). The divergence does not grow with context length.

The largest single difference is instructive. The filler text reads "The cook cleaned the glass in the ___", after earlier sentences ending "... the glass in the window":

File	P(" room")	P(" window")
Full precision	0.84	0.15
Q8_0	0.95	0.04
Q6_K	0.49	0.51
Q5_K_M	0.003	0.97

Q6_K is split, and Q5_K_M copies the earlier phrase. It is a genuinely ambiguous token, but it shows the fidelity ordering.

Short real text

This test covers 129, 200 and 1024 tokens of real, previously tokenized text, with every position compared with full precision:

File	Top-1 agreement (129 / 200 / 1024)	Mean KL (nats)
Q8_0	99.2 / 98.0 / 99.2%	0.003 / 0.014 / 0.003
Q6_K	93.0 / 93.0 / 97.5%	0.054 / 0.063 / 0.032
Q5_K_M	93.0 / 88.5 / 96.3%	0.077 / 0.211 / 0.056

This text is harder than the evaluation above, so compare files with each other here, not with the earlier tables.

Long documents

On six documents of about 30k tokens (about 2,000 scored tokens; a small sample, suggestive only), answers diverge 2.4-2.7x more than in short chat. Top-1 agreement:

File	Top-1 agreement
Q8_0	98.5%
Q6_K	94.6%
Q5_K_M	90.5%

---

Known issues (please read)

1. The model sometimes collapses into "!" (a property of the original model)

What happens. ZGCM-1-7B sometimes switches abruptly to emitting the "!" token (token id 0), often in long runs, sometimes interleaved with fragments of other text.

What we measured, at the authors' recommended sampling (temperature 1.0, top-p 1.0):

• Collapse rate: about 2.5-3% of answers, in two independent full-precision samples of 282 answers each (7 and 8 answers).
• Where: mostly Chinese chat and tool-use prompts. More Chinese answers contain stray "!" tokens without a full collapse.
• Onsets: at 10 of 13 onsets we checked, "!" was already the model's top prediction (62-100% probability). So it is usually not just an unlucky draw from the tail.

It comes from the original model. We ran the original Hugging Face implementation in float32 on the same token sequences. At all 13 onsets, its "!" probabilities matched llama.cpp's to within 0.0084.

The published quants show no sign of making it worse. These rates come from a fresh sample in which full precision collapsed in 8 of 282 answers; the main quality tables used an earlier sample with 7 of 282.

File	Collapse rate on its own answers (same 282 prompts)
Full precision	2.8%
Q8_0	1.4%
Q6_K	3.2%
Q5_K_M	2.1%

These are statistically indistinguishable from full precision. This is not proof of no increase, but at one-sided 95% confidence the data rule out an increase of more than about 2.4 percentage points.

Sampling experiments (not validated remedies):

• use the chat template;
• consider a lower temperature or llama.cpp's default sampler;
• evaluate repetition penalties cautiously; suppressing token 0 can also remove legitimate punctuation.

These are experiments to try, not validated remedies. Changing penalties or suppressing token 0 also changes the sampling distribution.

Independent report from the NVFP4 conversion

Kaleto/ZGCM-1-7B-NVFP4 independently reports repeated ! outputs. KaletoAI also published a reproduction on the original BF16 weights with the official Transformers implementation. That report describes content-sensitive failures with few-shot examples represented as multiple chat turns, including finite logits and high probability for token 0. Packing the examples into one user message avoided the two reported chemistry failures; enabling thinking did not.

This corroborates the existence of the behavior outside our GGUF conversion and Metal runtime. Their prompts, hardware, generation settings and failure definition differ from ours, so their rates must not be pooled with our table or used to rank NVFP4 against these GGUF files. We have not rerun their reproducer. Their separate warning about uncalibrated W4A4 activation quantization is another failure mechanism and is not evidence about these GGUF artifacts.

2. Do not use raw text completion

Outside the chat template, "!" is often the top prediction:

Text	"!" is the top prediction at
English prose with the [gMASK]<sop> prefix but no chat turns	52% of positions
The same text with no prefix	16%
Classical Chinese with the prefix	15%
Classical Chinese with no prefix	0%

Raw-text perplexity on this model therefore largely measures this effect, not language-modelling quality.

3. Backend coverage

• Validated: Metal on an Apple M2 Max; the CPU backend at short lengths, for the full-precision file.
• Not tested: CUDA, Vulkan, ROCm, SYCL, and other Apple chips.
• The BF16 layers exist because of a Metal-specific mechanism (below). Other backends may behave differently, in either direction.

4. Exploratory SVG generation

A direct llama-server test used only the prompt "Generate an SVG of a pelican riding a bicycle", with no Hermes or system message. All six tested artifacts (including the unpublished Q4 variants) returned renderable SVGs in both thinking-off and 1024-token thinking-budget modes. Visual inspection did not find a convincing depiction of the requested scene, including from the F16/BF16 reference. No response exhausted the 32768-token output allowance. This is one prompt and one seed per setting, not a task benchmark or a quantization ranking. A canonical HF generation comparison was not performed, so the cause remains unresolved. See outputs and settings.

5. If you quantize this model yourself

Plain llama-quantize will quantize the down-projections of layers 28-31 like any other weight. Keep them in BF16:

./build/bin/llama-quantize --tensor-type '^blk\.(28|29|30|31)\.ffn_down\.weight$=bf16' \
    --imatrix imatrix/ZGCM-1-7B-imatrix-172032.gguf \
    ZGCM-1-7B-BF16-downall32-F16.gguf ZGCM-1-7B-BF16-down28-31-IMATRIX-Q6_K.gguf Q6_K 8

Without this, the last layer's down-projection input (above 5e8) overflows on Metal. In our tests, an unprotected layer 31 produced non-finite values. For the reason, see the next section.

---

Numerical safety: massive activations and the BF16 layers

This model has "massive activations". The input to the FFN down-projection reaches:

• at least 5.5e8 in the last layer (layer 31);
• about 63,700 in layer 28, on the <sop> token at the start of every chat;
• about 27,900 in layer 10, and about 19,000 in layer 23, on that same token.

Half precision (F16) cannot represent values above 65,504. In the llama.cpp Metal matrix-multiplication kernels we inspected, activations are converted to half precision when the weights are F16 or quantized, but to bfloat16 when the weights are BF16. BF16 keeps the float32 exponent range.

What we did:

• Full-precision file: all 32 down-projection weights are stored as BF16. This costs nothing: the original weights are BF16, so the values are bit-exact.
• Quantized files: layers 28-31 are kept in BF16. They were chosen by an explicit rule from measured activations: protect any layer whose measured input, multiplied by a safety factor, would reach the half-precision limit. The factor is 1.25 for the fixed chat prefix and 2 for content.

That this choice holds up at every published level, across the full context, was then measured.

Margins measured on the published files (down-projection input, largest absolute value):

Input	Worst unprotected layer (0-27)	Headroom to 65,504
Chat-templated prompts (21 per file, 4k to 256k)	layer 10, 27,670-27,883, on <sop> (full precision: 27,852)	2.35-2.37x
Arbitrary token IDs, every position, 4k / 64k / 256k	layer 26 at 256k, 5,523-5,724	11.4-11.9x

• The layer-10 value is set by the fixed chat prefix, not by your content or context length. It was the same in every templated run.
• The largest content-driven value in an unprotected layer was 15,688, which is 4.2x headroom. It was layer 23, on a tool-use prompt, in Q5_K_M; Q8_0 had 12,513 and Q6_K 13,688 there. Over the six canary prompts, full precision's largest content-driven value was 13,210.
• No non-finite value appeared in any activation or logit the instruments inspected. The arbitrary-ID runs checked every logit at every position.

Protecting layers 28-31 changes nothing below them. For Q8_0 and Q6_K we also have files from an earlier release candidate with only layers 30-31 protected. At 4k, 64k and 256k, the recorded per-layer activation maxima of layers 0-28 (value, position and channel) are bit-identical to those of the earlier files. On short real text, top-1 agreement is identical and mean KL is within 5%.

This is precautionary engineering, not a proof. It holds for this build and these kernels. It does not prove that no input anywhere can overflow, and it does not show that the extra protection was necessary.

---

What was done

The llama.cpp port

ZGCM is not a standard Llama variant. The port implements:

• Hybrid attention. 27 sliding-window layers (window 128) and 5 global layers (5, 11, 17, 23, 29), using llama.cpp's interleaved sliding-window cache.
• The attention gate. An element-wise sigmoid gate on the sliding-window layers only; the global layers have no gate tensor.
• Partial rotary embedding. RoPE on 42 of 128 head dimensions (NeoX style), theta 1e7.
• Per-head QK RMSNorm before RoPE, and grouped-query attention with 32 query heads and 8 KV heads.
• Correct norm roles. The checkpoint's post_attention_layernorm and post_feedforward_layernorm are actually pre-norms. The generic name mapping would silently swap them.
• The GLM-family tokenizer and template. A [gMASK]<sop> prefix, no BOS token, vocabulary padded to 155,136.

The conversion used the authors' Hugging Face snapshot of 2026-09-14. A later upstream update (2026-09-26) changed the README and community QR images. Our checks: the configuration and tokenizer configuration files are identical; the tokenizer and weight files have identical sizes (their hashes were not compared).

How it was validated against the original Hugging Face code in float32

• Top-1 agreement. Zero top-1 disagreements across 1,353 positions of real text, on Metal and CPU. For comparison, the authors' own BF16 checkpoint, run in Hugging Face, disagreed with its float32 self at 9 of those positions.
• Per-layer hidden states stay within a narrow band relative to the BF16-versus-F32 noise floor.
• Cache behaviour. Sliding-window masking and eviction were verified, including bitwise-identical results when the prompt is processed in different chunk sizes.
• Full length. A full 262,144-token prefill completed with all logits finite, and needle retrieval was exact in 15 of 15 cells.

How the quantized files were made

• Tool: llama-quantize from the full-precision file, with the BF16 override shown under "Known issues". Q8_0 uses no importance matrix.
• Importance matrix (Q6_K and Q5_K_M): ZGCM-1-7B-imatrix-172032.gguf, published here.
  • Computed with llama-imatrix on 172,032 tokens (84 chunks of 2,048), with the output layer included.
  • Calibration text: synthetic math worked examples, synthetic multi-turn search-and-tool trajectories, and excerpts of source code.
  • The evaluation texts were held out.
  • The calibration contains no Chinese and no ordinary chat. Even so, Chinese chat is equally sensitive in Q8_0, which uses no importance matrix, so the imatrix does not explain that pattern.
• At Q4, the importance matrix helped. At that level, against a same-size control without it, it lowered KL by 9-13% in every task area, including Chinese chat. This was statistically clear.
• It may cost some robustness at low bit-widths. In Chinese chat, the importance-matrix Q4 collapsed into "!" in 13 of 48 answers, against 5 of 48 without it and 4 of 48 at full precision. This was not statistically established, and it is one reason Q4 is not published.

How quality was evaluated

Design. The design, the quality thresholds and the decision rules were fixed before any reported result existed.

Two changes were made after a first run stopped on a pre-set rule, and before any of the reported results were computed:

• answers where full precision itself collapsed into "!" are excluded from the main table;
• a corrected version of the "!" criterion is used. The original fired for every level, including Q8_0, because it counted events that full precision itself produces.

Prompts. 282 prompts in five areas, from public sources:

Area	Source
English chat	MT-Bench and Vicuna-bench questions
Chinese chat	BELLE eval set
Math, thinking mode	MATH-500 problems
Code	HumanEval prompts
Tool use	BFCL v4 live and web-search prompts

Reference texts (not written by the model):

Text	Source
English prose	self-instruct outputs
Math solutions	GSM8K
Code	MBPP
Tool calls	BFCL simple_python
Long documents (about 30k tokens)	Project Gutenberg and Chinese classics

Procedure.

• Full precision answered each prompt with pure sampling (temperature 1, no truncation).
• Every quantized model was scored on exactly the same token sequences.
• KL was computed in float64 over the full 155,136-token vocabulary.
• The 7 answers in which full precision itself collapsed (in that sample) are reported separately. Including them changes no conclusion.

Checks.

• On a subset, KL from batched scoring agreed with KL from token-by-token generation arithmetic: the ratio was 0.998-1.029, against a preset band of 0.9-1.1.
• Logits were bit-identical across processes and sessions.
• The metric agreed with an independent implementation.
• Wrong inputs were rejected.

Decision rules, fixed in advance:

• A level carries a quality warning if its worst task area's KL upper bound (one-sided 99%) exceeds 0.10 nats/token. Q5_K_M: 0.068, no warning. Q4_K_M: 0.24, warning.
• A level carries a disclosure if a collapse increase of 5 percentage points cannot be ruled out. None of the published levels does.

The long-context battery (the sections above) came afterwards, as its own pre-planned run on the final files:

• 121 steps, all completed, in 7.5 hours;
• the activation, logit and short-text instruments first had to reproduce earlier recorded runs bit for bit;
• warning criteria were declared in advance;
• none fired.

Not measured:

• benchmark accuracy (for example MATH-500 scores);
• human preference;
• other hardware.

Overlap of the evaluation prompts with the model's public training data (zgcagi/ZGCM-1-Data) was not checked.

---

Importance matrix and reproducibility

The calibration matrix is included under imatrix/, with its provenance, limitations and quantization commands. It is calibration data, not another model. Q6_K and Q5_K_M use this matrix; Q8_0 does not. The calibration corpus is not bundled, so the matrix can be reused to reproduce quantization without claiming its original collection is fully reproducible from this repository alone.

Files and checksums (SHA-256)

File	Bytes	SHA-256
ZGCM-1-7B-BF16-downall32-F16.gguf	14799663520	9d541c62fead71fe69aac06cd284957b8efdd9026daae47be5197018d715fb0a
ZGCM-1-7B-BF16-down28-31-Q8_0.gguf	8036348320	754ebcd4428c6abb368703e109d82eee25950ee277975091c87e78533594f8f8
ZGCM-1-7B-BF16-down28-31-IMATRIX-Q6_K.gguf	6289158912	d52a23eb9ad1a94577ea7d0adbbcaef9973a76401bc8d3bff9293a74530bebd0
ZGCM-1-7B-BF16-down28-31-IMATRIX-Q5_K_M.gguf	5496189696	d0b056949b131843024bfa3c196e31a05bd096e4483a7e8d6c4ade3105a63299
imatrix/ZGCM-1-7B-imatrix-172032.gguf	5053216	ba565377ec41f32b734b69cd387ea87128c8e654fde8171d8f73a79d35890bab

---

License and credits

• Model: ZGCM-1-7B by the Zhongguancun Academy and the Zhongguancun Institute of Artificial Intelligence, released under the MIT License. These GGUF files are derivative conversions under the same license. See the original model card and the technical report.
• Conversion, llama.cpp port and evaluation: 0xtb, GitHub ghthbl. Model training and authorship belong to ZGCAGI; no training or fine-tuning was performed for this conversion.
• AI assistance: The port, evaluation tooling and documentation were developed with substantial help from Anthropic Claude and OpenAI assistants, under the publisher's direction. Measurements were produced locally and checkpointed with commands and artifact hashes. The publisher is responsible for this community release.

If you use the model, please cite the authors' report:

@misc{zgcm1,
  title={ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search},
  author={Jiyan He and Guang Liang and Hao Liu and Haoxiang Guan and Jinbo Sun and Junyi Guo and Wenjun Feng and Yantai Xie and Yifei Shen and Bin Shao and Chuyang Wei and Kai Chen and Kexin Zhou and Minghang Zhu and Shuxin Zheng and Tie-Yan Liu and Taine Zhao and Wenhui Zhu and Xueyin Xu and Xiaoqing Zhang and Yatao Li and Yuxuan Ren},
  year={2026},
  eprint={2609.13356},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2609.13356}
}