Sage-S2.1-Flash Jammed XSFP4
Sage S2.1 Flash for two GB10s, with six methods in one checkpoint: 4-bit routed experts in blocks of 32 with a +5 mark and learned smoothing, the official FP8 attention, and BF16 elsewhere, 150 of its matrices stored losslessly smaller. A DFlash2 drafter comes with it.
To serve it, TensorFold needs two pull requests merged in: #3680 (BF16 dense MLPs) and #3807 (DFloat11). As of 2026-11-20 both are open, so no release reads this checkpoint, 1.3.0 the latest included. Build TensorFold with both merged in (The engine has the commands), or serve Mixed NVFP4, which release 1.2.9 serves.
| At a glance | |
|---|---|
| 🎯 Closeness to BF16 | KL 0.0454 in PyTorch, top-1 agreement 93.36%; Mixed NVFP4's are 0.0426 and 93.56% (lower KL is closer) |
| 🔢 What is 4-bit | The routed experts of layers 3 to 44: 4-bit codes with E2M1's −0 read as +5 (NVFP4_NZ5), an E4M3 scale every 32 weights stored as CSF0, smoothing factors learned on calibration text folded in. The MTP layer's routed experts: Mixed NVFP4's NVFP4, byte for byte |
| 🧩 What is not | The official FP8 release's bytes for MLA attention and the shared experts; BF16 for the rest, the dense MLPs included. 150 of the BF16 matrices are stored in DFloat11's lossless form and decoded at load |
| 💾 Size | 176.7 GB in 149 files, the bundled drafter's 2.34 GB included (Mixed NVFP4: 183.8 GB). At TP=2 the model takes 86.14 GiB on the first machine, against Mixed NVFP4's 92.51 GiB on the same engine build |
| ⚡ Speed | On one engine build, decode with drafts off is 4.3% slower than Mixed NVFP4's; a cold prompt of 38,400 tokens is not told apart from it, and one of 9,600 tokens took about 1% longer in three of four pairs. With the bundled DFlash2 drafter, one greedy code request runs at 42.5 to 42.7 tok/s, against 36.4 to 38.4 with MTP, and cold prompts take about 5% longer (In serving). TP=2 and TP=3 figures in the shape of the GLM-5.3-Flash cards' Speed tables, with this checkpoint's tool-eval-bench 3.0.0 scores at 94 to 96, are under Speed |
| 🔧 Engine | No release and no TensorFold main reads it yet, as of 2026-11-20: it needs two open pull requests, #3680 (BF16 dense MLPs) and #3807 (DFloat11). Served here on main at e3fb56cb09 with both merged in (The engine) |
TP=2 and TP=3 mean the model split across two and three systems.
Sage-S2.1-Flash Jammed XSFP4 is a derived checkpoint of internlm/Sage-S2.1-Flash for GB10 (DGX Spark class) systems, the second after Sage-S2.1-Flash Mixed NVFP4. Mixed NVFP4 is the plain starting point, made from the weights alone. This one puts in the methods of this account's GLM-5.3-Flash checkpoints (XSFP4, Jammed XSFP4 and Jammed XSFP4 NZ5), all at once, and adds DFloat11 for 150 of the BF16 matrices. Its name leaves out NZ5, which is in it: the +5 mark is the default from this checkpoint on (in GLM-5.3-Flash's line, Jammed XSFP4 and Jammed XSFP4 NZ5 are two checkpoints, without and with it). It is not a Shanghai AI Laboratory, InternLM, Z.AI, NVIDIA or incoai release.
Is it for you?
- ⏳ Not yet from a release, if you want to serve it. As of 2026-11-20 it needs #3680 and #3807, two pull requests to TensorFold that are open; no release and no
mainhas them (The engine). Mixed NVFP4 serves from TensorFold's release 1.2.9 today. - ✅ Yes, if you build TensorFold with those two changes and have two GB10s: at TP=2 with a 40,960-token window and two streams, it left at least 12.25 GiB free on each machine over its eight launches, against 8.54 GiB for Mixed NVFP4 on the same build. In one launch with a 1,048,576-token window, four streams and the vision tower loaded, it started and left at least 8.67 GiB free on each machine; Mixed NVFP4 did not start with those settings (Speed).
- ✅ Yes, if you study 4-bit formats. It is a whole model with smoothing, block 32, CSF0 and +5 in its routed experts, measured against Mixed NVFP4 on the same positions in PyTorch and on the same engine build in serving.
- ❌ No, if you have one GB10. At TP=2 the first machine alone holds 86.14 GiB of model.
- ❌ No, if you serve with anything else. Only TensorFold with the NVFP4_NZ5 kernels and the two changes above reads it.
- ❌ Not for the scientific-input projector. The source repository's
projectors/folder is not in this checkpoint. - 🖼️ Image input was not checked on this checkpoint. The vision tower is here, in BF16 as in the source. The launches of In serving had it off; those of Speed loaded it, but sent no image.
Six methods
| Method | What it does here | Where | How it was checked |
|---|---|---|---|
| Mixed precision | Each part at its own precision: 4-bit routed experts, the official FP8 release's bytes where that release uses FP8 blocks (MLA attention, shared experts), BF16 for the rest | The whole checkpoint | The config's recipe table (220 entries) against every tensor's dtype in the file headers |
| XSFP4 | Smoothing factors, learned on calibration text for each expert and folded into its up and down projections. Nothing is stored for them | Routed experts, layers 3 to 44 | The factors recovered from the stored up and down match those recovered from the block-16 checkpoint, in 126 sampled experts |
| Block32 | One scale every 32 weights instead of 16: half the scales | The same | The recipe's group_size 32 and the shape of the stored scales |
| CSF0 | The scales at a fixed length with no exceptions: a base byte a row and a 4-bit step a scale, read in place by the GPU kernels | The same | All 36,288 matrices read back after packing |
| NZ5 | E2M1's spare code, −0, read as +5: a mark in the widest gap, with no bit added. 2.03% of the weights take it | The same | Each shard's code_map, and the count of every code against the fit's record |
| DFloat11 | BF16 stored losslessly: the exponents Huffman-coded, decoded at load to the same bytes | 150 BF16 matrices, 12.07 GB stored as 8.16 GB | Every coded matrix decoded and compared with the BF16 bytes |
The last five come from earlier work: XSFP4, block 32 and CSF0, and +5 are described step by step on the GLM-5.3-Flash cards linked above, and DFloat11 is Zhang et al.'s format. The first, mixed precision, is Mixed NVFP4's layout with the dense MLPs moved from MXFP8 to BF16.
What it costs
- Closeness. KL is 0.0028 above Mixed NVFP4's, with an interval above zero (Closeness to BF16). The step to block 32, with its scales fitted again inside CSF0's window, costs it, and +5 takes back about half; the smoothing factors do not lower it (How it is made).
- Decode. With drafts off it is 4.3% slower than Mixed NVFP4's on the same build. Three things differ: the routed experts' format, the dense MLPs (BF16 here, MXFP8 there), and DFloat11 (decoded at load, so nothing of it is left at decode). Which of them costs what was not measured.
- Load. 136.5 to 137.5 s, against Mixed NVFP4's 113.6 to 116.3 s. The DFloat11 matrices are decoded on the host before they go to the GPU, which took 38 to 40 s more when #3807 measured the same 150 matrices on Mixed NVFP4; here the smaller expert files read faster (162.3 GiB in about 44 s, against 171.1 GiB in about 58 s).
- Engine. It needs two changes that are not in TensorFold yet (The engine).
What's quantized
The rows sum to the 176.63 GB of tensor data in the folder's 134 safetensors files, read from their headers.
| Component | Stored precision | How it was made |
|---|---|---|
| Decoder routed experts, layers 3 to 44 157.06 GB |
4-bit codes, E2M1 with 0b1000 read as +5 (NVFP4_NZ5); E4M3 block scales of 32 in CSF0, one base byte a row and a 4-bit step a scale; FP32 global and input scales; 36,288 matrices |
Quantized here from BF16. Calibration: 512 rows of 2,048 tokens (How it is made). Smoothing: XSFP4's factors, learned here at block 16, each expert on the tokens routed to it, and folded into up and down. Scales: the smallest squared error inside each row's 16-step window over the sixteen marks with +5, with no activation statistics. Compression: CSF0. |
| MTP routed experts, layer 45 4.08 GB |
NVFP4: E2M1 codes, E4M3 block scales of 16 stored directly, FP32 global scale (W4A16_NVFP4); 864 matrices |
Mixed NVFP4's, byte for byte: made from BF16 with no calibration data. |
MLA attention (q_a, q_b, kv_a, o) in 11 layers and the MTP layer, and the shared experts of layers 3 to 452.29 GB |
FP8 E4M3 in 128×128 blocks, an FP32 scale a block; 177 matrices | The official FP8 release's bytes, revision 716e6436905650b7be93e1b7dedfb0bb95afd3a3, as in Mixed NVFP4: its scale tensor weight_scale_inv is stored as weight_scale; the value multiplies the code in both. |
| Dense MLP, layers 0 to 2 0.91 GB |
BF16; 9 matrices | The BF16 checkpoint's bytes. Mixed NVFP4 stores MXFP8 here; BF16 dense layers need #3680. |
KDA attention (q, k, v, o) in 34 layers, MLA's kv_b in 12 layers, lm_head and the embeddings8.16 GB, 12.07 GB as BF16 |
BF16 in DFloat11's lossless form: per matrix, the exponents Huffman-coded with lookup tables, sign and mantissa as they are; 150 matrices, 0.676 of their BF16 size | The BF16 checkpoint's bytes, coded here. They need #3807, which decodes them on the host at load. |
| Vision tower 1.13 GB |
BF16 | The BF16 checkpoint's bytes. |
| Router, indexers, norms and the rest 0.67 GB |
BF16; F32 where the source has F32 | The BF16 checkpoint's bytes, revision 76f4513db7464485605a002a06a6190de0460574. |
DFlash2 drafter, in dflash2/2.34 GB |
BF16; TensorFold packs its linears to 4 bits at load by default | incoai/GLM-5.3-Flash-DFlash2, revision 6cc6c481eee930a5139619ed919f78726371adbc: its model.safetensors and config.json, unchanged. It was trained for GLM-5.3-Flash, not for Sage. |
The engine
| Engine | Serves this checkpoint? |
|---|---|
| TensorFold releases, 1.3.0 (the latest as of 2026-11-20) included | No. 1.2.10 and 1.3.0 have the NVFP4_NZ5 kernels but neither change. Without them, by the two pull requests' descriptions, a checkpoint with DFloat11-coded weights is refused by the missing tensor's name, and BF16 dense MLPs are taken only in an EXL3 checkpoint. Not run with this folder |
TensorFold main at e3fb56cb09, 53 commits after 1.2.10 |
No, for the same two reasons. Not run with this folder |
main at e3fb56cb09 with #3680 and #3807 merged in, built here |
Yes. Everything under In serving was measured on this build. #3680's code commit is the same patch as at its current head (the owner has since rebased it and dropped its changelog line); #3807 is at its current head, e5fd126b47 |
To serve it, merge both pull requests into TensorFold:
git clone https://github.com/ashhart/TensorFold && cd TensorFold
git fetch origin pull/3680/head:pr-3680 pull/3807/head:pr-3807
git merge --no-edit pr-3680
git merge --no-edit pr-3807
On 2026-11-20 both merge into main at dffe424859 without a conflict (checked with git merge-tree); that tree was not built or run here, and the build measured is the one in the table above. Build the server from the merged tree as TensorFold's Build a server for one GPU describes, put this repository's files in one folder on each machine, and follow TensorFold's GLM-5.3-Flash CUDA recipe, which places the ranks and names the link between the hosts for NCCL. The launches here named that folder to the engine's binary, the same line on both machines but for the rank, with 192.0.2.1 standing for the first machine's address on their direct link:
# on the second machine (rank 1); on the first, TF_GLM_RANK=0
TF_GLM_RANK=1 TF_GLM_WORLD=2 NCCL_COMM_ID=192.0.2.1:29583 \
TENSORFOLD_MEMORY_RESERVE_GIB=9 TENSORFOLD_PREFILL_SHARE=0.7 \
tensorfold-native serve /path/to/Sage-S2.1-Flash-Jammed-XSFP4 \
--context 40960 --parallel 2 --max-tokens 32768 --reasoning-effort high
# with the bundled drafter, add: --drafter /path/to/Sage-S2.1-Flash-Jammed-XSFP4/dflash2
Without --drafter the engine drafts with the MTP layer; the dflash2/ folder is not read. With it, DFlash2 drafts and uses the MTP layer's head, so the MTP layer stays in the checkpoint. Beside that line the launches set a name, host and port for the server, --no-update-check, the engine's heat limits (TF_HEAT_HIGH=92, TF_HEAT_LOW=88) and NCCL's settings for the link. For the pairs in In serving they also fixed the experts' kernel form, TF_GLM_EXPERT_FORM=factored, which TensorFold otherwise picks by timing both at load; a launch left to pick chose classic. The settings were not weighed against others.
Measured
Closeness to BF16
The 128 documents of Mixed NVFP4's card (32 each of code, English, Japanese and math; 144,285 scored positions) went through the model in PyTorch with each checkpoint's weights dequantized and 16-bit activations (W4A16), against the BF16 weights over the whole vocabulary. These are PyTorch numbers, not the serving engine's. The scoring read the plain BF16 bytes in place of the DFloat11 matrices, which decode to them.
| Checkpoint | KL (BF16 ‖ checkpoint) | Top-1 agreement | NLL above BF16 |
|---|---|---|---|
| This checkpoint | 0.0454 | 93.36% | +1.16% |
| Mixed NVFP4 | 0.0426 | 93.56% | +1.05% |
Both were scored on the same positions, so their differences have 95% intervals, from a paired bootstrap over the 128 documents (32 for one domain):
| This checkpoint against Mixed NVFP4 | Difference [95% interval] |
|---|---|
| KL | +0.0028 [+0.0008, +0.0046] |
| Top-1 agreement | −0.20 points [−0.36, −0.04] |
| NLL above BF16 | +0.10 points [−0.16, +0.38] |
| Routed experts shared with BF16's choice (Jaccard) | −0.0030 [−0.0043, −0.0015] |
| Domain | KL, Mixed NVFP4 | KL, this checkpoint | Difference [95% interval] | NLL above BF16, Mixed NVFP4 | NLL above BF16, this checkpoint |
|---|---|---|---|---|---|
| Code | 0.0315 | 0.0344 | [+0.0008, +0.0055] | +2.29% | +2.34% |
| English | 0.0534 | 0.0576 | [+0.0013, +0.0072] | +2.07% | +2.39% |
| Japanese | 0.0411 | 0.0426 | [−0.0025, +0.0051] | +0.34% | +0.34% |
| Math | 0.0306 | 0.0341 | [+0.0006, +0.0068] | +0.20% | +0.22% |
The NLL rise is not shown to differ: its interval holds zero overall. The experts the router picks for a token overlap BF16's picks with a Jaccard index of 0.841 on average over the 42 routed layers: 0.979 at layer 3, 0.766 at layer 37, the lowest.
Step by step, on the way here. Each row says what changes from the row above; the first four keep Mixed NVFP4's MXFP8 dense MLPs, and the last is this checkpoint.
| Step | KL |
|---|---|
| Mixed NVFP4: block 16, no smoothing | 0.0426 |
| The learned smoothing, with the scales fitted as XSFP4 fits them, on calibration activations | 0.0425 |
| Block 32: the scales fitted again by plain squared error inside each row's 16-step CSF0 window | 0.0492 |
| +5 | 0.0459 |
| Dense MLPs in BF16: this checkpoint | 0.0454 |
The steps' own intervals: smoothing at block 16, −0.0001 [−0.0018, +0.0015]; the step to block 32, +0.0067 [+0.0050, +0.0085]; dense MLPs to BF16, −0.0006 [−0.0017, +0.0005]. The step to +5 was not given one; on GLM-5.3-Flash the same step was −0.0030 [−0.0042, −0.0019].
In serving
Two GB10s at TP=2 over a direct link, on 2026-11-19, on the build above: a 40,960-token window, two streams (--parallel 2), vision off, a 9 GiB reserve. Thirteen launches in one run: this checkpoint eight times and Mixed NVFP4 five times, the first of them a warm-up. Four pairs, one launch of each checkpoint, ran next to each other, the order turned in each pair, with the experts' form fixed to factored. Decode is with drafts off, 256 tokens five times on each of two prompts, the median; prefill is a first request that reuses nothing from the prompt cache. The ranges are over the four pairs.
| This checkpoint | Mixed NVFP4 | |
|---|---|---|
| Model on the first machine | 86.14 GiB | 92.51 GiB |
| Loaded in | 136.5 to 137.5 s | 113.6 to 116.3 s |
| Pages of 512 rows in the engine's pool | 2,149 | 1,874 to 1,902 |
Free memory at its lowest (MemAvailable) |
13.54 to 13.63 GiB | 8.54 to 9.01 GiB |
Decode, fibonacci-raw |
19.01 to 19.06 tok/s | 19.39 to 19.89 tok/s |
Decode, gpu-chat-no-think |
18.99 to 19.07 tok/s | 19.64 to 19.95 tok/s |
| Cold prompt of 9,600 tokens | 6.77 to 6.79 s | 6.70 to 6.82 s |
| Cold prompt of 38,400 tokens | 26.30 to 26.44 s | 26.24 to 26.91 s |
| Drafts (MTP and copies), tokens a round: count, prose, code | 5.51 to 5.89, 2.05 to 2.12, 3.35 to 3.61 | 4.97 to 5.39, 1.98 to 2.03, 3.51 to 3.68 |
Pair by pair, this checkpoint's decode is 4.35, 4.37, 1.79 and 4.15% slower on fibonacci-raw (median 4.25%) and 4.29, 4.27, 3.22 and 4.59% slower on gpu-chat-no-think (median 4.28%): slower in every pair. At 38,400 tokens its cold prompts are not told apart from Mixed NVFP4's: two pairs each way, with a median of −0.10%. At 9,600 tokens it took 1.04 to 1.37% longer in three pairs and 0.48% less in one (median +1.20%): not settled by the rule set before the run, which asked for the same sign in all four pairs. #3807's own receipt found a DFloat11-coded folder's first 9,600-token prompt about 1% slower than the plain one's.
The tokens a round are counted on each checkpoint's own replies, which differ, so they do not rank the checkpoints, and they do not tell what the smoothing does: no launch served these experts without it.
Three fixed prompts, a count, an essay in Japanese and a Python module, 512 tokens each, were served with drafts on and with "draft": false in every launch. Every launch of each checkpoint gave its own three token_sha256, the same with drafts on and off, MTP or DFlash2, the experts' form fixed or not: this checkpoint 17bfc88c60bd…, a86ebb2ebec6… and 28805e7d7fca… in all eight launches, and Mixed NVFP4 the three of its card in all five. No line of any launch's engine logs says disabled, fallback or compile failed, the watchdog stopped nothing, and the hosts' clock-cap watcher held the GPU clock at its 2,200 MHz cap throughout.
The tokens are this checkpoint's own: it counts, and writes the essay and the module, as Mixed NVFP4 does, in its own words. No plain copy of this checkpoint was served, so DFloat11 is not checked against one here. What was checked: every coded matrix decodes to its BF16 bytes (Six methods); the same reader gave the plain checkpoint's tokens when it read a DFloat11-coded copy of Mixed NVFP4 (#3807); and this checkpoint gives one set of tokens in all eight launches.
The bundled DFlash2 drafter, on this checkpoint: four launches drafting with MTP and two with DFlash2, each beside the engine's copies of earlier text, the same build and settings. One request at a time, aggregate_tps, 256 tokens, the median of three:
| MTP and copies | DFlash2 and copies | |
|---|---|---|
| Tokens a round: count, prose, code | 5.51 to 5.89, 2.05 to 2.12, 3.35 to 3.61 | 5.02, 1.77, 3.66 |
| Greedy, code prompt | 36.4 to 38.4 tok/s | 42.5 to 42.7 tok/s |
| Greedy, chat prompt | 32.1 to 33.2 tok/s | 34.0 to 34.1 tok/s |
| Temperature 1.0, code prompt | 32.4 to 33.1 tok/s | 31.9 tok/s |
| Temperature 1.0, chat prompt | 29.5 to 30.1 tok/s | 30.7 to 30.8 tok/s |
| Cold prompt of 9,600 tokens | 6.77 to 6.79 s | 7.12 s |
| Cold prompt of 38,400 tokens | 26.30 to 26.44 s | 27.68 to 27.71 s |
| Model on the first machine | 86.14 GiB | 86.98 GiB |
The drafter's linears take 0.57 GiB on the device at 4 bits, by the engine's line, and its cold prompts take about 5% longer. It drafts more tokens a round than MTP on code and fewer on the count and the essay. Both greedy requests are faster with it; at temperature 1.0 the chat request is faster and the code request is not. Two requests at once were also run, but both sent the same prompt, which can flatter a mixture-of-experts model; those cells are not given here. With TF_GLM_DRAFT_CONFIDENCE=0.4, in one launch, the greedy code request ran at 43.4 tok/s.
Speed
The rows of the Speed tables on this account's GLM-5.3-Flash cards, on 2026-11-20, on the build of The engine, with the settings GLM-5.3-Flash BIZ 3.10.0's image gives each TP size: a 1,048,576-token window (Mixed NVFP4 at TP=2 ran at 524,288, below), four streams and 16 kept prompt states at TP=2, eight streams and 32 at TP=3, reserves of 9 and 14 GiB, the vision tower loaded (no image was sent), and at TP=3 a drafts confidence of 0.5 and the image's multirank RoCE setting (TF_GLM_ROCE_MULTI=1). Each request ran alone. Each column is one launch, Mixed NVFP4's before this checkpoint's (and at TP=3 a second launch of each for the 1,035,295-token prompt, after TP=2), so the columns are not paired and the two checkpoints are not compared here; In serving compares them.
| TP=2, Mixed NVFP4 | TP=2, this checkpoint | TP=3, Mixed NVFP4 | TP=3, this checkpoint | |
|---|---|---|---|---|
| Window (tokens) | 524,288 | 1,048,576 | 1,048,576 | 1,048,576 |
| Model on the first machine, vision tower loaded (GiB) | 94.05 | 87.99 | 66.38 | 62.01 |
Decode, bench --kinds decode (tok/s, median of three, drafts on) |
51.47 | 58.77 | 88.93 | 75.01 |
An edit reply, bench --kinds edit (tok/s, median of three, drafts on) |
133.68 | 132.04 | 176.77 | 177.79 |
| Prefill of about 38,960 tokens (tok/s, median of three) | 1,521.7 | 1,467.6 | 1,679.7 | 1,705.0 |
| A 199,307-token prompt, first token (s) | 135.7 ³ | 146.7 ³ | 117.7 | 117.1 |
| A 1,035,295-token prompt, first token (s) | not run: the window | 887.8, no answer ¹ ³ | 690.0 and 675.4 ² | 674.5 and 674.3 ² |
| tool-eval-bench 3.0.0, straight to the engine | 96 | 94 | 94 | 95 |
| tool-eval-bench 3.0.0, through GLM-5.3-Flash BIZ's tool-argument gate | 97 | 96 | 96 | 96 |
Mixed NVFP4 did not start at TP=2 with the 1,048,576-token window, four streams and the vision tower. The engine's line: "--parallel 4 needs 1.16 GiB for its other streams' fixed parts, state runs and a page each, and the CUDA memory budget leaves 13.60 GiB: serve fewer, or free device memory". It started with the next window tried, 524,288 tokens; fewer streams were not tried. The columns also differ in more than the TP size: TP=3 has the drafts confidence and TF_GLM_ROCE_MULTI=1, TP=2's Mixed NVFP4 the smaller window, and the experts' kernel form, left to the engine's timing at load, came out classic in both TP=2 launches and in Mixed NVFP4's first TP=3 launch, and factored in this checkpoint's (the second TP=3 launches swapped them). In serving fixes the form, which moves decode by about 2%.
¹ The first token came after 887.8 s, and the 256 tokens allowed for the reply ended with no answer text, so the three passphrases were not found. At TP=3 both checkpoints answered all three, in 170 tokens, in every take. ² Two takes. In each, the hosts' clock-cap watcher lowered the GPU clock cap from 2,200 to 2,000 MHz on two of the three machines while the prompt was read; the second take came after the caps had gone back up. It did the same in TP=2's take for this checkpoint. Every other speed row ran before any such switch. ³ The engine's own heat band (92 and 88 °C) also paused TP=2 launches for 2 s at a time: in this checkpoint's 1,035,295-token prompt 39 times, its 199,307-token prompt 5 times and the third of its 38,960-token prompts twice (not the median run), and in Mixed NVFP4's 199,307-token prompt twice. No TP=3 speed row was paused. The TP=2 rows with pauses are not headlines.
Decode and the edit reply include drafts (MTP and copies), whose acceptance depends on the text each checkpoint writes, so they compare only with runs of the same prompts. tool-eval-bench ran its 69 scenarios with seed 42, temperature 0, one at a time, 4,096 tokens and reasoning effort low; every run was valid for comparison, and the tool's Safety Gate passed in all eight with no warning. Each score is one run, with no interval, so they do not rank the checkpoints. The gate is GLM-5.3-Flash BIZ v3's glm53_tf tool-gate, which checks tool-call arguments in front of the engine. Free memory at its lowest on the first machine was 8.73 GiB (Mixed NVFP4) and 8.67 GiB (this checkpoint) at TP=2, against the 8 GiB at which this setup's watchdog stops the engine, and 13.89 and 13.92 GiB at TP=3. These figures do not line up with the GLM-5.3-Flash cards' Speed tables: another model, another engine build, and there tool-eval-bench 2.8.0.
Known behaviour
The chat template is the source model's, the same file as Mixed NVFP4's. An assistant turn with no reasoning_content whose content contains </think> is re-rendered by the template: the text before the first </think> goes into the thinking block, and the text between the first and the last </think> is dropped. Passing reasoning_content, even an empty string, keeps the content as written. The template was rendered with plain jinja2 and with transformers' apply_chat_template, outside any serving engine, and the two gave the same text.
In user, system and tool messages, in reasoning_content and in tool-call arguments, <|user|>, <tool_call> and </think> stay ordinary text (checked with TensorFold's /tokenize on this checkpoint).
Not run
- The 100 result-only requests of the GLM-5.3-Flash cards.
- TP=4 or wider, and three or more requests at once.
- Image or video input.
- The source's
deployment_guide.mdnotes that in a non-streaming reply, the model writing</think>or<tool_call>as text can end the reasoning early or hide a tool call; replies were not checked for it here. - Other engines, and TensorFold's 12-bit storage of BF16 matrices (#3509, a draft).
- The MTP layer's draft agreement in PyTorch with this checkpoint's main model. Its experts are Mixed NVFP4's, byte for byte, and Mixed NVFP4's card measures that layer with Mixed NVFP4's main model.
How it is made
The calibration, the smoothing, block 32, +5, the packing and DFloat11
Calibration. 512 rows of 2,048 tokens, in the mix of XSFP4's: Japanese 35% (Nemotron-Post-Training-Dataset-v2 Japanese; LLM-jp Corpus v3's Japanese Wikipedia), local work logs 25% (coding-agent sessions; 42 of the 128 rows are sessions in which Sage itself answered), math 15%, STEM 10% and English conversation 15% (Nemotron-Post-Training-Dataset-v2), rendered with Sage's chat template at reasoning effort high. Sage's tokenizer.json is GLM-5.3-Flash's, byte for byte, so the LLM-jp rows, which use no chat template, were taken from the file made for GLM-5.3-Flash's calibration. Of the 128 scoring documents' 142,493 runs of 16 tokens, 44 also appear in the calibration (0.03%): set phrases of Japanese Wikipedia, a run of the numbers 0 to 5, and a formula. No document is in both.
The smoothing. XSFP4's factors were learned here at block 16, as on GLM-5.3-Flash: each routed expert of layers 3 to 44 on the calibration tokens the router sends it, every expert receiving some, 21 hours on one GB10. They are folded into up and down, so nothing is stored for them. They do not lower KL on Sage. Both at block 32 with +5, with MXFP8 dense MLPs, KL is 0.0459 with the factors and 0.0449 without: a difference of +0.0010 whose interval over documents, [−0.0004, +0.0025], holds zero, while over 605 blocks of 256 tokens it is [+0.0001, +0.0019]. GLM-5.3-Flash showed the same.
Block 32. The factors were not learned again for block 32. They were recovered from the block-16 weights and the experts fitted again: each row's 16-step window and each block's scale by the smallest squared error, among 25 E4M3 candidates around the block's largest weight divided by 6, with no activation statistics. Recovered from the finished up and down, the factors match those recovered the same way from the block-16 checkpoint: in each of the 126 sampled experts they are nearer than for a fit without smoothing, by a mean distance (in |log2| per channel) of at most 0.0030 against at least 0.388. This shows the fold came through the fitting again; the learned values themselves are not stored anywhere to compare.
+5. A weight is rounded to E2M1, a tie going down; then a positive weight within (4.5, 5.5] units of its block's scale takes +5, code 0b1000. Negative weights keep E2M1's marks, and every zero is 0b0000. 2.03% of the weights take +5 (gate 2.00%, up 1.98%, down 2.12%), and the experts' squared error is 0.882 to 0.895 of block 32's without it, layer by layer. In this fit no block's best scale fell at either end of the candidates. The MTP layer's experts keep E2M1. Jammed XSFP4 NZ5's card draws the marks and tells why +5 and not +8; the idea comes from RaZeR.
The packing. CSF0, as in Jammed XSFP4: a base byte a row, and two 4-bit steps a byte for the scales. Every packed matrix was read back and compared with the fitted bytes.
DFloat11. The 150 matrices are stored in the format of Zhang et al. (the reference code, LeanModels/DFloat11, is Apache-2.0), version 0.2.0 of the config's dfloat11_config: each matrix's 8 exponent bits Huffman-coded, its sign and mantissa kept as a byte, with the decoder's lookup tables and positions beside them. The coder was written here. Every coded matrix was decoded with a reference decoder and compared with the BF16 bytes; the 1,361 other tensors of the rewritten files were compared with the plain build's by sha256, and the 129 expert files are the plain build's files. TensorFold decodes them on the host at load (#3807), so in serving they are BF16 matrices like the source's. #3807's GLM loader takes coded weights for lm_head, the embeddings, KDA attention and MLA's kv_b alone, so the dense MLPs, the vision tower and the rest of the BF16 are left plain.
Nothing else learned. No distillation and no training went into this checkpoint; only the smoothing factors come from calibration text.
Where it comes from
| Part | Origin |
|---|---|
| BF16 weights | internlm/Sage-S2.1-Flash, revision 76f4513db7464485605a002a06a6190de0460574 |
| FP8 weights of MLA attention and the shared experts | internlm/Sage-S2.1-Flash-FP8, revision 716e6436905650b7be93e1b7dedfb0bb95afd3a3 |
| MTP layer's routed experts | Bizuayeu/Sage-S2.1-Flash-Mixed-NVFP4, byte for byte |
| Architecture | GLM-5.3-Flash's (zai-org/GLM-5.3-Flash, MIT): the same tensor names and shapes |
| Methods | XSFP4, block 32 and CSF0, and +5 from this account's GLM-5.3-Flash checkpoints (linked above); DFloat11 from Zhang et al. (arXiv 2504.11651) |
| DFlash2 drafter | incoai/GLM-5.3-Flash-DFlash2, revision 6cc6c481eee930a5139619ed919f78726371adbc (Apache-2.0), unchanged |
| License | MIT, by the source README's statement; GLM-5.3-Flash's MIT text in LICENSE-GLM-5.3-Flash; the drafter's Apache-2.0 in dflash2/LICENSE |
Files
| Files | What they hold |
|---|---|
nz5-{gate,up,down}-L03.safetensors to -L44 (126 files, 157.06 GB of tensor data) |
The routed experts of one layer and one projection each: 288 experts' codes, CSF0 scales (weight_scale.csf0_bases, weight_scale.csf0_codes), global and input scales; "code_map": "e2m1_nz5" in each header's __metadata__ |
e2m1-{gate,up,down}-L45.safetensors (3 files, 4.08 GB) |
The MTP layer's routed experts, Mixed NVFP4's files |
model-shell-0000{1..4}-of-00004.safetensors (13.15 GB) |
Everything else: FP8, BF16, F32, and the DFloat11 groups (luts, encoded_exponent, sign_mantissa, output_positions, gaps, split_positions under each coded matrix's name) |
model.safetensors.index.json |
Which file holds each of the 187,441 tensors |
config.json, hf_quant_config.json |
The model's config with ModelOpt's MIXED_PRECISION table (220 entries: 42 NVFP4_NZ5, 1 W4A16_NVFP4, 177 FP8_PB_WO) and dfloat11_config, which names the 150 coded matrices |
df11-manifest.json |
The coded matrices and their sizes before and after |
dflash2/model.safetensors, dflash2/config.json (2.34 GB) |
The DFlash2 drafter, as published by incoai at the revision above |
dflash2/LICENSE, dflash2/README.md |
The Apache License 2.0 text, and a note of where the two files come from with their sha256, both added here |
tokenizer.json, tokenizer_config.json, chat_template.jinja, generation_config.json, processor_config.json, processing_glm5.py, tokenization_glm5.py, deployment_guide.md |
Copied from the source repository at the revision above, the same files as Mixed NVFP4's |
LICENSE-GLM-5.3-Flash |
GLM-5.3-Flash's MIT text |
assets/sage-overview.png, sage-methods.png, sage-closeness.png, sage-speed.png |
This card's figures, drawn here from the card's own numbers |
SHA256SUMS |
The sha256 of the checkpoint's 149 files, the drafter's included: every row above but the license and the figures |
Left out from the source repository: its README.md and README_CN.md (linked above), assets/, projectors/ and vllm_patch/.
License
MIT. The source repository, internlm/Sage-S2.1-Flash, states in its README: "The code and model weights in this repository are released under the MIT License." The official FP8 repository, internlm/Sage-S2.1-Flash-FP8, whose bytes are 2.29 GB of this checkpoint, has the same sentence. As of 2026-11-20 neither ships a LICENSE file or has a licence field on its card, so that sentence is this checkpoint's basis, and there is no copyright line of Sage's to reproduce here. Sage S2.1 Flash has GLM-5.3-Flash's architecture; GLM-5.3-Flash is under the MIT License (Copyright (c) 2026 Z.AI Co., Ltd), whose text is in LICENSE-GLM-5.3-Flash.
The DFlash2 drafter in dflash2/ is incoai's. Its model card gives its licence as Apache-2.0; its repository has no LICENSE file, so dflash2/LICENSE holds the Apache License 2.0 text from apache.org.
The calibration data's Nemotron-Post-Training-Dataset-v2 rows are CC-BY-4.0 (NVIDIA), and its LLM-jp Corpus v3 rows are Japanese Wikipedia text, CC-BY-SA-3.0 as the corpus states. Neither the data nor the local work logs are distributed.
- Downloads last month
- 15
Model tree for Bizuayeu/Sage-S2.1-Flash-Jammed-XSFP4
Base model
internlm/Sage-S2.1-Flash


