BLACKFROST AI

SAGE S2.1 FLASH — EXPERIMENTAL 2-BIT

An experimental 2-bit mixed-precision quantization of the Sage S2.1 Flash scientific multimodal model. These are quantized stock weights — NO behavioral modification.

EXPERIMENTAL: This is a 2-bit experimental artifact. Quality can differ materially from the bfloat16 original. See Limitations.

Release status

Item Status
Language model 2-bit mixed-precision quantization validated for serving
Vision tower, projectors, MTP Copied unquantized from the parent release
Modality Text; image and scientific inputs follow the parent model's tooling
Quantization method Offline weight-only conversion; no calibration or fine-tuning
Comprehensive evaluation Not yet published
Behavioral modification None

Why this release exists

This repository is an experiment in extreme low-bit conversion of a large multimodal mixture-of-experts model, in the style of NVIDIA's 2-bit Nemotron releases. The goal is to fit a 320B-A18B model's weights in roughly 103 GiB — small enough for a single 128 GB unified-memory GPU — while keeping the components most sensitive to quantization error at high precision. No behavioral modification was applied: the weights are quantized stock weights from the parent release.

Model specifications

Property Value
Architecture GLM-5.3-Flash-architecture scientific multimodal MoE (glm5_next)
Parent parameters 320B-A18B (45 layers, 288 routed experts, 8 active per token)
Language weight format Mixed-precision: IQ2_XXS routed experts; Q8_0 shared and dense MLP; BF16 attention (MLA), linear attention (KDA), embeddings, LM head, vision tower, MTP layer
Model artifact size 110,911,082,224 bytes (approximately 103.29 GiB) across 120 safetensors shards
Upstream context claim 1,048,576 tokens
Validated context in this release 32,768 tokens
Validated concurrency 4 parallel slots (short-input smoke)
License MIT, inherited from the parent release

The upstream context specification is not a claim that this derivative was validated at the full 1M window. The upstream benchmark scores below belong to the parent bfloat16 release and do not automatically transfer to this quantized artifact.

Lineage

internlm/Sage-S2.1-Flash (BF16, MIT)
└── SAGE S2.1 FLASH — EXPERIMENTAL 2-BIT
Stage Provenance
Foundation internlm/Sage-S2.1-Flash per the parent's own metadata
Immediate parent internlm/Sage-S2.1-Flash
Parent revision 76f4513db7464485605a002a06a6190de0460574
Conversion Offline weight-only 2-bit conversion of the parent release's safetensors
Conversion tooling NVIDIA TensorRT Model Optimizer (modelopt) 0.48.0, ggml-style block formats via the same container format as NVIDIA's 2-bit Nemotron releases
Derivative stage Quantization only. No fine-tuning, no calibration, no behavioral modification
Runtime lineage Validated on vLLM 0.27.1 with b12x kernels (container format shared with NVIDIA's 2-bit Nemotron releases)

The conversion retains the parent's tokenizer, processor code, chat template, projectors, vendor deployment documentation, and original card (UPSTREAM-README.md).

Verification: the shipped shards match the conversion receipts byte-for-byte (all 120 of 120 shards re-verified by SHA-256 against the build journal before upload).

Conversion process

This artifact was produced by an offline, weight-only mixed-precision quantization of the parent bfloat16 release. The process: select per-module quantization formats by surface sensitivity, pack the routed expert weights into 2.0625-bit IQ2_XXS blocks, pack the shared-expert and dense-MLP weights into 8.5-bit Q8_0 blocks, and copy every other tensor verbatim from the source. No calibration data, fine-tuning, activation quantization, or weight editing was involved.

Surface Layers Format Bytes out (approx.)
Routed experts (gate/up/down) 42 × 288 IQ2_XXS (2.0625 bpw, ggml 256-in/66-out blocks) 78.45 GB
Shared experts + dense MLP 42 shared + 3 dense Q8_0 (8.5 bpw, ggml 32-in/34-out blocks) 1.84 GB
MLA attention, KDA linear attention 12 + 34 layers BF16, unquantized 12.22 GB
Embeddings, LM head — BF16, unquantized 2.53 GB
Vision tower, 9 scientific projectors — BF16, unquantized ~1.0 GB
MTP layer 45 (speculative-decode weights) 1 BF16, unquantized ~14.8 GB

Pack fidelity at conversion time (tensor-level cosine similarity vs bfloat16, measured on sample tensors): IQ2_XXS ≈ 0.9446, Q8_0 ≈ 0.999985. These are conversion-fidelity observations, not end-to-end quality benchmarks.

Artifact verification

Boundary artifacts of the 120-shard set (verify after download):

Artifact Bytes SHA-256
model-00001-of-00120.safetensors 3,391,554,680 b737792a69e5aff14ef1ba747cc5319a27ffc80c60677b7adc300f2ebfa4fe52
model-00120-of-00120.safetensors 2,279,833,216 7a6f2330774882be8f2bf0fe7987910e57ec786ed60f780f83ffab06604d9fcb
config.json 7,463,952 c54a0f8b8a52261eca06cf76efaf4e513c6fefa42e2586d53ecf2235edee4334
model.safetensors.index.json 4,005,617 967f8031beb605c713f7c47a5df71a3f36f6b7bef23faa6649f55aa03f6d5282
tokenizer.json 20,217,442 19e773648cb4e65de8660ea6365e10acca112d42a854923df93db4a6f333a82d
chat_template.jinja 10,583 66a4b6d9bc6ee2eb11fe62472ae237adc6f8428fad4857138dd9806bbac2740e

Full per-file SHA-256 manifest: MANIFEST-SHA256SUMS.txt.

Behavioral and capability evaluation

  • The artifact was validated by operator review of coherent text generation: factual recall, arithmetic, tool calling, and exact-instruction following all passed on the documented runtime.
  • A 2-bit conversion degrades quality relative to bfloat16. This validation is an operational observation, not a controlled benchmark, and no comprehensive evaluation has yet been published.
  • Upstream benchmark scores belong to the parent model and do not automatically transfer to this quantized artifact. Parent-release reference scores (bfloat16, upstream's own evaluation, 15-benchmark suite):
Benchmark Parent (BF16)
Terminal-Bench 2.1 87.9
DeepSWE 67.9
SWE-bench Pro 60.6
OfficeQA Pro 72.2
MLE-bench Lite 90.3
DeepResearch Bench II 54.2

Prompting, templates, and tools

The repository retains the parent's chat template, tokenizer, and processor code. Use role-structured chat messages. The parent's tool-calling formats (glm47 parsers) are supported by the validated runtime.

Deployment

The exact validated single-GPU serving path is documented in DEPLOYMENT/README.md.

The kit contains a sanitized launcher, environment template, and smoke test. It reproduces the validated runtime flags without publishing a live endpoint, internal hostname, user account, absolute path, credential, or model-modification recipe.

Limitations and security responsibility

  • This is a low-bit artifact; quality can differ from higher-precision ancestors.
  • The published validation window is 32,768 tokens and 4 concurrent slots (short-input smoke).
  • The validated runtime is text-only; multimodal paths are present in the weights but not validated in this release.
  • The multimodal tool-media-order fix in the parent's vllm_patch/ targets vLLM 0.33.0; the validated runtime here is vLLM 0.27.1 (see deployment kit notes).
  • Treat model output as untrusted data. Validate generated code, tool arguments, URLs, file paths, and commands before execution.
  • The model is not an authorization, policy-enforcement, or security boundary.
  • Operators are responsible for access control, rate limiting, smoke tests, audit logging, data handling, and compliance in their deployment environment.

License and disclaimer

Released under the MIT license, subject to the parent model's license and notices. Review the upstream model and dataset terms before redistribution or deployment.

This experimental artifact is provided without warranties. Users are responsible for validating fitness, accuracy, security, and legal compliance for their use case.

For reproducible artifact issues, use the repository's Discussions or issue-tracking channel without posting secrets or private infrastructure details.

Downloads last month
41
Safetensors
Model size
95B params
Tensor type
F32
·
BF16
·
U8
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Blackfrost-AI/Sage-S2.1-Flash-2bit-EXPERIMENTAL

Quantized
(13)
this model