Qwen2.5-1.5B-Instruct-HCGM

Paper: HCGM Paper: No GPU, No KV Cache Paper: Compute Once
GitHub mrail State Neural Engine License

Qwen2.5-1.5B-Instruct compiled by the Holographic Causal Graph Machine (HCGM) into a source-free bundle (holo-source-free-qwen/v1): no original checkpoint, no PyTorch, no KV cache, no training.

The HCGM as a flow of bytes, drawn to scale

The machine as a flow of bytes, drawn to scale: checkpoint β†’ lossless C0 graph β†’ source-free bundle; a generic executor serves any number of live states of constant size.

  • Knowledge as a causal graph. Every SwiGLU neuron is stored as an exact nonlinear hyperedge d_j Β· SiLU(g_jα΅€x) Β· (u_jα΅€x) in pages of six-bit codes (one BF16 scale per 16 weights); the tied embedding is a lexical graph with a SimHash index; the attention cores stay BF16. 1.79 GB in total.
  • Constant state instead of a KV cache. Attention over the past is holographic Ward recall: per layer and key–value head an exact frontier of the last 128 tokens, 383 atoms (count, mean key, mean value) merged by Ward's minimum-variance criterion, and one pinned atom for the first token. The complete state of a conversation is 14.4 MiB at any length β€” the source model's cache after 528 tokens, growing by 28 KiB with every further token.
  • Exact within 512 tokens, close beyond: the recall is exact while a context fits the 512 rows, and costs +0.031 nat per token at 2 048, +0.080 at 4 096 and +0.145 at 8 192 tokens.

Holographic Ward recall

Holographic Ward recall: one read of one query head over the pinned first token, the exact frontier and the atoms (left); measured atom sizes after 4 096 tokens (right).

Three papers in paper/ describe the model and what its constant state makes possible:

  • The Holographic Causal Graph Machine β€” the compiler, the bundle, holographic Ward recall with the pinned first-token atom, and the full evaluation against the source model.
  • No GPU, No KV Cache β€” the Apple Neural Engine runtime: 64 conversations share one pass, 470 tokens/s on a Mac mini M4, 99.97 % token identity with the fp32 reference runtime.
  • Compute Once β€” answers become reusable tracks: a map of earlier runs (.mrail) answers any known context in milliseconds with the model's own output, restores exact states, drafts new answers for exact verification, and grows with use.

Quality against the source model

fp32 source model and fp32 reference runtime of the bundle, same tokenizer, raw results and scripts in eval/.

Qwen2.5-1.5B-Instruct HCGM difference
WikiText-2 test, 32 Γ— 512 tokens, nat/token 2.4454 2.4493 +0.0038
HumanEval reference programs, nat/token 0.9432 0.9429 βˆ’0.0003
top-1 agreement with the source, WikiText-2 / code – 96.7 % / 98.5 %
WikiText-2, 8 Γ— 2 048 tokens, nat/token 2.1300 2.1615 +0.031
WikiText-2, 6 Γ— 4 096 tokens, nat/token 2.1202 2.1997 +0.080
WikiText-2, 4 Γ— 8 192 tokens, nat/token 2.0928 2.2374 +0.145
HumanEval pass@1 (greedy, chat template, 384 new tokens) 89/164 (54.3 %) 99/164 (60.4 %)

The HCGM HumanEval answers were generated by the Neural Engine runtime (64 conversations per pass) and every token was checked against the fp32 reference runtime by teacher forcing: 30 268 of 30 278 tokens identical (99.97 %); at the ten remaining positions the reference prefers its own token by at most 0.03 nat.

State size against quality

Cost of the constant state by position

Top: state per conversation and quality at 2 048 tokens for five state sizes, with and without the pinned first-token atom; the source model's KV cache at 2 048 tokens is 56 MiB. Bottom: Ξ”NLL against the source by position, 2k–8k-token windows.

Speed on a Mac mini M4 (16 GB), Neural Engine, no GPU

1 conversation, greedy 18.0 tok/s
1 conversation + coding map (1 641 runs), held-out coding tasks 39.4 tok/s (2.27Γ— greedy)
a question that is already in the map 160 tokens in 3–4 ms
64 conversations per pass 470 tok/s total
state per conversation 14.4 MiB, constant

The service loop on the Neural Engine

One pass of 64 conversations on the Neural Engine: 28 layer calls back to back while eight CPU workers plan the constant state and a copier fills the device buffers (top); all 64 slots busy over a run of 240 requests (bottom).

Cost of a pass

A pass costs almost the same for 1 and for 64 token columns; that flat cost is spent on 64 conversations per pass.

Quickstart (reference runtime, any OS)

pip install numpy msgpack tokenizers huggingface_hub
hf download tfwnotops/Qwen2.5-1.5B-Instruct-HCGM --local-dir Qwen2.5-1.5B-Instruct-HCGM
cd Qwen2.5-1.5B-Instruct-HCGM
python3 run.py "What is the capital of Germany?"

The reference runtime (runtime/holo_surgery) is the exact fp32 definition of the model; sessions default to the released configuration (frontier 128, 384 atoms, first token pinned). The Neural Engine runtime below is the fast path.

Apple Neural Engine runtime (macOS, Apple silicon)

ane/ holds the compiled wheel of the Neural Engine runtime: block decoding, draft-tree verification with maps, continuous batching of up to 64 conversations, and an OpenAI-compatible HTTP service with the map in front of the model. It is distributed as a binary under the research license in ane/LICENSE (free for non-commercial research and evaluation; commercial use by license).

pip install ane/hcgm_ane-0.2.0-*.whl          # CPython 3.13, macOS 15+, Apple silicon
python -m hcgm_ane serve --model . --map maps/coding.mrail --port 8811
curl -s localhost:8811/v1/chat/completions \
  -d '{"messages":[{"role":"user","content":"Write a Python quicksort."}],"max_tokens":256}'

The first start compiles 28 layer programs for the Neural Engine (about a minute) and caches them.

Maps

Maps of this exact model, format .mrail version 3 (specification and dependency-free reader/writer: github.com/DT-Foss/mrail):

  • maps/coding.mrail β€” 1 641 coding runs, 296k tokens, 1.2 MB
  • maps/general.mrail β€” 3 000 general runs, 463k tokens, 1.9 MB
  • maps/humaneval.mrail β€” the 164 HumanEval answers above

A map changes speed, never output: every draft is verified by the model, and a stored answer is served only for a context that lies exactly on a stored run of this model. The service appends every verified answer to its map, so a map grows finer with use.

Share your map. Maps merge like files (python -m mrail merge team.mrail a.mrail b.mrail). Runs from a foreign map are drafts only, so a shared map makes this model faster on its topic without changing a single output token. Maps of repositories, documentation sets, tool protocols or everyday questions are welcome as community contributions.

Answers as tracks

Answers as tracks: a stored answer is read from the file in 3–4 ms instead of 10 s, a new task restores the last station and decodes with drafts from the map, a follow-up computes only its 23 new tokens.

Speed grows with use

Speed grows with use: accepted tokens per verification pass on held-out coding requests against the number of runs in the map.

Files

  • artifacts/, tokenizer files β€” the bundle (Apache-2.0)
  • runtime/, run.py, restore_artifacts.py β€” the reference runtime (research license, runtime/LICENSE)
  • ane/ β€” Neural Engine runtime wheel (research license)
  • maps/ β€” .mrail maps
  • paper/ β€” the three papers
  • assets/ β€” the figures of this card, from the papers
  • eval/ β€” evaluation scripts (research license, runtime/LICENSE) and raw results (results/; results_f64r128/: the same evaluation with the compact state of 64 frontier tokens and 128 atoms, 5.6 MiB, a point on the size–quality curve of the HCGM paper)

Citation

@misc{foss2026hcgm,
  author = {Foss, David Tom},
  title  = {The Holographic Causal Graph Machine: Compiling a Pretrained Transformer into a Constant-State Model without Training},
  year   = {2026},
  doi    = {10.13140/RG.2.2.28206.68167}
}
@misc{foss2026ane,
  author = {Foss, David Tom},
  title  = {No GPU, No KV Cache: A Constant-State Language Model on the Neural Engine of a Mac mini},
  year   = {2026},
  doi    = {10.13140/RG.2.2.31562.12484}
}
@misc{foss2026computeonce,
  author = {Foss, David Tom},
  title  = {Compute Once: Constant-State Language Models Make Answers Reusable and Agents Cheap},
  year   = {2026},
  doi    = {10.13140/RG.2.2.18140.35202}
}

License

Model files: Apache License 2.0 (LICENSE, NOTICE), compiled from Qwen/Qwen2.5-1.5B-Instruct. Reference runtime (runtime/, run.py, restore_artifacts.py) and the scripts in eval/: research license (runtime/LICENSE; free for non-commercial research and evaluation, commercial use by license from Wirkung Inc.). Neural Engine runtime in ane/: research license (ane/LICENSE). Author: David Tom Foss (d.foss@ieee.org, ORCID 0009-0004-0289-7154).

Downloads last month
246
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for tfwnotops/Qwen2.5-1.5B-Instruct-HCGM

Finetuned
(2,164)
this model