jialinyyzz commited on
Commit
6b2f6b0
·
verified ·
1 Parent(s): 766072b

Docs: fact chart and a new quantization chart up front; quantization details on GitHub (docs/QUANTIZATION.md)

Browse files
Files changed (5) hide show
  1. .gitattributes +1 -0
  2. README.md +35 -40
  3. USAGE.md +9 -14
  4. USAGE.zh.md +26 -31
  5. assets/quant-top1-en.png +3 -0
.gitattributes CHANGED
@@ -65,3 +65,4 @@ assets/compare-en-forum.png filter=lfs diff=lfs merge=lfs -text
65
  humanizer-12b-bf16.gguf filter=lfs diff=lfs merge=lfs -text
66
  humanizer-12b-IQ2_XS-QAT.gguf filter=lfs diff=lfs merge=lfs -text
67
  humanizer-12b-Q3-QAT.gguf filter=lfs diff=lfs merge=lfs -text
 
 
65
  humanizer-12b-bf16.gguf filter=lfs diff=lfs merge=lfs -text
66
  humanizer-12b-IQ2_XS-QAT.gguf filter=lfs diff=lfs merge=lfs -text
67
  humanizer-12b-Q3-QAT.gguf filter=lfs diff=lfs merge=lfs -text
68
+ assets/quant-top1-en.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -22,63 +22,60 @@ tags:
22
 
23
  <img src="assets/banner-en.png" alt="humanizer: rewrites AI drafts so they read like a person wrote them" width="100%">
24
 
25
- <img src="assets/app-showcase-en.png" alt="The humanizer app running the 12B model locally" width="100%">
26
-
27
  # humanizer
28
 
29
- **A 12B model that rewrites AI-written drafts (emails, essays, reports, forum posts; English and Chinese) so they read like a person wrote them.** It is trained to keep every number, unit, date, name and quote, and to add nothing. It runs locally. No AI detector was used anywhere in training.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
 
31
- **[Usage without the app](USAGE.md)** · [不用 App 怎么用](USAGE.zh.md) · [AGENTS.md (for AI agents)](AGENTS.md) · [GitHub](https://github.com/sgaofen/humanizer-local-model) · [Desktop app (macOS, Windows)](https://github.com/sgaofen/humanizer-local-model/releases/latest) · [Install guide](https://github.com/sgaofen/humanizer-local-model/blob/main/docs/INSTALL.md) · [中文说明](https://github.com/sgaofen/humanizer-local-model/blob/main/README.zh.md)
32
 
33
- > **GGUF files now have their own repo: [jialinyyzz/humanizer-GGUF](https://huggingface.co/jialinyyzz/humanizer-GGUF)**: every quantization (Q8_0, Q6_K, Q4_K_M, Q3, 2-bit) with its size, memory, KL to bf16 and quality on the task, and how they were made. The same files stay here too, so existing download commands keep working.
34
 
35
  > **Setting this up with an AI agent?** Point it at [AGENTS.md](AGENTS.md): exact files, server command, prompt byte for byte, and a self-test.
36
 
37
  ## Quick start
38
 
39
- **App:** download the `.dmg` (Mac with Apple silicon) or the Windows installer from [Releases](https://github.com/sgaofen/humanizer-local-model/releases/latest) (current: [app 0.3.2](https://github.com/sgaofen/humanizer-local-model/releases/tag/app-v0.3.2), which can import `.docx` and PDF files and keeps any passage you mark with **Create fact** word for word). On first run it suggests one of five sizes for your memory (Q8_0, Q6_K, Q4_K_M, Q3 or 2-bit, down to 8 GB machines; app 0.3.0 and later) and downloads it once; after that it works offline. Sizes and quality notes: [Quantized versions](#quantized-versions).
40
 
41
  **Command line, for long documents and agents:** `pipx install git+https://github.com/sgaofen/humanizer-local-model`, then `hz paper.md -o paper.out.md` (also `.txt` and `.docx`). It uses the app or a llama-server, keeps headings, code, tables and links, rewrites the prose piece by piece and flags any piece where a number went missing. See [USAGE.md, section 14](USAGE.md#14-hz-command-line-tool).
42
 
43
  **Without the app:** pick a file below, then follow [Usage without the app](#usage-without-the-app). The full guide, with a batch script, long documents, Chinese and troubleshooting, is **[USAGE.md](USAGE.md)** ([中文](USAGE.zh.md)).
44
 
45
- ## Files
46
-
47
- | File | Size | For |
48
- |---|---|---|
49
- | `humanizer-12b-Q8_0.gguf` | 12,669,630,368 bytes (about 12.7 GB) | 32 GB of memory or more. Recommended. |
50
- | `humanizer-12b-Q6_K.gguf` | 10,029,799,584 bytes (about 10.0 GB) | 16 GB of memory. |
51
- | `humanizer-12b-Q4_K_M.gguf` | 7,625,160,864 bytes (about 7.6 GB) | About 14 GB of memory, or when disk is tight. Quantization-aware trained. |
52
- | `humanizer-12b-Q3-QAT.gguf` | 5,587,794,816 bytes (about 5.6 GB) | 12 GB of memory. 3-bit class, quantization-aware trained; a few more fact slips in English than Q4_K_M (see below). |
53
- | `humanizer-12b-IQ2_XS-QAT.gguf` | 3,893,632,896 bytes (about 3.9 GB) | 8 GB of memory; the smallest. 2-bit; more fact mistakes than the larger files (see below). |
54
- | `humanizer-12b-bf16.gguf` | 23,832,049,568 bytes (about 23.8 GB) | Unquantised weights as one GGUF, for reference or for quantising yourself. |
55
- | `model.safetensors` + `config.json`, `generation_config.json`, `tokenizer.json`, `tokenizer_config.json` | about 24 GB (bf16) | transformers, vLLM, converting to MLX (see [Usage without the app](#usage-without-the-app)). |
56
- | `prompt_format.json` | tiny | The instruction and separator, verbatim. |
57
-
58
- ### Quantized versions
59
 
60
- | File | In short | Bits / type | Size | Peak memory (Mac) ¹ | KL to bf16, EN / ZH ² | Standard `llama-quantize` build, same size class: KL EN / ZH (top token same) ² | Top token same as bf16, EN / ZH ² | Fact check (GLM): rewrites flagged · spots listed ³ | Flagged as AI, same 60 drafts ⁵ |
61
- |---|---|---|---|---|---|---|---|---|---|
62
- | `humanizer-12b-bf16.gguf` | Reference (full precision) | 16-bit (bf16) | 23.8 GB | about 24.8 GB (est.) | 0 (reference) | | 100% (reference) | EN 52 · 160 spots <br> ZH 50 · 216 spots | 4 / 60 |
63
- | `humanizer-12b-Q8_0.gguf` | **Best quality** | 8-bit (Q8_0) | 12.7 GB | 13.7 GB | 0.0017 / 0.0017 | same file: Q8_0 is a standard build | 98.45% / 98.25% | EN 44 · 135 spots <br> ZH 54 · 236 spots | 7 / 60 |
64
- | `humanizer-12b-Q6_K.gguf` | No measurable loss | 6-bit (Q6_K) | 10.0 GB | about 11.0 GB (est.) | 0.0031 / 0.0035 | same file: Q6_K is a standard build | 97.95% / 97.48% | EN 56 · 153 spots <br> ZH 56 · 255 spots | not measured |
65
- | `humanizer-12b-Q4_K_M.gguf` | Slight loss | 4-bit (Q4_K_M), quantization-aware trained | 7.6 GB | 10.0 GB | **0.0136 / 0.0146** | Q4_K_M, 7.6 GB: 0.0203 / 0.0225 (94.46% / 93.46%) | 95.62% / 94.77% | EN 58 · 162 spots <br> ZH 51 · 194 spots <br> (standard Q4_K_M: EN 56 · 163, ZH 48 · 218) | not measured |
66
- | `humanizer-12b-Q3-QAT.gguf` | Small loss; a few more fact slips in English | 3-bit class, mixed precision (about 3.9 bits per weight) ⁴, quantization-aware trained | 5.6 GB | 8.0 GB | **0.0300 / 0.0318** ² | IQ3_XXS (3-bit), 4.7 GB: 0.138 / 0.190 (85.72% / 82.55%) | 93.63% / 92.64% | EN 64 · 210 spots <br> ZH 53 · 233 spots | 4 / 60 |
67
- | `humanizer-12b-IQ2_XS-QAT.gguf` | Lowest AI-detector score; a few more fact slips (mostly single words or numbers): check numbers and names before sending | 2-bit (mostly IQ2_XS) ⁴, quantization-aware trained | 3.9 GB | 6.2 GB | **0.106 / 0.106** ⁴ | IQ2_XS, 3.8 GB: 0.474 / 0.769 (74.43% / 65.53%) | 87.72% / 87.07% | EN 70 · 265 spots <br> ZH 66 · 361 spots | 0 / 60 |
68
 
69
- **What KL means:** how far a file's next-word probabilities drift from the full bf16 model, averaged over every word of real drafts and rewrites. Lower is better; 0 means identical. "Top token same" is how often the file and bf16 would pick the same most likely next word. "Standard" means a plain `llama-quantize` run of the same type with the same importance matrix our build started from, with no extra training.
 
 
 
 
 
 
 
 
 
70
 
71
- The Q4_K_M was refined with quantization-aware training on 2026-10-04: same size and format, about 1/3 lower KL than the standard Q4_K_M. The 2-bit build is closer to the full model than a standard 3-bit build that is 0.8 GB larger, and Chinese, which plain 2-bit quantization hurts most, comes out the same as English. It still makes more fact mistakes than the larger files (see ³), so use it only when memory is tight and check its output with extra care.
72
 
73
- The Q3 build (added 2026-10-05) sits between them: at 5.6 GB it is 2 GB smaller than Q4_K_M, and its KL, about 0.03 in both languages, is a fraction of a standard 3-bit IQ3_XXS (0.138 / 0.190), which is 0.9 GB smaller. Its Chinese fact check and its AI-detector result are on par with bf16. In English it makes a few more fact slips (64 of 420 rewrites flagged, against 52 for bf16 and 58 for Q4_K_M), almost all a single word or phrase, so check numbers and names before you send.
74
 
75
- ¹ Maximum resident memory of llama-server (llama.cpp, Metal) on an Apple M5 Max with the app's settings (8,192-token context, one request at a time) while rewriting. Q4_K_M, Q3 and 2-bit were measured side by side with the app's llama.cpp build on four real drafts; generation ran at about 52, 65 and 69 tokens per second. Q8_0 was measured in an earlier run on one 470-token draft, which read about 1.3 GB lower for the same file (2-bit: 4.9 GB there, 6.2 GB here), so it may need a little more than shown; Q6_K and bf16 are estimated (est.). A 32,768-token context adds about 0.4 GB. Leave room for the system and other apps (the app keeps about 4 GB free).
76
- ² llama.cpp `llama-perplexity --kl-divergence`, 30 chunks of 2,048 tokens per language (about 30,700 scored tokens each) of held-out drafts and rewrites from the evaluation set (no overlap with calibration or training data), reference = the bf16 GGUF, measured on A100 and A30 GPUs (the same file measures within about 3% across GPU models, far below the gaps between files). A full comparison with standard llama.cpp quantization and a third-party imatrix GGUF, IQ1_M to Q8_0, is on the [GGUF page](https://huggingface.co/jialinyyzz/humanizer-GGUF#how-it-compares-to-standard-quantization). Q8_0, Q6_K and Q4_K_M keep the token embeddings and output layer at 8-bit; the IQ2_XS and IQ3_XXS comparison builds use 4-bit embeddings.
77
- ³ The strict fact judge from [Results](#results) (GLM-5.3, one vote per rewrite) on all 420 English and 204 Chinese rewrites (two per draft; for bf16, 4 Chinese rewrites could not be judged). "Rewrites flagged" = rewrites it found a factual problem in; a second pass re-read each flagged rewrite against its draft and listed every problem ("spots"). Most spots are a single word, number or phrase: about 9 in 10 for every file (2-bit: 243 of 265 English, 330 of 361 Chinese). Compared draft by draft with Q8_0, Q6_K and Q4_K_M are within noise, and the quantization-aware Q4_K_M is within noise of the standard one. The 2-bit build is not: in English it was flagged on 70 rewrites against 44 for Q8_0 on the same drafts, a difference beyond noise; in Chinese 66 against 54, within noise. Against the Q4_K_M it is 70 vs. 58 and 66 vs. 51, within noise. Q3: 64 English rewrites flagged (bf16 52, Q4_K_M 58), and 191 of its 210 English spots are a single word or short phrase; in Chinese 53 (bf16 50).
78
- ⁴ The Q3 and 2-bit builds keep about 130,000 of the 262,144 vocabulary tokens: the ones English and Chinese text actually uses, plus everything needed to spell any input. Any text still encodes and decodes exactly; rare symbols, emoji and other scripts just take a few more tokens. Pruning alone moves the model by KL 0.0013 (English) / 0.0002 (Chinese), and their KL above is measured against a bf16 with the same pruned vocabulary. Bits are spread by sensitivity: the 2-bit build is mostly IQ2_XS, with 3- and 4-bit types where they matter; in the Q3 build about half the bytes are Q4_K and a fifth Q6_K, with IQ3_XXS and IQ2_XS on the least sensitive tensors (about 3.9 bits per weight on average). Then each layer is tuned with quantization-aware training and the scales are distilled from the bf16 model on English and Chinese rewriting data. The files report their types as IQ2_XS and IQ3_XXS, but neither is a plain build of that type.
79
- ⁵ The 2-bit build had the best AI-detector result of all versions: 0/60 flagged (Q8_0 7/60, bf16 4/60; Originality.ai strictest setting). A likely reason: the small drift from quantization makes the wording a bit less predictable, which detectors read as less machine-like. Q3: 4/60, the same as bf16 (3 drafts flagged only with Q3, 3 only with bf16).
80
 
81
- sha256 checksums are in [USAGE.md](https://github.com/sgaofen/humanizer-local-model/blob/main/docs/USAGE.md#2-pick-a-file).
82
 
83
  ## Prompt format
84
 
@@ -300,7 +297,7 @@ Public baseline: the `blader/humanizer` skill (v3.1.0, 53k GitHub stars), applie
300
 
301
  <img src="assets/results-detector-en.png" alt="Originality.ai: 95% of rewrites judged human" width="100%">
302
 
303
- **Fact fidelity.** **376 of 420 English rewrites came back with no factual problem** from a strict LLM judge (GLM-5.3, one vote per rewrite), measured on the `humanizer-12b-Q8_0.gguf` file you download. The previous release: 369 of 420.
304
 
305
  | | **v2, this release (Q8_0 file)** | v1, previous 12B |
306
  |---|---|---|
@@ -313,8 +310,6 @@ Where the judge did find a problem, the fix is usually small: a second pass re-r
313
 
314
  **Note: the fact judge is deliberately strict.** In a separate audit of the same judge model, a second judge (Claude Opus) re-read 150 rewrites it had flagged as serious (drawn from training, not from this evaluation set). On the fact the first judge pointed to, Opus agreed it was a serious error in 99, rated it minor in 41 (for example "may reduce" became "will help decrease", or a long name was shortened), and found the fact unchanged in 10. So some flags are harmless rewording. Still read numbers, dates and names before you send.
315
 
316
- <img src="assets/results-fidelity-en.png" alt="Fact fidelity compared with the previous releases" width="100%">
317
-
318
  ## Training
319
 
320
  **No AI detector was used anywhere in training**: not as a reward, not as a filter, not to pick a checkpoint.
 
22
 
23
  <img src="assets/banner-en.png" alt="humanizer: rewrites AI drafts so they read like a person wrote them" width="100%">
24
 
 
 
25
  # humanizer
26
 
27
+ **A 12B model that rewrites AI-written drafts, in English or Chinese, so they read like a person wrote them.** It runs on your own computer and is trained to keep every number, date, name and quote.
28
+
29
+ **Quick start:** get the app for **[Mac (Apple silicon)](https://github.com/sgaofen/humanizer-local-model/releases/download/app-v0.3.2/Humanizer-0.3.2-macos-arm64.dmg)** or **[Windows](https://github.com/sgaofen/humanizer-local-model/releases/download/app-v0.3.2/Humanizer-0.3.2-windows-x64-setup.exe)**, open it and paste a draft; it works offline. Without the app: [llama.cpp and other runtimes](#usage-without-the-app), or [AGENTS.md](AGENTS.md) for coding agents.
30
+
31
+ ## Facts survive the rewrite
32
+
33
+ <img src="assets/results-fidelity-en.png" alt="Fact fidelity: 376 of 420 English rewrites with no factual problem; most fixes are a single word or phrase" width="100%">
34
+
35
+ On our held-out evaluation set, a strict LLM judge found no factual problem in **376 of 420 English rewrites** (210 drafts, two samples each, on the Q8_0 file you download); where it did find one, more than 9 in 10 fixes are a single word or phrase. Originality.ai at its strictest setting judged **95% of the English rewrites human-written** (bf16), although no AI detector was used anywhere in training. Details: [Results](#results).
36
+
37
+ ## Quantization: smaller files, closer to the full model
38
+
39
+ <img src="assets/quant-top1-en.png" alt="Top-1 agreement with bf16 against file size: our GGUF files (green) sit above standard llama.cpp quantization (gray) at 2-bit, Q3 and Q4_K_M; at 3.9 GB, 87% against 70%" width="100%">
40
+
41
+ Most GGUF files come from one `llama-quantize` pass that rounds every weight to the target type; our 2-bit, Q3 and Q4_K_M files go further. In the 2-bit and Q3 files, we measured how much each weight tensor hurts the model when it is squeezed and gave the sensitive tensors more bits and the robust ones fewer. All three were then trained layer by layer to reproduce the full model (quantization-aware training) and distilled from the full bf16 model, used as the teacher, on English and Chinese rewriting data. For the 2-bit file, a standard build of the same size picks the same next token as bf16 70% of the time; these steps take it to 81%, 85% and then 87%. The result is still an ordinary GGUF that any recent llama.cpp runs; Q6_K and Q8_0 are standard builds, because at those sizes there is almost nothing left to recover.
42
 
43
+ <sub>Numbers behind every point, KL, per-file fact checks and how each file was made: [docs/QUANTIZATION.md](https://github.com/sgaofen/humanizer-local-model/blob/main/docs/QUANTIZATION.md). All GGUF files also have their own repo: [jialinyyzz/humanizer-GGUF](https://huggingface.co/jialinyyzz/humanizer-GGUF).</sub>
44
 
45
+ **[Usage without the app](USAGE.md)** · [不用 App 怎么用](USAGE.zh.md) · [AGENTS.md](AGENTS.md) · [GitHub](https://github.com/sgaofen/humanizer-local-model) · [GGUF files](https://huggingface.co/jialinyyzz/humanizer-GGUF) · [Install guide](https://github.com/sgaofen/humanizer-local-model/blob/main/docs/INSTALL.md) · [中文说明](https://github.com/sgaofen/humanizer-local-model/blob/main/README.zh.md)
46
 
47
  > **Setting this up with an AI agent?** Point it at [AGENTS.md](AGENTS.md): exact files, server command, prompt byte for byte, and a self-test.
48
 
49
  ## Quick start
50
 
51
+ **App:** download the `.dmg` (Mac with Apple silicon) or the Windows installer from [Releases](https://github.com/sgaofen/humanizer-local-model/releases/latest) (current: [app 0.3.2](https://github.com/sgaofen/humanizer-local-model/releases/tag/app-v0.3.2), which can import `.docx` and PDF files and keeps any passage you mark with **Create fact** word for word). On first run it suggests one of five sizes for your memory (Q8_0, Q6_K, Q4_K_M, Q3 or 2-bit, down to 8 GB machines; app 0.3.0 and later) and downloads it once; after that it works offline. Sizes and quality notes: [Files](#files).
52
 
53
  **Command line, for long documents and agents:** `pipx install git+https://github.com/sgaofen/humanizer-local-model`, then `hz paper.md -o paper.out.md` (also `.txt` and `.docx`). It uses the app or a llama-server, keeps headings, code, tables and links, rewrites the prose piece by piece and flags any piece where a number went missing. See [USAGE.md, section 14](USAGE.md#14-hz-command-line-tool).
54
 
55
  **Without the app:** pick a file below, then follow [Usage without the app](#usage-without-the-app). The full guide, with a batch script, long documents, Chinese and troubleshooting, is **[USAGE.md](USAGE.md)** ([中文](USAGE.zh.md)).
56
 
57
+ <img src="assets/app-showcase-en.png" alt="The humanizer app running the 12B model locally: draft on the left, rewrite on the right, new wording highlighted" width="100%">
 
 
 
 
 
 
 
 
 
 
 
 
 
58
 
59
+ ## Files
 
 
 
 
 
 
 
60
 
61
+ | File | Size | Peak memory (Mac) ¹ | Top-1 vs. bf16 ² | For |
62
+ |---|---|---|---|---|
63
+ | `humanizer-12b-Q8_0.gguf` | 12,669,630,368 bytes (12.7 GB) | 13.7 GB | 98.4% | 32 GB of memory or more. **Recommended**, best quality. |
64
+ | `humanizer-12b-Q6_K.gguf` | 10,029,799,584 bytes (10.0 GB) | about 11.0 GB | 97.7% | 16 GB. No measurable loss. |
65
+ | `humanizer-12b-Q4_K_M.gguf` | 7,625,160,864 bytes (7.6 GB) | 10.0 GB | 95.2% | About 14 GB, or when disk is tight. Slight loss; quantization-aware trained. |
66
+ | `humanizer-12b-Q3-QAT.gguf` | 5,587,794,816 bytes (5.6 GB) | 8.0 GB | 93.1% | 12 GB. Small loss; a few more fact slips in English. |
67
+ | `humanizer-12b-IQ2_XS-QAT.gguf` | 3,893,632,896 bytes (3.9 GB) | 6.2 GB | 87.4% | 8 GB; the smallest (2-bit). Lowest AI-detector score, a few more fact slips: check numbers and names. |
68
+ | `humanizer-12b-bf16.gguf` | 23,832,049,568 bytes (23.8 GB) | about 24.8 GB | 100% (reference) | Unquantised weights as one GGUF, for reference or for quantising yourself. |
69
+ | `model.safetensors` + `config.json`, `generation_config.json`, `tokenizer.json`, `tokenizer_config.json` | about 24 GB (bf16) | | | transformers, vLLM, converting to MLX (see [Usage without the app](#usage-without-the-app)). |
70
+ | `prompt_format.json` | tiny | | | The instruction and separator, verbatim. |
71
 
72
+ ¹ `llama-server` (Metal) on an M5 Max with the app's settings, while rewriting; Q6_K and bf16 are estimates. Leave room for the system (the app keeps about 4 GB free).
73
 
74
+ ² How often the file's most likely next token is the same as bf16's, English and Chinese averaged (llama.cpp `--kl-divergence`).
75
 
76
+ ### Quantized versions
 
 
 
 
77
 
78
+ The same GGUF files are also in **[jialinyyzz/humanizer-GGUF](https://huggingface.co/jialinyyzz/humanizer-GGUF)**; they stay here too, so existing download commands keep working. KL to bf16, the fact check and AI-detector results for every file, the full comparison with standard llama.cpp quantization and how each file was made: **[docs/QUANTIZATION.md](https://github.com/sgaofen/humanizer-local-model/blob/main/docs/QUANTIZATION.md)**. sha256 checksums are in [USAGE.md](USAGE.md#2-pick-a-file).
79
 
80
  ## Prompt format
81
 
 
297
 
298
  <img src="assets/results-detector-en.png" alt="Originality.ai: 95% of rewrites judged human" width="100%">
299
 
300
+ **Fact fidelity.** **376 of 420 English rewrites came back with no factual problem** from a strict LLM judge (GLM-5.3, one vote per rewrite), measured on the `humanizer-12b-Q8_0.gguf` file you download. The previous release: 369 of 420. The chart is at the [top of this page](#facts-survive-the-rewrite).
301
 
302
  | | **v2, this release (Q8_0 file)** | v1, previous 12B |
303
  |---|---|---|
 
310
 
311
  **Note: the fact judge is deliberately strict.** In a separate audit of the same judge model, a second judge (Claude Opus) re-read 150 rewrites it had flagged as serious (drawn from training, not from this evaluation set). On the fact the first judge pointed to, Opus agreed it was a serious error in 99, rated it minor in 41 (for example "may reduce" became "will help decrease", or a long name was shortened), and found the fact unchanged in 10. So some flags are harmless rewording. Still read numbers, dates and names before you send.
312
 
 
 
313
  ## Training
314
 
315
  **No AI detector was used anywhere in training**: not as a reward, not as a filter, not to pick a checkpoint.
USAGE.md CHANGED
@@ -84,20 +84,7 @@ Also in the repo:
84
  - `humanizer-12b-bf16.gguf` (23,832,049,568 bytes, about 23.8 GB): the unquantised 12B as one GGUF, for reference or for quantising yourself.
85
  - `model.safetensors` (bf16, about 24 GB) with `config.json`, `generation_config.json`, `tokenizer.json`, `tokenizer_config.json` and `prompt_format.json` at the root: what transformers, vLLM and the MLX converter use.
86
 
87
- **How much the quantised files differ from bf16.** In Q8_0, Q6_K and Q4_K_M the token embeddings and the output layer stay at 8-bit; Q6_K and Q4_K_M are also imatrix-calibrated on our own rewriting data. KL was measured over about 33,000 tokens of drafts and rewrites from the evaluation set (no overlap with the calibration data). The last column is the strict fact judge from the README on all 420 English rewrites of the evaluation set, run on each file with llama.cpp.
88
-
89
- | File | Mean KL vs. bf16 | Top token same as bf16 | Perplexity | No factual problem (English) |
90
- |---|---|---|---|---|
91
- | bf16 (reference) | | | | 368 / 420 |
92
- | Q8_0 | 0.0015 | 98.4% | +0.3% | 376 / 420 |
93
- | Q6_K | 0.0031 | 97.7% | +0.6% | 364 / 420 |
94
- | Q4_K_M (updated 2026-10-04, see below) | 0.0136 ¹ | 95.6% ¹ | | 362 / 420 |
95
- | Q3 (`Q3-QAT`, added 2026-10-05) | 0.0300 ¹ | 93.6% ¹ | | 356 / 420 |
96
- | 2-bit (`IQ2_XS-QAT`) | 0.106 ¹ | 87.7% ¹ | | 350 / 420 |
97
-
98
- Compared draft by draft with bf16, Q8_0, Q6_K and Q4_K_M are within noise on the fact judge. Q3 and 2-bit make a few more fact slips in English (64 and 70 rewrites flagged, against 52 for bf16), mostly a single word or number; in Chinese Q3 is on par with bf16 (53 vs. 50 of 204 flagged). Full table, Chinese results and how these two were made: [README, Quantized versions](https://github.com/sgaofen/humanizer-local-model#quantized-versions) or the [GGUF repo](https://huggingface.co/jialinyyzz/humanizer-GGUF).
99
-
100
- **Q4_K_M was refined on 2026-10-04 with quantization-aware training:** same size and format, about 1/3 lower KL to the full-precision model than a standard Q4_K_M. ¹ Measured on a larger KL set (30 blocks of English drafts and rewrites), where the standard Q4_K_M scores 0.0203 and 94.5% (Chinese: 0.0146 vs. 0.0225). On the fact judge, compared draft by draft with the standard Q4_K_M, it is within noise: 58 vs. 56 of 420 English rewrites flagged, 162 vs. 163 problems listed by the second pass, more than 9 in 10 of them a single word or phrase. Q3 and 2-bit were measured on the same larger set (Chinese: 0.0318 and 0.106; measured on A100 and A30 GPUs, which differ by about 3%).
101
 
102
  **Download:**
103
 
@@ -124,6 +111,13 @@ sha256 (`shasum -a 256 FILE` on macOS, `sha256sum FILE` on Linux, `certutil -has
124
 
125
  All GGUF files were replaced on 2026-10-04: their metadata now holds a chat template (for LM Studio and other chat front ends; see [section 3](#start-a-server)) and the recommended sampling defaults (temperature 1.0, top-p 0.95, top-k off, min-p off, repetition penalty 1.0), so llama.cpp uses them when a request doesn't set its own. The weights inside are byte for byte the same; only the header changed. Copies downloaded before that have the earlier sizes and checksums, and still work through the completion endpoint with the sampling settings passed explicitly. `humanizer-12b-bf16.gguf` (23,832,049,568 bytes, about 23.8 GB) was added the same day: the unquantised weights as one GGUF, for reference or for quantising yourself. `humanizer-12b-Q3-QAT.gguf` (added 2026-10-05) carries the same template and defaults.
126
 
 
 
 
 
 
 
 
127
  ## 3. llama.cpp (recommended)
128
 
129
  Works on macOS (Metal), Windows and Linux (CUDA, Vulkan or CPU). The app itself runs llama.cpp build `b11335`; that build or a newer one is fine.
@@ -557,6 +551,7 @@ The copy ratio here is a rough measure (share of the rewrite's 5-word or 5-chara
557
  - Each piece is rewritten without seeing the others, so tone can shift a little between pieces and a fact can't move from one piece to another. Read the joined result once from top to bottom.
558
  - Headings, code blocks and bullet lists are often dropped or turned into prose. Rewrite only the prose and keep headings and code yourself; [`humanizer/markdown_guard.py`](https://github.com/sgaofen/humanizer-local-model/blob/main/humanizer/markdown_guard.py) does this block by block.
559
  - **[`hz`](#14-hz-command-line-tool) does all of this in one command**, for `.md`, `.txt` and `.docx`: it keeps headings, code, tables and links, splits the prose into pieces of about 350 words (600 Chinese characters) without crossing a heading, and checks every piece.
 
560
 
561
  ## 11. Chinese
562
 
 
84
  - `humanizer-12b-bf16.gguf` (23,832,049,568 bytes, about 23.8 GB): the unquantised 12B as one GGUF, for reference or for quantising yourself.
85
  - `model.safetensors` (bf16, about 24 GB) with `config.json`, `generation_config.json`, `tokenizer.json`, `tokenizer_config.json` and `prompt_format.json` at the root: what transformers, vLLM and the MLX converter use.
86
 
87
+ **How much the quantised files differ from bf16.** How often each file picks the same most likely next token as bf16, English and Chinese averaged: Q8_0 98.4%, Q6_K 97.7%, Q4_K_M 95.2%, Q3 93.1%, 2-bit 87.4%. Q4_K_M, Q3 and 2-bit are quantization-aware trained and distilled from bf16, so they stay closer to it than standard llama.cpp files of the same size. On the strict fact judge from the README, Q8_0, Q6_K and Q4_K_M are within noise of bf16; Q3 and 2-bit make a few more fact slips in English (64 and 70 of 420 rewrites flagged, against 52 for bf16), mostly a single word or number. The comparison with standard quantization, KL, the full fact-check table and how the files were made: [docs/QUANTIZATION.md](https://github.com/sgaofen/humanizer-local-model/blob/main/docs/QUANTIZATION.md).
 
 
 
 
 
 
 
 
 
 
 
 
 
88
 
89
  **Download:**
90
 
 
111
 
112
  All GGUF files were replaced on 2026-10-04: their metadata now holds a chat template (for LM Studio and other chat front ends; see [section 3](#start-a-server)) and the recommended sampling defaults (temperature 1.0, top-p 0.95, top-k off, min-p off, repetition penalty 1.0), so llama.cpp uses them when a request doesn't set its own. The weights inside are byte for byte the same; only the header changed. Copies downloaded before that have the earlier sizes and checksums, and still work through the completion endpoint with the sampling settings passed explicitly. `humanizer-12b-bf16.gguf` (23,832,049,568 bytes, about 23.8 GB) was added the same day: the unquantised weights as one GGUF, for reference or for quantising yourself. `humanizer-12b-Q3-QAT.gguf` (added 2026-10-05) carries the same template and defaults.
113
 
114
+ **Is my file the newest?** Files are sometimes re-uploaded under the same name. Compare your file's SHA-256 with the one Hugging Face reports, and download again if they differ (the app does this itself: **…** → **Check for updates**):
115
+
116
+ ```bash
117
+ curl -sI https://huggingface.co/jialinyyzz/humanizer/resolve/main/humanizer-12b-Q8_0.gguf | grep -i '^x-linked-etag'
118
+ shasum -a 256 ./humanizer-model/humanizer-12b-Q8_0.gguf # Linux: sha256sum
119
+ ```
120
+
121
  ## 3. llama.cpp (recommended)
122
 
123
  Works on macOS (Metal), Windows and Linux (CUDA, Vulkan or CPU). The app itself runs llama.cpp build `b11335`; that build or a newer one is fine.
 
551
  - Each piece is rewritten without seeing the others, so tone can shift a little between pieces and a fact can't move from one piece to another. Read the joined result once from top to bottom.
552
  - Headings, code blocks and bullet lists are often dropped or turned into prose. Rewrite only the prose and keep headings and code yourself; [`humanizer/markdown_guard.py`](https://github.com/sgaofen/humanizer-local-model/blob/main/humanizer/markdown_guard.py) does this block by block.
553
  - **[`hz`](#14-hz-command-line-tool) does all of this in one command**, for `.md`, `.txt` and `.docx`: it keeps headings, code, tables and links, splits the prose into pieces of about 350 words (600 Chinese characters) without crossing a heading, and checks every piece.
554
+ - The app (0.3.2 and later) can import a `.docx` or PDF and shows a note on long drafts, but it rewrites the whole draft in one request; its **Create fact** keeps a selected passage word for word by sending a placeholder in its place and rewriting again if the placeholder doesn't come back intact. For long or structured documents, `hz` is still the better tool.
555
 
556
  ## 11. Chinese
557
 
USAGE.zh.md CHANGED
@@ -4,9 +4,9 @@
4
 
5
  这份文档写给不想用[桌面 App](https://github.com/sgaofen/humanizer-local-model/releases/latest)、要在自己的代码或命令行里跑 humanizer 的人。每个代码块都可以直接复制。
6
 
7
- **只是想改写文件?**命令行工具 `hz` 替你处理提示词、采样参数、长文切块和检查,后端用 App 或 llama-server 都行:`pipx install git+https://github.com/sgaofen/humanizer-local-model`,然后 `hz draft.md -o out.md`。见[第 14 节](#14-命令行工具-hz)。
8
 
9
- **目录:**[1. 这个模型哪里特殊](#1-这个模型哪里特殊) · [2. 选哪个文件](#2-选哪个文件) · [3. llama.cpp](#3-llamacpp推荐) · [4. MLX](#4-mlxapple-芯片) · [5. transformers](#5-transformerscuda) · [6. vLLM](#6-vllm) · [7. Ollama](#7-ollama) · [8. LM Studio](#8-lm-studio) · [9. 批量改写一个文件夹](#9-批量改写一个文件夹) · [10. 长文](#10-长文) · [11. 中文](#11-中文) · [12. 质量检查清单](#12-质量检查清单) · [13. 排错](#13-排错) · [14. 命令行工具 hz](#14-命令行工具-hz)
10
 
11
  ## 1. 这个模型哪里特殊
12
 
@@ -14,10 +14,10 @@ humanizer 是一个 12B 的**文本续写**模型,由 `google/gemma-4-12B` 微
14
 
15
  | 规则 | 原因 |
16
  |---|---|
17
- | **它是文本续写模型,不是聊天模型。**把一个纯文本字符串发到续写接口。聊天模式只能靠 GGUF 文件里自带的对话模板(2026-10-04 起才有):它把最后一条用户消息拼成和下面一模一样的字符串,系统提示词和之前的对话都不用。 | 它训练时见到的就是下面这种原始文本。通用的聊天模板(Gemma 轮次、ChatML)会给草稿套上它训练时从没见过的标记。safetensors 权重(transformers、vLLM、MLX)没有对话模板。 |
18
  | **提示词必须逐字一致。** | 指令改写过、翻译过,或者少一个空行,效果都会变差。 |
19
- | **只靠 EOS 停。**不要设停止符,尤其不要用 `###`。 | 模型会自己结束。少数正常输出里本来就有 `###`,会被截断。 |
20
- | **采样:temperature 1.0、top-p 0.95,别的都关掉。**top-k 关(0),min-p 关(0),重复惩罚 1.0。 | 评测就是这样跑的。好几个运行环境默认会开别的采样:llama.cpp 默认 top-k 40、min-p 0.05;transformers 会从随权重附带的 `generation_config.json` 里读到 top-k 64。 |
21
 
22
  ### 提示词
23
 
@@ -65,7 +65,7 @@ def build_prompt(draft: str) -> str:
65
  assert hashlib.sha256(build_prompt("X").encode("utf-8")).hexdigest()[:16] == "cc51d66b4c593fbe"
66
  ```
67
 
68
- **输出长度。**改写通常和草稿差不多长。输出上限留草稿 token 数的 2.5 倍左右;App 用的是 `min(2048, max(256, 2.5 × 草稿 token 数))`。指令、草稿和改写加起来共用 8192 token 的上下文,见[长文](#10-长文)。
69
 
70
  ## 2. 选哪个文件
71
 
@@ -84,20 +84,7 @@ assert hashlib.sha256(build_prompt("X").encode("utf-8")).hexdigest()[:16] == "cc
84
  - `humanizer-12b-bf16.gguf`(23,832,049,568 字节,约 23.8 GB):不量化的 12B,单个 GGUF,给需要参照或想自己量化的人。
85
  - 根目录的 `model.safetensors`(bf16,约 24 GB),以及 `config.json`、`generation_config.json`、`tokenizer.json`、`tokenizer_config.json`、`prompt_format.json`:transformers、vLLM 和 MLX 转换都用它们。
86
 
87
- **量化版和 bf16 差多少。**Q8_0、Q6_K、Q4_K_M 的词表和输出层都保留 8 bit;Q6_K 和 Q4_K_M 另外用我们自己的改写数据做了 imatrix 校准。KL 在评测集草稿和改写上测了约 33,000 个 token(和校准数据不重叠)。最后一栏是 README 里那个从严的事实判官,用 llama.cpp 分别跑每个文件、判评测集全部 420 篇英文改写。
88
-
89
- | 文件 | 与 bf16 的平均 KL | 首选词与 bf16 一致 | 困惑度 | 没挑出事实问题(英文) |
90
- |---|---|---|---|---|
91
- | bf16(基准) | | | | 368 / 420 |
92
- | Q8_0 | 0.0015 | 98.4% | +0.3% | 376 / 420 |
93
- | Q6_K | 0.0031 | 97.7% | +0.6% | 364 / 420 |
94
- | Q4_K_M(2026-10-04 更新,见下) | 0.0136 ¹ | 95.6% ¹ | | 362 / 420 |
95
- | Q3(`Q3-QAT`,2026-10-05 加入) | 0.0300 ¹ | 93.6% ¹ | | 356 / 420 |
96
- | 2 bit(`IQ2_XS-QAT`) | 0.106 ¹ | 87.7% ¹ | | 350 / 420 |
97
-
98
- 逐篇和 bf16 对比,Q8_0、Q6_K、Q4_K_M 在事实判官上的差别都在噪声范围内。Q3 和 2 bit 英文事实小错稍多(标出 64 篇、70 篇,bf16 是 52 篇),多是一个词或一个数字;中文 Q3 和 bf16 持平(204 篇里标出 53 篇对 50 篇)。完整表格、中文结果和这两档怎么做的,见 [README 的量化版本一节](https://github.com/sgaofen/humanizer-local-model/blob/main/README.zh.md#量化版本)或 [GGUF 仓库](https://huggingface.co/jialinyyzz/humanizer-GGUF)。
99
-
100
- **Q4_K_M 在 2026-10-04 用量化感知训练重新做过一遍:**大小和格式不变,和全精度模型的 KL 比普通 Q4_K_M 低约三分之一。¹ 这两个数是在更大的一套 KL 测试上量的(30 块英文草稿和改写),普通 Q4_K_M 在同一套上是 0.0203 和 94.5%(中文:0.0146 对 0.0225)。事实判官上,和普通 Q4_K_M 逐篇对比在噪声范围内:英文 420 篇改写里标出 58 篇对 56 篇,第二遍复核列出的问题 162 处对 163 处,9 成以上只是一个词或短语。Q3 和 2 bit 也是在这套更大的测试上量的(中文:0.0318 和 0.106;在 A100 和 A30 两种卡上测,卡型之间只差约 3%)。
101
 
102
  **下载:**
103
 
@@ -124,6 +111,13 @@ sha256 校验值(macOS 用 `shasum -a 256 文件名`,Linux 用 `sha256sum
124
 
125
  所有 GGUF 文件在 2026-10-04 换过一次:元数据里加了对话模板(给 LM Studio 等聊天软件用,见[第 3 节](#起服务)),并写入推荐的采样默认值(temperature 1.0、top-p 0.95、top-k 关、min-p 关、重复惩罚 1.0),请求里没设采样参数时 llama.cpp 就用它们。里面的权重逐字节没变,只改了文件头。在那之前下载的文件大小和校验值都是旧的,用续写接口、显式传采样参数照样能用。同一天还加了 `humanizer-12b-bf16.gguf`(23,832,049,568 字节,约 23.8 GB):不量化的完整权重,单个 GGUF,给需要参照或想自己量化的人。`humanizer-12b-Q3-QAT.gguf`(2026-10-05 加入)带同样的模板和默认值。
126
 
 
 
 
 
 
 
 
127
  ## 3. llama.cpp(推荐)
128
 
129
  macOS(Metal)、Windows、Linux(CUDA、Vulkan 或 CPU)都能用。App 自己用的是 llama.cpp `b11335`,这个版本或更新的都可以。
@@ -151,7 +145,7 @@ llama-server -m ./humanizer-model/humanizer-12b-Q8_0.gguf -c 8192 -np 1 -ngl 99
151
 
152
  用 `/completion` 接口,见下面的例子;不管哪天下载的文件都能用。
153
 
154
- **聊天接口。**如果你的 GGUF 是 2026-10-04 或之后下载的(大小和 sha256 见[第 2 节](#2-选哪个文件)),并且服务带 `--jinja`(新版默认就开),`/v1/chat/completions` 也能用。文件里自带的对话模板把最后一条用户消息当作草稿,拼出和上面一模一样的提示词;系统提示词和之前的对话都不用,所以一次请求发一篇草稿。我们用 llama.cpp 核对过:模板拼出的提示词逐字相同;同一个 seed 下两个接口给出同一篇改写,都会自己停。更早下载的文件没有对话模板,用聊天接口会出怪输出。
155
 
156
  ```bash
157
  jq -n --rawfile d draft.txt '{messages: [{role: "user", content: $d}],
@@ -384,7 +378,7 @@ LM Studio 我们自己没测过。LM Studio 用的是 GGUF 里存的对话模板
384
 
385
  1. 加载 `humanizer-12b-Q8_0.gguf`(或 Q6_K / Q4_K_M / Q3-QAT / IQ2_XS-QAT),加载时把上下文长度设成 8192。
386
  2. 在模型的采样设置里设 **Temperature 1.0、Top P 0.95、Top K 0、Min P 0、Repeat Penalty 1.0**,删掉所有停止符。提示词模板保持文件自带的,别改。
387
- 3. **聊天页面:**系统提示词留空(模板反正不用它),一条消息贴一篇草稿。每条消息各改各的,之前的对话不会发给模型。本地服务的 `/v1/chat/completions` 也是这样。
388
  4. **文本续写**(哪天下载的文件都能用):在 Developer 页开本地服务,把完整提示词(指令 + 草稿 + 分隔符)发到 **`/v1/completions`**:
389
 
390
  ```python
@@ -557,6 +551,7 @@ if __name__ == "__main__":
557
  - 每段改写时看不到其他段,所以段与段之间语气可能略有变化,事实也不会在段之间挪动。拼好之后从头到尾读一遍。
558
  - 标题、代码块和列表常被删掉或改成正文。只改写正文,标题和代码自己留着;[`humanizer/markdown_guard.py`](https://github.com/sgaofen/humanizer-local-model/blob/main/humanizer/markdown_guard.py) 就是这样逐块处理的。
559
  - **[`hz`](#14-命令行工具-hz) 一行命令就能做完上面这些**,支持 `.md`、`.txt` 和 `.docx`:标题、代码、表格、链接原样保留,正文按约 350 个英文词(中文约 600 字)一块切开,不跨标题,每块都做检查。
 
560
 
561
  ## 11. 中文
562
 
@@ -569,11 +564,11 @@ if __name__ == "__main__":
569
 
570
  ## 12. 质量检查清单
571
 
572
- - **通读一遍改写。**核对每个数字、日期、单位和人名,以及每个论断的方向(谁做了什么、多还是少、先还是后)。在我们的评测集上,英文 420 篇改写里有 376 篇判官没挑出事实问题;有问题的,9 成以上改一个词或短语就好,比如“37 complaints”(37 条投诉)变成了“37% of complaints”(37% 的投诉)。
573
- - **补回需要的格式:**主题行、列表、标题、落款有时会丢(420 发里有 28 发)。
574
- - **和草稿太像?再采一次。**每次都是重新采样。
575
- - **随意体裁里语气会跑。**Reddit 一类的帖子里,它有时会加上原文没有的俚语或粗口,请删掉。
576
- - **不保证过检测器。**我们的检测数字是一个检测器在某一天的一次测量;模板化的体裁(带 emoji、井号的社交帖,政策备忘)仍然最常被判 AI。这里不对任何检测结果做承诺。
577
  - 这是给你改自己草稿的写作工具。学校、单位或出版方对 AI 辅助有规定的,请按规定来。
578
 
579
  ## 13. 排错
@@ -592,7 +587,7 @@ if __name__ == "__main__":
592
  | 下载时 404 | 文件名写错 | 用[第 2 节](#2-选哪个文件)里的文件名 |
593
  | 指纹自检不过 | 提示词拼法和训练时不一样 | 直接复制[第 1 节](#提示词)里的 `build_prompt` |
594
 
595
- **自检。**[AGENTS.md 第 8 节](https://github.com/sgaofen/humanizer-local-model/blob/main/AGENTS.md#8-self-test-verify-the-install)有一个只用标准库的脚本:它把评测集里的一篇真实草稿发给 llama-server,检查提示词格式、接口、聊天模板有没有漏进来、有没有照抄。设置正确会打印 `PASS`。
596
 
597
  ## 14. 命令行工具 hz
598
 
@@ -643,7 +638,7 @@ hz paper.md --server http://127.0.0.1:8080 # 指定某个 llama-server
643
  - 用空行分块。下面这些**原样保留**,不送给模型:Markdown 标题(`#`、下划线式标题,或整行只有 `**粗体**`)、围栏和缩进代码块、表格、只有图片或链接的行、HTML 块和注释、`$$…$$` 与 `\[…\]` 公式、引用块、分隔线、YAML front matter、脚注和链接定义、紧挨在列表/表格/代码块前面以冒号结尾的短句(如"主要成效如下:"),以及 References、Bibliography、Works Cited、Sources、参考文献等标题下面的全部内容。
644
  - 正文段落按顺序拼成块,每块最多约 **350 个英文词或 600 个汉字**(`--max-words`、`--max-chars`)。块不会跨过标题或任何保留块;同一段连续正文切出的几块大小大致相同。
645
  - 单段超长时按句子切开,各块改完再拼回同一段。
646
- - **列表:**每一项保留原来的项目符号或编号,以及开头的 `**小标题:**`。一项有约 15 个英文词(26 个汉字)以上就单独改写,更短的保持原样。短列表项是模型最弱的地方,请重点看。
647
  - 改写后的块放回原来的位置,块与块之间的空行照旧。正文里的行内格式(粗体、行内代码、链接)由模型决定,有时会丢。
648
 
649
  ### 检查和重写
@@ -662,7 +657,7 @@ hz paper.md --server http://127.0.0.1:8080 # 指定某个 llama-server
662
 
663
  `added_numbers`(改写里有、草稿里没有的数字)**只报告、不重写**:模型有时是做了正确的算术("成本从 $480k 降到 $305k"被写成"省了 $175k"),有时是编出来的数字。不管哪种,都要看一眼。
664
 
665
- **语言不对。**在这些检查之前,写成了另一种语言的改写(英文草稿写成中文,或中文草稿写成英文;只数字符)会用同样的参数重新生成,最多 3 次。`pieces[].language_resampled` 记重采了几次;3 次都不对的话,这块会标上 `language`。
666
 
667
  重写之后仍有问题的块会在 stderr 列出行号(`.docx` 为段落序号);用 `--json` 时标为 `"flagged": true`。这种情况退出码仍是 0。
668
 
@@ -729,7 +724,7 @@ hz paper.md --server http://127.0.0.1:8080 # 指定某个 llama-server
729
 
730
  ### 局限
731
 
732
- - 检查只管数字、链接、照抄、长度、重复和多余的 HTML 标签。**改了一个词、一个人名或句子意思,它查不出来。**真机中文测试里,"覆盖全部 48 家门店"被改成"覆盖全省 48 家门店","时有发生"被改成"经常出现",数字却一个没错。请通读结果。
733
  - 每块改写时看不到其他块,所以块与块之间语气可能略有变化。
734
  - 短列表项最弱:模型有时会丢掉短项的主语,或者把同一句话写两遍。
735
  - 用汉字写的数字只能部分识别,三和 3 之间来回换写法时,两个方向都可能漏报或误报。
 
4
 
5
  这份文档写给不想用[桌面 App](https://github.com/sgaofen/humanizer-local-model/releases/latest)、要在自己的代码或命令行里跑 humanizer 的人。每个代码块都可以直接复制。
6
 
7
+ **只是想改写文件**?命令行工具 `hz` 替你处理提示词、采样参数、长文切块和检查,后端用 App 或 llama-server 都行:`pipx install git+https://github.com/sgaofen/humanizer-local-model`,然后 `hz draft.md -o out.md`。见[第 14 节](#14-命令行工具-hz)。
8
 
9
+ **目录**:[1. 这个模型哪里特殊](#1-这个模型哪里特殊) · [2. 选哪个文件](#2-选哪个文件) · [3. llama.cpp](#3-llamacpp推荐) · [4. MLX](#4-mlxapple-芯片) · [5. transformers](#5-transformerscuda) · [6. vLLM](#6-vllm) · [7. Ollama](#7-ollama) · [8. LM Studio](#8-lm-studio) · [9. 批量改写一个文件夹](#9-批量改写一个文件夹) · [10. 长文](#10-长文) · [11. 中文](#11-中文) · [12. 质量检查清单](#12-质量检查清单) · [13. 排错](#13-排错) · [14. 命令行工具 hz](#14-命令行工具-hz)
10
 
11
  ## 1. 这个模型哪里特殊
12
 
 
14
 
15
  | 规则 | 原因 |
16
  |---|---|
17
+ | **它是文本续写模型,不是聊天模型**。把一个纯文本字符串发到续写接口。聊天模式只能靠 GGUF 文件里自带的对话模板(2026-10-04 起才有):它把最后一条用户消息拼成和下面一模一样的字符串,系统提示词和之前的对话都不用。 | 它训练时见到的就是下面这种原始文本。通用的聊天模板(Gemma 轮次、ChatML)会给草稿套上它训练时从没见过的标记。safetensors 权重(transformers、vLLM、MLX)没有对话模板。 |
18
  | **提示词必须逐字一致。** | 指令改写过、翻译过,或者少一个空行,效果都会变差。 |
19
+ | **只靠 EOS 停**。不要设停止符,尤其不要用 `###`。 | 模型会自己结束。少数正常输出里本来就有 `###`,会被截断。 |
20
+ | **采样:temperature 1.0、top-p 0.95,别的都关掉**。top-k 关(0),min-p 关(0),重复惩罚 1.0。 | 评测就是这样跑的。好几个运行环境默认会开别的采样:llama.cpp 默认 top-k 40、min-p 0.05;transformers 会从随权重附带的 `generation_config.json` 里读到 top-k 64。 |
21
 
22
  ### 提示词
23
 
 
65
  assert hashlib.sha256(build_prompt("X").encode("utf-8")).hexdigest()[:16] == "cc51d66b4c593fbe"
66
  ```
67
 
68
+ **输出长度**。改写通常和草稿差不多长。输出上限留草稿 token 数的 2.5 倍左右;App 用的是 `min(2048, max(256, 2.5 × 草稿 token 数))`。指令、草稿和改写加起来共用 8192 token 的上下文,见[长文](#10-长文)。
69
 
70
  ## 2. 选哪个文件
71
 
 
84
  - `humanizer-12b-bf16.gguf`(23,832,049,568 字节,约 23.8 GB):不量化的 12B,单个 GGUF,给需要参照或想自己量化的人。
85
  - 根目录的 `model.safetensors`(bf16,约 24 GB),以及 `config.json`、`generation_config.json`、`tokenizer.json`、`tokenizer_config.json`、`prompt_format.json`:transformers、vLLM 和 MLX 转换都用它们。
86
 
87
+ **量化版和 bf16 差多少**。量化文件认为最可能的下一个词和 bf16 相同的比例(中英文平均):Q8_0 98.4%、Q6_K 97.7%、Q4_K_M 95.2%、Q3 93.1%、2-bit 87.4%。Q4_K_M、Q3 和 2-bit 做过量化感知训练,并从 bf16 蒸馏过,所以比同样大小的普通 llama.cpp 量化更接近完整模型。用 README 里那个从严的事实判官看,Q8_0、Q6_K、Q4_K_M 和 bf16 的差别都在噪声范围内;Q3 和 2-bit 英文事实小错稍多(420 篇改写里标出 64 篇、70 篇,bf16 是 52 篇),多是一个词或一个数字。和普通量化的对比、KL、完整的事实检查表以及这些文件是怎么做的,见 [docs/QUANTIZATION.zh.md](https://github.com/sgaofen/humanizer-local-model/blob/main/docs/QUANTIZATION.zh.md)。
 
 
 
 
 
 
 
 
 
 
 
 
 
88
 
89
  **下载:**
90
 
 
111
 
112
  所有 GGUF 文件在 2026-10-04 换过一次:元数据里加了对话模板(给 LM Studio 等聊天软件用,见[第 3 节](#起服务)),并写入推荐的采样默认值(temperature 1.0、top-p 0.95、top-k 关、min-p 关、重复惩罚 1.0),请求里没设采样参数时 llama.cpp 就用它们。里面的权重逐字节没变,只改了文件头。在那之前下载的文件大小和校验值都是旧的,用续写接口、显式传采样参数照样能用。同一天还加了 `humanizer-12b-bf16.gguf`(23,832,049,568 字节,约 23.8 GB):不量化的完整权重,单个 GGUF,给需要参照或想自己量化的人。`humanizer-12b-Q3-QAT.gguf`(2026-10-05 加入)带同样的模板和默认值。
113
 
114
+ **手上的文件是不是最新的**?文件偶尔会用同一个名字重新上传。拿本地文件的 SHA-256 和 Hugging Face 报的比一下,不一样就重新下载(App 会自己做这件事:「…」→「检查更新」):
115
+
116
+ ```bash
117
+ curl -sI https://huggingface.co/jialinyyzz/humanizer/resolve/main/humanizer-12b-Q8_0.gguf | grep -i '^x-linked-etag'
118
+ shasum -a 256 ./humanizer-model/humanizer-12b-Q8_0.gguf # Linux 用 sha256sum
119
+ ```
120
+
121
  ## 3. llama.cpp(推荐)
122
 
123
  macOS(Metal)、Windows、Linux(CUDA、Vulkan 或 CPU)都能用。App 自己用的是 llama.cpp `b11335`,这个版本或更新的都可以。
 
145
 
146
  用 `/completion` 接口,见下面的例子;不管哪天下载的文件都能用。
147
 
148
+ **聊天接口**。如果你的 GGUF 是 2026-10-04 或之后下载的(大小和 sha256 见[第 2 节](#2-选哪个文件)),并且服务带 `--jinja`(新版默认就开),`/v1/chat/completions` 也能用。文件里自带的对话模板把最后一条用户消息当作草稿,拼出和上面一模一样的提示词;系统提示词和之前的对话都不用,所以一次请求发一篇草稿。我们用 llama.cpp 核对过:模板拼出的提示词逐字相同;同一个 seed 下两个接口给出同一篇改写,都会自己停。更早下载的文件没有对话模板,用聊天接口会出怪输出。
149
 
150
  ```bash
151
  jq -n --rawfile d draft.txt '{messages: [{role: "user", content: $d}],
 
378
 
379
  1. 加载 `humanizer-12b-Q8_0.gguf`(或 Q6_K / Q4_K_M / Q3-QAT / IQ2_XS-QAT),加载时把上下文长度设成 8192。
380
  2. 在模型的采样设置里设 **Temperature 1.0、Top P 0.95、Top K 0、Min P 0、Repeat Penalty 1.0**,删掉所有停止符。提示词模板保持文件自带的,别改。
381
+ 3. **聊天页面**:系统提示词留空(模板反正不用它),一条消息贴一篇草稿。每条消息各改各的,之前的对话不会发给模型。本地服务的 `/v1/chat/completions` 也是这样。
382
  4. **文本续写**(哪天下载的文件都能用):在 Developer 页开本地服务,把完整提示词(指令 + 草稿 + 分隔符)发到 **`/v1/completions`**:
383
 
384
  ```python
 
551
  - 每段改写时看不到其他段,所以段与段之间语气可能略有变化,事实也不会在段之间挪动。拼好之后从头到尾读一遍。
552
  - 标题、代码块和列表常被删掉或改成正文。只改写正文,标题和代码自己留着;[`humanizer/markdown_guard.py`](https://github.com/sgaofen/humanizer-local-model/blob/main/humanizer/markdown_guard.py) 就是这样逐块处理的。
553
  - **[`hz`](#14-命令行工具-hz) 一行命令就能做完上面这些**,支持 `.md`、`.txt` 和 `.docx`:标题、代码、表格、链接原样保留,正文按约 350 个英文词(中文约 600 字)一块切开,不跨标题,每块都做检查。
554
+ - App(0.3.2 及以后)可以导入 `.docx` 和 PDF,草稿太长时会提示,但它是整篇一次改写;它的「原样保留」(Create fact)把选中的文字换成占位符送给模型,占位符没原样回来就重写,从而保证这段文字一字不改。长文档或有结构的文档仍然推荐用 `hz`。
555
 
556
  ## 11. 中文
557
 
 
564
 
565
  ## 12. 质量检查清单
566
 
567
+ - **通读一遍改写**。核对每个数字、日期、单位和人名,以及每个论断的方向(谁做了什么、多还是少、先还是后)。在我们的评测集上,英文 420 篇改写里有 376 篇判官没挑出事实问题;有问题的,9 成以上改一个词或短语就好,比如“37 complaints”(37 条投诉)变成了“37% of complaints”(37% 的投诉)。
568
+ - **补回需要的格式**:主题行、列表、标题、落款有时会丢(420 发里有 28 发)。
569
+ - **和草稿太像?再采一次**。每次都是重新采样。
570
+ - **随意体裁里语气会跑**。Reddit 一类的帖子里,它有时会加上原文没有的俚语或粗口,请删掉。
571
+ - **不保证过检测器**。我们的检测��字是一个检测器在某一天的一次测量;模板化的体裁(带 emoji、井号的社交帖,政策备忘)仍然最常被判 AI。这里不对任何检测结果做承诺。
572
  - 这是给你改自己草稿的写作工具。学校、单位或出版方对 AI 辅助有规定的,请按规定来。
573
 
574
  ## 13. 排错
 
587
  | 下载时 404 | 文件名写错 | 用[第 2 节](#2-选哪个文件)里的文件名 |
588
  | 指纹自检不过 | 提示词拼法和训练时不一样 | 直接复制[第 1 节](#提示词)里的 `build_prompt` |
589
 
590
+ **自检**。[AGENTS.md 第 8 节](https://github.com/sgaofen/humanizer-local-model/blob/main/AGENTS.md#8-self-test-verify-the-install)有一个只用标准库的脚本:它把评测集里的一篇真实草稿发给 llama-server,检查提示词格式、接口、聊天模板有没有漏进来、有没有照抄。设置正确会打印 `PASS`。
591
 
592
  ## 14. 命令行工具 hz
593
 
 
638
  - 用空行分块。下面这些**原样保留**,不送给模型:Markdown 标题(`#`、下划线式标题,或整行只有 `**粗体**`)、围栏和缩进代码块、表格、只有图片或链接的行、HTML 块和注释、`$$…$$` 与 `\[…\]` 公式、引用块、分隔线、YAML front matter、脚注和链接定义、紧挨在列表/表格/代码块前面以冒号结尾的短句(如"主要成效如下:"),以及 References、Bibliography、Works Cited、Sources、参考文献等标题下面的全部内容。
639
  - 正文段落按顺序拼成块,每块最多约 **350 个英文词或 600 个汉字**(`--max-words`、`--max-chars`)。块不会跨过标题或任何保留块;同一段连续正文切出的几块大小大致相同。
640
  - 单段超长时按句子切开,各块改完再拼回同一段。
641
+ - **列表**:每一项保留原来的项目符号或编号,以及开头的 `**小标题:**`。一项有约 15 个英文词(26 个汉字)以上就单独改写,更短的保持原样。短列表项是模型最弱的地方,请重点看。
642
  - 改写后的块放回原来的位置,块与块之间的空行照旧。正文里的行内格式(粗体、行内代码、链接)由模型决定,有时会丢。
643
 
644
  ### 检查和重写
 
657
 
658
  `added_numbers`(改写里有、草稿里没有的数字)**只报告、不重写**:模型有时是做了正确的算术("成本从 $480k 降到 $305k"被写成"省了 $175k"),有时是编出来的数字。不管哪种,都要看一眼。
659
 
660
+ **语言不对**。在这些检查之前,写成了另一种语言的改写(英文草稿写成中文,或中文草稿写成英文;只数字符)会用同样的参数重新生成,最多 3 次。`pieces[].language_resampled` 记重采了几次;3 次都不对的话,这块会标上 `language`。
661
 
662
  重写之后仍有问题的块会在 stderr 列出行号(`.docx` 为段落序号);用 `--json` 时标为 `"flagged": true`。这种情况退出码仍是 0。
663
 
 
724
 
725
  ### 局限
726
 
727
+ - 检查只管数字、链接、照抄、长度、重复和多余的 HTML 标签。**改了一个词、一个人名或句子意思,它查不出来**。真机中文测试里,"覆盖全部 48 家门店"被改成"覆盖全省 48 家门店","时有发生"被改成"经常出现",数字却一个没错。请通读结果。
728
  - 每块改写时看不到其他块,所以块与块之间语气可能略有变化。
729
  - 短列表项最弱:模型有时会丢掉短项的主语,或者把同一句话写两遍。
730
  - 用汉字写的数字只能部分识别,三和 3 之间来回换写法时,两个方向都可能漏报或误报。
assets/quant-top1-en.png ADDED

Git LFS Details

  • SHA256: 67652ee0cbfb951d7cdd46334f0e1dabc92920954eae7946b0857aca69e29bf1
  • Pointer size: 131 Bytes
  • Size of remote file: 283 kB