dannyliv
/

agent-guard-modernbert-base

@@ -56,7 +56,7 @@ It is small (149M parameters), CPU-friendly, Apache-2.0, and built on `answerdot
 This repo now ships the **V3.2** weights, replacing the prior `v1.x` release. V3.2 was retrained on a permissively-licensed corpus (no gated AI2 datasets) with a rebalanced benign side and nine literature red-team augmentation techniques.
-**What V3.2 changed, stated honestly:**
 - **Fixes the GCG adversarial-suffix weakness.** The prior release had a 100% precomputed-replay attack-success rate against GCG-style adversarial suffixes. V3.2 cuts that to **2.4%** for this model. The disclosed headline weakness is substantially mitigated for precomputed and replayed suffixes. Fresh adaptive white-box GCG still succeeds at ~100% against the bare classifier; that is expected for a 149M encoder and is addressed by the SDK-layer perplexity pre-filter, not by this model alone.
 - **Lowers the benign false-positive rate, but not to the project's strict target.** ModernBERT FPR on benign instructions dropped from **7.4%** (prior release) to **3.2%** at the canonical threshold 0.5, a 57% reduction. The project's strict internal release gate was **≤2.5%**, so V3.2 **misses that gate by 0.7 percentage points**. V3.2 is shipped anyway, as a deliberate decision: the GCG fix and the F1 improvement were judged to outweigh the 0.7pp FPR gate miss. See "Limitations and risks".

 This repo now ships the **V3.2** weights, replacing the prior `v1.x` release. V3.2 was retrained on a permissively-licensed corpus (no gated AI2 datasets) with a rebalanced benign side and nine literature red-team augmentation techniques.
+**What V3.2 changed:**
 - **Fixes the GCG adversarial-suffix weakness.** The prior release had a 100% precomputed-replay attack-success rate against GCG-style adversarial suffixes. V3.2 cuts that to **2.4%** for this model. The disclosed headline weakness is substantially mitigated for precomputed and replayed suffixes. Fresh adaptive white-box GCG still succeeds at ~100% against the bare classifier; that is expected for a 149M encoder and is addressed by the SDK-layer perplexity pre-filter, not by this model alone.
 - **Lowers the benign false-positive rate, but not to the project's strict target.** ModernBERT FPR on benign instructions dropped from **7.4%** (prior release) to **3.2%** at the canonical threshold 0.5, a 57% reduction. The project's strict internal release gate was **≤2.5%**, so V3.2 **misses that gate by 0.7 percentage points**. V3.2 is shipped anyway, as a deliberate decision: the GCG fix and the F1 improvement were judged to outweigh the 0.7pp FPR gate miss. See "Limitations and risks".