Instructions to use mudler/LFM2.5-24B-A2B-APEX-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use mudler/LFM2.5-24B-A2B-APEX-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mudler/LFM2.5-24B-A2B-APEX-GGUF
# Run inference directly in the terminal:
llama cli -hf mudler/LFM2.5-24B-A2B-APEX-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mudler/LFM2.5-24B-A2B-APEX-GGUF
# Run inference directly in the terminal:
llama cli -hf mudler/LFM2.5-24B-A2B-APEX-GGUF

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf mudler/LFM2.5-24B-A2B-APEX-GGUF
# Run inference directly in the terminal:
./build/bin/llama cli -hf mudler/LFM2.5-24B-A2B-APEX-GGUF

Use Docker

docker model run hf.co/mudler/LFM2.5-24B-A2B-APEX-GGUF

Llama new
LM Studio
Jan
Ollama
How to use mudler/LFM2.5-24B-A2B-APEX-GGUF with Ollama:
```
ollama run hf.co/mudler/LFM2.5-24B-A2B-APEX-GGUF
```
Unsloth Desktop

How to use mudler/LFM2.5-24B-A2B-APEX-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mudler/LFM2.5-24B-A2B-APEX-GGUF

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mudler/LFM2.5-24B-A2B-APEX-GGUF"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use mudler/LFM2.5-24B-A2B-APEX-GGUF with Docker Model Runner:
```
docker model run hf.co/mudler/LFM2.5-24B-A2B-APEX-GGUF
```

Lemonade

How to use mudler/LFM2.5-24B-A2B-APEX-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mudler/LFM2.5-24B-A2B-APEX-GGUF

Run and chat with the model

lemonade run user.LFM2.5-24B-A2B-APEX-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use mudler/LFM2.5-24B-A2B-APEX-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mudler/LFM2.5-24B-A2B-APEX-GGUF

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mudler/LFM2.5-24B-A2B-APEX-GGUF

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use mudler/LFM2.5-24B-A2B-APEX-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mudler/LFM2.5-24B-A2B-APEX-GGUF

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "mudler/LFM2.5-24B-A2B-APEX-GGUF" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

LFM2.5-24B-A2B APEX GGUF

APEX (Adaptive Precision for EXpert Models) quantizations of LFM2.5-24B-A2B by LiquidAI.

Brought to you by the LocalAI team | APEX Project | Technical Report

Benchmark Results

Perplexity is measured on wikitext-2-raw. KL divergence is measured against the BF16 model on the same text. HellaSwag and Winogrande use 400 tasks each, zero-shot. The LiquidAI rows are the quantizations published by LiquidAI, measured the same way. Perplexity differences of about 0.2 are close to the measurement error (+/- 0.14).

Model	Size (GiB)	Perplexity	KL vs BF16 (mean)	HellaSwag	Winogrande
APEX I-Quality	14.3	15.99	0.077	69.5	61.8
APEX Quality	14.3	16.36	0.083	67.2	60.2
APEX I-Balanced	16.1	15.93	0.060	67.5	62.5
APEX Balanced	16.1	16.20	0.065	66.2	62.0
APEX I-Compact	10.4	16.20	0.219	66.8	62.8
APEX Compact	10.4	16.83	0.266	65.2	65.0
APEX I-Mini	8.5	16.43	0.436	63.0	62.2
APEX I-Nano	7.1	17.55	0.567	62.2	63.8
LiquidAI Q8_0	23.7	15.94	n/a	67.0	63.0
LiquidAI Q5_K_M	15.8	15.70	0.092	68.0	62.5
LiquidAI Q4_K_M	13.5	15.94	0.164	65.5	63.0

Only these benchmarks were run. No MMLU, ARC or TruthfulQA numbers are claimed.

What is APEX?

APEX is a quantization strategy for Mixture-of-Experts (MoE) models. It classifies tensors by role (routed expert, shared expert, attention) and applies a layer-wise precision gradient: edge layers get higher precision, middle layers get more aggressive compression. The I-variants use imatrix-guided quantization.

See the APEX project for details and the technical report.

Files

File	Size
LFM2.5-24B-A2B-APEX-I-Quality.gguf	14.3 GiB
LFM2.5-24B-A2B-APEX-Quality.gguf	14.3 GiB
LFM2.5-24B-A2B-APEX-I-Balanced.gguf	16.1 GiB
LFM2.5-24B-A2B-APEX-Balanced.gguf	16.1 GiB
LFM2.5-24B-A2B-APEX-I-Compact.gguf	10.4 GiB
LFM2.5-24B-A2B-APEX-Compact.gguf	10.4 GiB
LFM2.5-24B-A2B-APEX-I-Mini.gguf	8.5 GiB
LFM2.5-24B-A2B-APEX-I-Nano.gguf	7.1 GiB

Architecture

Model: LFM2.5-24B-A2B (lfm2_moe) by LiquidAI
Layers: 40 (30 convolutional + 10 full attention, hybrid), the first 2 dense
Experts: 64 routed, 4 active per token
Total parameters: about 24B
Active parameters: about 2B per token

Run with LocalAI

local-ai run mudler/LFM2.5-24B-A2B-APEX-GGUF@LFM2.5-24B-A2B-APEX-I-Balanced.gguf

Credits

APEX is brought to you by the LocalAI team. Developed through human-driven, AI-assisted research. Built on llama.cpp.

Downloads last month: 4,188

GGUF

Model size

24B params

Architecture

lfm2moe

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for mudler/LFM2.5-24B-A2B-APEX-GGUF

Base model

LiquidAI/LFM2.5-24B-A2B-Base

Finetuned

LiquidAI/LFM2.5-24B-A2B

Quantized

(86)

this model

Collection including mudler/LFM2.5-24B-A2B-APEX-GGUF

APEX Quants (GGUF)

Collection

MoE models quantized with the APEX Quantization technique ( https://github.com/mudler/apex-quant ) • 54 items • Updated 10 days ago • 153