Instructions to use PaddlePaddle/PaddleOCR-VL-1.5-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PaddlePaddle/PaddleOCR-VL-1.5-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="PaddlePaddle/PaddleOCR-VL-1.5-GGUF",
	filename="PaddleOCR-VL-1.5-mmproj.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use PaddlePaddle/PaddleOCR-VL-1.5-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf PaddlePaddle/PaddleOCR-VL-1.5-GGUF
# Run inference directly in the terminal:
llama-cli -hf PaddlePaddle/PaddleOCR-VL-1.5-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf PaddlePaddle/PaddleOCR-VL-1.5-GGUF
# Run inference directly in the terminal:
llama-cli -hf PaddlePaddle/PaddleOCR-VL-1.5-GGUF

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf PaddlePaddle/PaddleOCR-VL-1.5-GGUF
# Run inference directly in the terminal:
./llama-cli -hf PaddlePaddle/PaddleOCR-VL-1.5-GGUF

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf PaddlePaddle/PaddleOCR-VL-1.5-GGUF
# Run inference directly in the terminal:
./build/bin/llama-cli -hf PaddlePaddle/PaddleOCR-VL-1.5-GGUF

Use Docker

docker model run hf.co/PaddlePaddle/PaddleOCR-VL-1.5-GGUF

LM Studio
Jan
Ollama
How to use PaddlePaddle/PaddleOCR-VL-1.5-GGUF with Ollama:
```
ollama run hf.co/PaddlePaddle/PaddleOCR-VL-1.5-GGUF
```

Unsloth Studio new

How to use PaddlePaddle/PaddleOCR-VL-1.5-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for PaddlePaddle/PaddleOCR-VL-1.5-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for PaddlePaddle/PaddleOCR-VL-1.5-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for PaddlePaddle/PaddleOCR-VL-1.5-GGUF to start chatting

Docker Model Runner
How to use PaddlePaddle/PaddleOCR-VL-1.5-GGUF with Docker Model Runner:
```
docker model run hf.co/PaddlePaddle/PaddleOCR-VL-1.5-GGUF
```

Lemonade

How to use PaddlePaddle/PaddleOCR-VL-1.5-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull PaddlePaddle/PaddleOCR-VL-1.5-GGUF

Run and chat with the model

lemonade run user.PaddleOCR-VL-1.5-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsing

🔥 Official Website 📝 Technical Report

Introduction

PaddleOCR-VL-1.5 is an advanced next-generation model of PaddleOCR-VL, achieving a new state-of-the-art accuracy of 94.5% on OmniDocBench v1.5. To rigorously evaluate robustness against real-world physical distortions—including scanning artifacts, skew, warping, screen photography, and illumination—we propose the Real5-OmniDocBench benchmark. Experimental results demonstrate that this enhanced model attains SOTA performance on the newly curated benchmark. Furthermore, we extend the model’s capabilities by incorporating seal recognition and text spotting tasks, while remaining a 0.9B ultra-compact VLM with high efficiency.

Model Architecture

PaddleOCR-VL-1.5 Usage with llama.cpp

Install Dependencies

Install PaddlePaddle and PaddleOCR:

# The following command installs the PaddlePaddle version for CUDA 12.6. For other CUDA versions and the CPU version, please refer to https://www.paddlepaddle.org.cn/en/install/quick?docurl=/documentation/docs/en/develop/install/pip/linux-pip_en.html
python -m pip install paddlepaddle-gpu==3.2.1 -i https://www.paddlepaddle.org.cn/packages/stable/cu126/
python -m pip install -U "paddleocr[doc-parser]>=3.4.0"

Please ensure that you install PaddlePaddle framework version 3.2.1 or above, along with the special version of safetensors. For macOS users, please use Docker to set up the environment.

Basic Usage

Start the VLM inference server:

llama-server \
    -m /path/to/PaddleOCR-VL-1.5-GGUF.gguf \
    --mmproj /path/to/PaddleOCR-VL-1.5-GGUF-mmproj.gguf  \
    --port 8080  \
    --host 0.0.0.0 \
    --temp 0

Call the PaddleOCR CLI or Python API:

paddleocr doc_parser \
    -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/paddleocr_vl_demo.png \
    --pipeline_version v1.5 \
    --vl_rec_backend llama-cpp-server \
    --vl_rec_server_url http://127.0.0.1:8080/v1

from paddleocr import PaddleOCRVL
pipeline = PaddleOCRVL(pipeline_version="v1.5", vl_rec_backend="llama-cpp-server", vl_rec_server_url="http://127.0.0.1:8080/v1")
output = pipeline.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/paddleocr_vl_demo.png")
for res in output:
    res.print()
    res.save_to_json(save_path="output")
    res.save_to_markdown(save_path="output")

For more usage details and parameter explanations, see the documentation.

PaddleOCR-VL-1.5-0.9B Usage with llama.cpp

Currently, the PaddleOCR-VL-1.5-0.9B model facilitates seamless inference via the transformers library, supporting comprehensive text spotting and the recognition of complex elements including formulas, tables, charts, and seals. Below is a simple script we provide to support inference using the PaddleOCR-VL-1.5-0.9B model with llama.cpp.

Usage Tips

We have six types of element-level recognition:

Text recognition, indicated by the prompt OCR:.
Formula recognition, indicated by the prompt Formula Recognition:.
Table recognition, indicated by the prompt Table Recognition:.
Chart recognition, indicated by the prompt Chart Recognition:.
Seal recognition, indicated by the prompt Seal Recognition:.

Spotting, indicated by the prompt Spotting:, and need to set image_max_pixels to 1605632:

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
python -m pip install gguf
python ./gguf-py/gguf/scripts/gguf_set_metadata.py /path/to/PaddleOCR-VL-1.5-GGUF/PaddleOCR-VL-1.5-mmproj.gguf clip.vision.image_max_pixels 1605632 --force

# back to default value (1003520): 
# python ./gguf-py/gguf/scripts/gguf_set_metadata.py /path/to/PaddleOCR-VL-1.5-GGUF/PaddleOCR-VL-1.5-mmproj.gguf clip.vision.image_max_pixels 1003520 --force

llama-cli Usage

llama-cli \
    -m /path/to/PaddleOCR-VL-1.5-GGUF/PaddleOCR-VL-1.5.gguf \
    --mmproj /path/to/PaddleOCR-VL-1.5-GGUF/PaddleOCR-VL-1.5-mmproj.gguf \
    -p 'OCR:' \
    --image 'test_image.jpg'

llama-server Usage

llama-server -m /path/to/PaddleOCR-VL-1.5.gguf --mmproj /path/to/PaddleOCR-VL-1.5-GGUF/PaddleOCR-VL-1.5-mmproj.gguf --temp 0

Acknowledgements

We extend our gratitude to @megemini for their significant pull request, which adds support for PaddleOCR-VL-1.5-0.9B and PaddleOCR-VL-0.9B models to llama.cpp.