Instructions to use yarenty/qwen2.5-3B-datafusion-instruct-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use yarenty/qwen2.5-3B-datafusion-instruct-gguf with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="yarenty/qwen2.5-3B-datafusion-instruct-gguf",
	filename="model.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use yarenty/qwen2.5-3B-datafusion-instruct-gguf with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf yarenty/qwen2.5-3B-datafusion-instruct-gguf
# Run inference directly in the terminal:
llama-cli -hf yarenty/qwen2.5-3B-datafusion-instruct-gguf

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf yarenty/qwen2.5-3B-datafusion-instruct-gguf
# Run inference directly in the terminal:
llama-cli -hf yarenty/qwen2.5-3B-datafusion-instruct-gguf

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf yarenty/qwen2.5-3B-datafusion-instruct-gguf
# Run inference directly in the terminal:
./llama-cli -hf yarenty/qwen2.5-3B-datafusion-instruct-gguf

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf yarenty/qwen2.5-3B-datafusion-instruct-gguf
# Run inference directly in the terminal:
./build/bin/llama-cli -hf yarenty/qwen2.5-3B-datafusion-instruct-gguf

Use Docker

docker model run hf.co/yarenty/qwen2.5-3B-datafusion-instruct-gguf

LM Studio
Jan
Ollama
How to use yarenty/qwen2.5-3B-datafusion-instruct-gguf with Ollama:
```
ollama run hf.co/yarenty/qwen2.5-3B-datafusion-instruct-gguf
```

Unsloth Studio new

How to use yarenty/qwen2.5-3B-datafusion-instruct-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for yarenty/qwen2.5-3B-datafusion-instruct-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for yarenty/qwen2.5-3B-datafusion-instruct-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for yarenty/qwen2.5-3B-datafusion-instruct-gguf to start chatting

Pi new

How to use yarenty/qwen2.5-3B-datafusion-instruct-gguf with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf yarenty/qwen2.5-3B-datafusion-instruct-gguf

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "yarenty/qwen2.5-3B-datafusion-instruct-gguf"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use yarenty/qwen2.5-3B-datafusion-instruct-gguf with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf yarenty/qwen2.5-3B-datafusion-instruct-gguf

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default yarenty/qwen2.5-3B-datafusion-instruct-gguf

Run Hermes

hermes

Docker Model Runner
How to use yarenty/qwen2.5-3B-datafusion-instruct-gguf with Docker Model Runner:
```
docker model run hf.co/yarenty/qwen2.5-3B-datafusion-instruct-gguf
```

Lemonade

How to use yarenty/qwen2.5-3B-datafusion-instruct-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull yarenty/qwen2.5-3B-datafusion-instruct-gguf

Run and chat with the model

lemonade run user.qwen2.5-3B-datafusion-instruct-gguf-{{QUANT_TAG}}

List all available models

lemonade list

Qwen2.5-3B-DataFusion-Instruct GGUF Model

Model Overview

Model Name: Qwen2.5-3B-DataFusion-Instruct
Model Type: Fine-tuned Large Language Model
Base Model: Qwen2.5-3B
Specialization: DataFusion SQL Engine and Rust Programming
Format: GGUF (GGML Universal Format)
License: Apache 2.0

Model Description

This is a specialized fine-tuned version of the Qwen2.5-3B model, specifically trained on comprehensive DataFusion ecosystem data to excel at Rust programming, DataFusion SQL queries, and data processing tasks. The model has been optimized to provide accurate, idiomatic code examples and clear technical explanations.

Model Files

Main Model

File: model.gguf (5.8GB)
Type: Full precision GGUF model
Use Case: Production environments, highest accuracy requirements
Recommended For: Development, debugging, complex queries

Quantized Model

File: qwen2.5-3B-datafusion.gguf (1.8GB)
Type: Quantized GGUF model (optimized for inference)
Use Case: Resource-constrained environments, faster inference
Recommended For: Deployment, testing, resource-limited scenarios

Training Data

Dataset Composition

Total QA Pairs: 265,180
Source Projects: 36 different repositories
Content Types: Code implementation, documentation, usage examples
Coverage: Comprehensive DataFusion ecosystem

Training Projects

Core DataFusion: datafusion, datafusion-ballista, datafusion-federation
DataFusion Extensions: datafusion-functions-json, datafusion-postgres, datafusion-python
Arrow Ecosystem: arrow-rs, arrow-zarr
Related Tools: blaze, exon, feldera, greptimedb, horaedb, influxdb
Modern Data Stack: iceberg-rust, LakeSoul, lance, openobserve, parseable

Data Quality Features

Structured JSONL format with source attribution
Code examples with best practices and common pitfalls
Error handling guidance and troubleshooting solutions
Performance optimization tips and best practices

Model Capabilities

Primary Strengths

Rust Programming Expertise
- Idiomatic Rust code generation
- DataFusion API usage patterns
- Error handling and testing best practices
- Performance optimization techniques
DataFusion SQL Mastery
- Complex SQL query construction
- Table provider implementations
- UDF (User-Defined Function) development
- Query optimization and execution planning
Data Processing Knowledge
- Arrow format operations
- Parquet file handling
- Data transformation pipelines
- Streaming and batch processing
System Architecture Understanding
- Distributed query execution
- Federation and integration patterns
- Observability and tracing
- Performance monitoring

Technical Domains

SQL Engine Internals: Query planning, optimization, execution
Data Formats: Arrow, Parquet, JSON, CSV, Avro
Storage Systems: Object storage, databases, file systems
Distributed Computing: Ray, Ballista, cluster management
Streaming: Real-time data processing, windowing, aggregations

Usage Instructions

System Prompt

The model is configured with a specialized system prompt:

You are a helpful, concise, and accurate coding assistant specialized in Rust and the DataFusion SQL engine. Always provide high-level, idiomatic Rust code, DataFusion SQL examples, clear documentation, and robust test cases. Your answers should be precise, actionable, and end with '### End'.

Prompt Template

### Instruction:
{{ .Prompt }}

### Response:

Stop Sequences

### Instruction:
### Response:
### End

Generation Parameters

num_predict: 1024 (maximum tokens to generate)
repeat_penalty: 1.2 (prevents repetitive output)
temperature: 0.7 (balanced creativity vs consistency)
top_p: 0.9 (nucleus sampling for quality)

Performance Characteristics

Accuracy

Code Generation: High accuracy for Rust and DataFusion patterns
SQL Queries: Correct syntax and best practices
Documentation: Clear, actionable explanations
Error Handling: Comprehensive coverage of common issues

Efficiency

Main Model: Highest accuracy, larger memory footprint
Quantized Model: Optimized inference, reduced memory usage
Response Time: Fast generation with proper stop sequences
Memory Usage: Efficient token management

Installation and Setup

Ollama (Recommended)

# Pull the model
ollama pull jaro/qwen_datafusion

# Run inference
ollama run jaro/qwen_datafusion

Direct GGUF Usage

# Using llama.cpp or compatible tools
./llama -m model.gguf -p "How do I create a custom UDF in DataFusion?"

Model Comparison

Aspect	Main Model (5.8GB)	Quantized Model (1.8GB)
Accuracy	Highest	High (slight degradation)
Memory Usage	Higher	Lower
Inference Speed	Standard	Faster
Deployment	Development/Production	Production/Resource-constrained
Use Case	Maximum quality	Balanced performance

Resources

DataFusion Documentation: https://docs.datafusion.org/
Apache Arrow: https://arrow.apache.org/
Rust Programming Language: https://www.rust-lang.org/
Training Dataset: Available in https://huggingface.co/datasets/yarenty/datafusion_QA

Citation

When using this model in research or publications, please cite:

@software{qwen2.5_3b_datafusion_instruct,
  title={Qwen2.5-3B-DataFusion-Instruct: A Specialized Model for DataFusion Ecosystem},
  author={Fine-tuned on DataFusion Ecosystem QA Dataset},
  year={2025},
  url={https://github.com/yarenty/trainer},
  license={Apache-2.0}
}

License

This model is licensed under the Apache 2.0 License. See the LICENSE file for full details.

This model represents a significant advancement in specialized AI assistance for the DataFusion ecosystem, combining the power of large language models with domain-specific expertise in data processing and Rust programming.

Downloads last month: 8

GGUF

Model size

3B params

Architecture

qwen2

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for yarenty/qwen2.5-3B-datafusion-instruct-gguf

Base model

Qwen/Qwen2.5-3B

Finetuned

Qwen/Qwen2.5-3B-Instruct

Quantized

(220)

this model

yarenty
/

qwen2.5-3B-datafusion-instruct-gguf