---
datasets:
- xlangai/spider
base_model:
- Qwen/Qwen3-4B-Instruct-2507
tags:
- unsloth
- text-to-sql
- fine-tuning
- trl
---
# Qwen3-4B NL → SQL (Spider Fine-Tuned)

## Model Overview

- **Model name:** qwen3-4b-nl2sql  
- **Base model:** Qwen/Qwen3-4B-Instruct-2507  
- **Model type:** Decoder-only Transformer (Causal Language Model)  
- **Task:** Natural Language → SQL Query Generation  
- **Language:** English  
- **License:** Apache-2.0  
- **Author:** Neeharika

> ✅ This repository contains **merged model weights** and can be used directly with  
> `AutoModelForCausalLM.from_pretrained`.

This model is a fine-tuned version of Qwen3-4B, optimized to translate natural language questions into executable SQL queries using database schema context. It is designed for structured data querying and analytics use cases rather than open-ended conversation.

---

## Intended Use

### Primary Use Cases
- Natural language interfaces for SQL databases
- Backend services that auto-generate SQL
- Data analytics assistants
- Research in semantic parsing and text-to-SQL

### Out-of-Scope Uses
- General-purpose chatbots
- Code generation beyond SQL
- Autonomous decision-making in high-risk domains (medical, legal, financial)

---

## Training Data

### Dataset
- **Name:** Spider Dataset  
- **Type:** Public benchmark for cross-domain text-to-SQL tasks  
- **Domains:** Multiple real-world databases with diverse schemas  
- **Size:** ~7k–10k question–SQL pairs (after preprocessing)

### Data Characteristics
- Multi-table joins  
- Nested and correlated subqueries  
- Aggregations (COUNT, AVG, SUM, GROUP BY, HAVING)  
- Schema-dependent reasoning

### Preprocessing
- Converted samples into instruction-style format  
- Injected full database schema into prompts  
- Cleaned malformed or ambiguous samples  
- Normalized SQL formatting

---

## Prompt Format

The model was trained using schema-aware instruction prompts:
```
### Instruction:
Convert the question into an SQL query.

### Database Schema:
<Table and column definitions>

### Question:
<User query>

### SQL:
```

## Training Procedure

### Frameworks & Libraries
- Hugging Face Transformers  
- Hugging Face TRL (Supervised Fine-Tuning)  
- Unsloth (memory-efficient training)  
- PyTorch  

### Fine-Tuning Details
- **Method:** Supervised Fine-Tuning (SFT)  
- **Quantization during training:** 4-bit (bnb)  
- **Precision:** Mixed precision  
- **Optimization:** LoRA adapters (merged post-training)

### Key Hyperparameters
- Max sequence length: 2048  
- Learning rate: 2e-4  
- Optimizer: AdamW  
- Padding: Right-padding  
- Gradient checkpointing: Enabled  

---

## Example Usage

```python
from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Neeharika20/qwen3-4b-nl2sql",
    max_seq_length=2048,
    load_in_4bit=True,
    device_map="auto",
)

model.eval()

prompt = """### Instruction:
Convert the question into an SQL query.

### Database Schema:
CREATE TABLE students(
    id INT,
    name TEXT,
    age INT
);

### Question:
List the names of all students.

### SQL:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=64,
        temperature=0.2,
        do_sample=False,
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```