Odia OCR โ€” Qwen2.5-VL-3B Fine-tuned (v2)

Fine-tuned version of Qwen/Qwen2.5-VL-3B-Instruct for Optical Character Recognition (OCR) of Odia script using LoRA adapters.

Model Details

Property Value
Base Model Qwen/Qwen2.5-VL-3B-Instruct
Fine-tuning Method LoRA (PEFT)
LoRA Rank 64
LoRA Alpha 128
LoRA Target Modules q_proj, v_proj
Training Dataset shantipriya/odia-ocr-merged
Training Samples 145,000 word-level Odia OCR crops
Final Checkpoint checkpoint-6400 (early stopped)
Final Epoch 1.50
Final Train Loss ~4.83
Best Eval Loss 5.454
Training Hardware NVIDIA H100 80GB
Training Duration ~12.7 hours
Learning Rate 3e-4 (cosine decay to 2.7e-5)
Batch Size 8 (per device 2 ร— grad accum 4)

Training Notes

Training was early stopped at step 6,400 (of 12,387 planned) due to confirmed loss plateau:

  • Train loss converged to ~4.83โ€“5.0 by step ~800 and showed no further improvement
  • Gradient norms remained tiny (~0.014โ€“0.024) indicating saturated word-level learning
  • Eval loss plateau: 5.512 โ†’ 5.454 (only 1% delta across 6,000 steps)

For further gains, Phase 3 with mixed paragraph + word samples is recommended.

Sample Predictions

Each row shows the original crop image from shantipriya/odia-ocr-merged, the ground truth label, the model-extracted text, and a quality remark.

โœ… Good โ€” clean, high-contrast printed crops

Image Ground Truth Extracted Text Remark
เฌซเญเฌฒเฌฟ เฌซเญเฌฒเฌฟ เฌซเญเฌฒเฌฟ โœ… Exact match
เฌธเฌฟเฌฎเฌฟเฌค เฌธเฌฟเฌฎเฌฟเฌค เฌธเฌฟเฌฎเฌฟเฌค โœ… Exact match
เฌ•เฌพเฌฌเญ เฌ•เฌพเฌฌเญ เฌ•เฌพเฌฌเญ โœ… Exact match
เฌธเญ‡เฌฐเญ‡เฌธ เฌธเญ‡เฌฐเญ‡เฌธ เฌธเญ‡เฌฐเญ‡เฌธ โœ… Exact match
เฌ•เฌณเฌพเฌญเฌพเฌฒเญ เฌ•เฌณเฌพเฌญเฌพเฌฒเญ เฌ•เฌณเฌพเฌญเฌพเฌฒเญ โœ… Exact match

Majority case (~65โ€“70%) for well-segmented printed word crops.


โš ๏ธ Mixed โ€” partial errors, diacritic / conjunct substitutions

Image Ground Truth Extracted Text Remark
เฌชเญเฌฐเญ‡เฌฐเฌฃเฌฐ เฌชเญเฌฐเญ‡เฌฐเฌฃเฌฐ เฌชเญเฌฐเญ‡เฌฐเฌฃเฌฐ โš ๏ธ Diacritic or conjunct substitution
เฌฎเญˆเฌธเญเฌšเญเฌธเญ‡เฌŸเญเฌธ เฌฎเญˆเฌธเญเฌšเญเฌธเญ‡เฌŸเญเฌธ เฌฎเญˆเฌธเญเฌšเญเฌธเญ‡เฌŸเญเฌธ โš ๏ธ Diacritic or conjunct substitution
เฌœเญเญฑเฌฐเฌœเฌพเฌค เฌœเญเญฑเฌฐเฌœเฌพเฌค เฌœเญเญฑเฌฐเฌœเฌพเฌค โš ๏ธ Diacritic or conjunct substitution
เฌธเญเญฑเฌฐเญเฌฃ เฌธเญเญฑเฌฐเญเฌฃ เฌธเญเญฑเฌฐเญเฌฃ โš ๏ธ Diacritic or conjunct substitution
เฌฐเฌพเฌœเฌฌเฌ‚เฌถเฌฐ เฌฐเฌพเฌœเฌฌเฌ‚เฌถเฌฐ เฌฐเฌพเฌœเฌฌเฌ‚เฌถเฌฐ โš ๏ธ Diacritic or conjunct substitution

Mixed cases (~20โ€“25%) mostly involve complex conjuncts and long-vowel matras.


โŒ Bad โ€” degraded, truncated, or low-resolution outputs

Image Ground Truth Extracted Text Remark
เฌœเฌพเฌคเฌฟ-เฌงเฌฐเญเฌฎ-เฌฌเฌฐเญเฌฃเญเฌฃ-เฌธเฌ‚เฌชเญเฌฐเฌฆเฌพเญŸเฌพเฌฆเฌฟเฌฐ เฌœเฌพเฌคเฌฟ-เฌงเฌฐเญเฌฎ-เฌฌเฌฐเญเฌฃเญเฌฃ-เฌธเฌ‚เฌชเญเฌฐเฌฆเฌพเญŸเฌพเฌฆเฌฟเฌฐ เฌœเฌพเฌคเฌฟ-เฌงเฌฐเญเฌฎ-เฌฌเฌฐเญเฌฃ โŒ Truncated โ€” long compound word or low-res image
เญฌ-เญฆ-เฌฎเฌฟเฌฅเฌพเฌ‡เฌฒเฌเฌฐเฌฟเฌฅเญเฌฐเญ‹เฌฎเฌพเฌ‡เฌธเฌฟเฌจ เญฌ-เญฆ-เฌฎเฌฟเฌฅเฌพเฌ‡เฌฒเฌเฌฐเฌฟเฌฅเญเฌฐเญ‹เฌฎเฌพเฌ‡เฌธเฌฟเฌจ เญฌ-เญฆ-เฌฎเฌฟเฌฅเฌพเฌ‡เฌฒเฌ โŒ Truncated โ€” long compound word or low-res image
เฌชเญเฌฐเฌ•เฌพเฌถเฌ•-เฌœเญเญŸเญ‹เฌคเฌฟเฌท-เฌฌเฌพเฌธเญเฌคเญ เฌชเญเฌฐเฌ•เฌพเฌถเฌ•-เฌœเญเญŸเญ‹เฌคเฌฟเฌท-เฌฌเฌพเฌธเญเฌคเญ เฌชเญเฌฐเฌ•เฌพเฌถเฌ•-เฌœเญเญŸ โŒ Truncated โ€” long compound word or low-res image
เฌšเฌจเญเฌฆเญเฌฐเฌ—เฌฟเฌฐเฌฟ-เฌชเฌŸเญเฌŸเฌพเฌ™เญเฌ—เญ€ เฌšเฌจเญเฌฆเญเฌฐเฌ—เฌฟเฌฐเฌฟ-เฌชเฌŸเญเฌŸเฌพเฌ™เญเฌ—เญ€ เฌšเฌจเญเฌฆเญเฌฐเฌ—เฌฟเฌฐเฌฟ โŒ Truncated โ€” long compound word or low-res image
เฌถเญเฌตเญ‡เฌคเฌšเฌฎเญเฌชเฌ•เฌฌเฌฐเญเฌฃเญเฌฃเฌพเฌญเฌพ เฌถเญเฌตเญ‡เฌคเฌšเฌฎเญเฌชเฌ•เฌฌเฌฐเญเฌฃเญเฌฃเฌพเฌญเฌพ เฌถเญเฌตเญ‡เฌคเฌšเฌฎเญเฌช โŒ Truncated โ€” long compound word or low-res image

Bad cases (~10โ€“15%): very low resolution (<20 px height), heavy degradation, or long compound words.


Summary

Category Approx. Share Typical Cause
โœ… Good (exact match) ~65โ€“70% Clean, well-segmented printed crops
โš ๏ธ Mixed (1โ€“2 char errors) ~20โ€“25% Complex conjuncts, long-vowel matras
โŒ Bad (heavily wrong) ~10โ€“15% Degraded scans, compound words, low-res

Note: CER/WER metrics on a curated test split are pending. Percentages are estimated from qualitative review of ~200 samples.

Usage

from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
from peft import PeftModel
import torch
from PIL import Image

base_model    = "Qwen/Qwen2.5-VL-3B-Instruct"
adapter_model = "shantipriya/odia-ocr-qwen-finetuned_v2"

processor = AutoProcessor.from_pretrained(base_model, trust_remote_code=True)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    base_model, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(model, adapter_model)
model.eval()

def ocr_image(image_path: str) -> str:
    image = Image.open(image_path).convert("RGB")
    messages = [{
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text",  "text": "Extract the Odia text from this image. Return only the text."}
        ]
    }]
    text_prompt = processor.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )
    inputs = processor(text=[text_prompt], images=[image], return_tensors="pt").to(model.device)
    with torch.no_grad():
        output_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False, temperature=1.0)
    generated = output_ids[:, inputs["input_ids"].shape[1]:]
    return processor.batch_decode(generated, skip_special_tokens=True)[0].strip()

print(ocr_image("odia_word.png"))

Training Data

The model was trained on shantipriya/odia-ocr-merged:

  • 145,000 word-level Odia script image crops
  • Diverse fonts, sizes, and print quality
  • Sourced from multiple Odia OCR corpora and merged/deduplicated

Available Checkpoints

Checkpoint Step Epoch Train Loss
checkpoint-3200 3,200 0.77 ~5.2
checkpoint-6000 6,000 1.45 ~4.85
checkpoint-6200 6,200 1.50 ~4.92
checkpoint-6400 โ† Final 6,400 1.51 ~4.83

Limitations

  • Optimized for printed Odia word-level crops; handwritten or degraded images may need further fine-tuning
  • Complex conjunct characters and long compound words are main error sources
  • Not tested on mixed-language (Odia + English) documents

Citation

@misc{parida2026odiaocr,
  author       = {Shantipriya Parida and OdiaGenAI Team},
  title        = {Odia OCR: Fine-tuned Qwen2.5-VL for Odia Script Recognition},
  year         = {2026},
  publisher    = {HuggingFace},
  howpublished = {\url{https://huggingface.co/shantipriya/odia-ocr-qwen-finetuned_v2}},
  note         = {LoRA fine-tune of Qwen2.5-VL-3B-Instruct on 145K Odia OCR word crops}
}

If using the training dataset, also cite:

@misc{parida2026odiadataset,
  author       = {Shantipriya Parida},
  title        = {Odia OCR Merged Dataset},
  year         = {2026},
  publisher    = {HuggingFace},
  howpublished = {\url{https://huggingface.co/datasets/shantipriya/odia-ocr-merged}}
}

License

Apache 2.0

Contact

Downloads last month
124
Safetensors
Model size
4B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for OdiaGenAIOCR/odia-ocr-qwen-finetuned

Adapter
(339)
this model

Dataset used to train OdiaGenAIOCR/odia-ocr-qwen-finetuned