metadata
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- medical
- ecg
- question-answering
- multimodal
- pytorch
metrics:
- exact_match
- bertscore
- bleu
- rouge
base_model:
- meta-llama/Llama-3.2-1B-Instruct
Q-HEART: ECG Question Answering via Knowledge-Informed Multimodal LLMs (ECAI 2025)
Usage
After we have access to meta-llama/Llama-3.2-1B-Instruct model and install suitable transformers package version, we can run:
# transformers==4.43.3 accelerate==1.0.1 peft==0.13.2
from transformers import AutoModel
model = AutoModel.from_pretrained("Manhph2211/Q-HEART", trust_remote_code=True, dtype="auto")
Or
git clone https://github.com/manhph2211/Q-HEART.git && cd Q-HEART
conda create -n qheart python=3.9
conda activate qheart
pip install torch --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt
Download the checkpoint from here and place it at ckpts/pytorch_model.bin, then run evaluation:
python main.py --model_type meta-llama/Llama-3.2-1B-Instruct --mapping_type Transformer
Citation
@article{pham2025q,
title={Q-Heart: ECG Question Answering via Knowledge-Informed Multimodal LLMs},
author={Pham, Hung Manh and Tang, Jialu and Saeed, Aaqib and Ma, Dong},
journal={arXiv preprint arXiv:2505.06296},
year={2025}
}
@inproceedings{pham2025qheart,
title = {Q-HEART: ECG Question Answering via Knowledge-Informed Multimodal LLMs},
author = {Pham, Hung Manh and Tang, Jialu and Saeed, Aaqib and Ma, Dong},
booktitle = {Proceedings of the European Conference on Artificial Intelligence (ECAI)},
series = {Frontiers in Artificial Intelligence and Applications},
volume = {413},
pages = {4545--4552},
year = {2025},
publisher = {IOS Press},
doi = {10.3233/FAIA251356}
}
Please refer to our GitHub repo for more details!