Does anyone know the minimum hardware requirements to fine-tune this Flan-T5-Large model?
#16
by LeandroArg - opened
Or what hardware did you use to fine-tune it?
Are 2 NVIDIA A30 GPUs with 24GB each sufficient? 🤔
Hi @LeandroArg
if you use LoRA or QLoRA this should be more than sufficient. By fine-tuning only adapters you drastically reduce the number of trainable parameters of the model, making it possible to fine-tune large models on consumer-type hardware.
Please have a look at: https://huggingface.co/docs/transformers/peft or the examples here: https://github.com/huggingface/peft/tree/main/examples to understand how to use PEFT to fine-tune large models at low cost.