MixLoRA
MixLoRA
MixLoRA is a type of PEFT method which construct a resource-efficient sparse MoE model based on LoRA.
MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
https://paperswithcode.com/paper/mixlora-enhancing-large-language-models-fine
MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts