Child-Tuning
Child-Tuning
**Child-Tuning** is a fine-tuning technique that updates a subset of parameters (called child network) of large pretrained models via strategically masking out the gradients of the non-child network during the backward process. It decreases the hypothesis space of the model via a task-specific mask applied to the full gradients, helping to effectively adapt the large-scale pretrained model to various tasks and meanwhile aiming to maintain its original generalization ability.
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning
https://paperswithcode.com/paper/raise-a-child-in-large-language-model-towards
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning