# Continue from pretrained

**URL:** <https://discuss.huggingface.co/t/continue-from-pretrained/40472>\
**Category:** 🤗Transformers\
**Created:** [May 20, 2023, 3:14pm UTC](https://discuss.huggingface.co/t/continue-from-pretrained/40472 "2023-05-20T15:14:09Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![hamedhm](https://avatars.discourse-cdn.com/v4/letter/h/7ab992/32.png) [@hamedhm](https://discuss.huggingface.co/u/hamedhm)\
**Post date:** [May 20, 2023, 3:14pm UTC](https://discuss.huggingface.co/t/continue-from-pretrained/40472/1 "2023-05-20T15:14:09Z")

</div>

I am training a transformer model and since it is huge, I need to stop training after each epoch and continue. I am doing the following. Please can you check and confirm that this indeed continues training from the previous epoch:

```auto
from transformers import RobertaForMaskedLM
model = RobertaForMaskedLM.from_pretrained('path to the previous epoch pretrained')
model.train()
...

```

---

<div class="post-metadata">

**Author:** ![ianuragbhatt](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ianuragbhatt/32/16236_2.png) [@ianuragbhatt](https://discuss.huggingface.co/u/ianuragbhatt)\
**Post date:** [May 21, 2023, 10:44am UTC](https://discuss.huggingface.co/t/continue-from-pretrained/40472/2 "2023-05-21T10:44:20Z")

</div>

I think it’s right we load our saved model like that and then retraining from there.