Cuda memory error on unchanged workshop 1 notebooks

kjackson
Sat, 27 Nov 2021 02:44:59 +0000

I am running notebooks 1 and 3 unchanged from https://github.com/philschmid/huggingface-sagemaker-workshop-series/tree/main/workshop_1_getting_started_with_amazon_sagemaker

And I get the following error:

RuntimeError: CUDA out of memory. Tried to allocate 192.00 MiB (GPU 0; 15.78 GiB total capacity; 14.80 GiB already allocated; 44.75 MiB free; 14.83 GiB reserved in total by PyTorch)

I am trying with different batch sizes and learning rates, but can someone help me understand why not everyone got the same error if we’re all using the same AWS resources?

Read full topic