Type of dataset in Trainer class

mikaelsouza
Sat, 18 Jul 2020 04:50:00 +0000

For more context, he/she is talking about this page: https://huggingface.co/transformers/training.html

I also got confused by this bit of the documentation, but I think this code expects datasets like the ones provided by Hugging Face’s NLP package.

I think they are all based on Pytorch’s Dataset Class, but I could be mistaken.

Try to use one of the datasets provided by their NLP package and check if it works correctly.

Hope this helps!