muril-en-hi-codemixed

muril-en-hi-codemixed is a masked language model, based on the MuRIL multilingual model.

muril-en-hi-codemixed replaces the tokenizer, vocabulary and the embeddings layer of the MuRIL model. The tokenizer and vocabulary used are the same as in the roberta-en-hi-codemixed model. The new embedding weights were initialized from the MuRIL embeddings.

The new muril-en-hi-codemixed model was further pre-trained for two epochs on the same codemixed English and Hindi corpora as the roberta-en-hi-codemixed model.