# Multilingual batches

**URL:** <https://discuss.huggingface.co/t/multilingual-batches/130788>\
**Category:** 🤗Datasets\
**Created:** [December 12, 2024, 4:03pm UTC](https://discuss.huggingface.co/t/multilingual-batches/130788 "2024-12-12T16:03:36Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![lingvenvist](https://avatars.discourse-cdn.com/v4/letter/l/8edcca/32.png) [@lingvenvist](https://discuss.huggingface.co/u/lingvenvist)\
**Post date:** [December 12, 2024, 4:03pm UTC](https://discuss.huggingface.co/t/multilingual-batches/130788/1 "2024-12-12T16:03:37Z")

</div>

Hi everyone,

I’m trying to create multilingual batches in a controlled way. Specifically, I want each batch (e.g., with batch\_size=32) to contain items from 4 different languages, with 8 examples sampled randomly per language.

I’ve already created a custom list of batches meeting this requirement, but I’m struggling with how to pass these batches to the Trainer. Currently, my Trainer setup looks like this: trainer = SFTTrainer(  
model=model,  
train\_dataset=datasets[‘train’],  
eval\_dataset=datasets[‘eval’], # Add evaluation dataset  
peft\_config=peft\_config,  
tokenizer=tokenizer,  
max\_seq\_length=512,  
args=training\_arguments,  
formatting\_func=formatting\_prompts\_func  
)  
Does anyone know how to properly integrate my custom batches with the Trainer? Any guidance would be greatly appreciated!

Thank you in advance!

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [December 12, 2024, 6:35pm UTC](https://discuss.huggingface.co/t/multilingual-batches/130788/2 "2024-12-12T18:35:36Z")

</div>

Hi ! you can use `datasets.interleave_datasets()` to make a dataset with alternating languages (just give it as input your 4 datasets - one per language) and feed it to the Trainer 🙂

You might want to pre-shuffle each language dataset and disable the Trainer’s shuffling to make sure your end up with batches of 8 samples per language

---

<div class="post-metadata">

**Author:** ![lingvenvist](https://avatars.discourse-cdn.com/v4/letter/l/8edcca/32.png) [@lingvenvist](https://discuss.huggingface.co/u/lingvenvist)\
**Post date:** [December 12, 2024, 8:06pm UTC](https://discuss.huggingface.co/t/multilingual-batches/130788/3 "2024-12-12T20:06:03Z")

</div>

Thank you very much!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [December 13, 2024, 8:06am UTC](https://discuss.huggingface.co/t/multilingual-batches/130788/4 "2024-12-13T08:06:48Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.