# Why does padding = 'max\_length' cause much slower model inference?

**URL:** <https://discuss.huggingface.co/t/why-does-padding-max-length-cause-much-slower-model-inference/14184>\
**Category:** Models\
**Created:** [January 28, 2022, 12:05am UTC](https://discuss.huggingface.co/t/why-does-padding-max-length-cause-much-slower-model-inference/14184 "2022-01-28T00:05:58Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jackkwok](https://avatars.discourse-cdn.com/v4/letter/j/ecae2f/32.png) [@jackkwok](https://discuss.huggingface.co/u/jackkwok)\
**Post date:** [January 28, 2022, 12:05am UTC](https://discuss.huggingface.co/t/why-does-padding-max-length-cause-much-slower-model-inference/14184/1 "2022-01-28T00:05:58Z")

</div>

I trained a bert-based-uncase AutoModelForSequenceClassification model and found that model inference is at least 2x faster if I comment out padding = ‘max\_length’ in the encode step. My understanding is BERT expects a fix length of 512 tokens… doesn’t that imply input must be padded to 512? Please help me understand.

```auto
sequence = tokenizer.encode_plus(question,
                                        passage,
                                        max_length = 256,
                                        padding = 'max_length',
                                        truncation = \"longest_first\",
                                        return_tensors=\"pt\")['input_ids'].to(device)

```

---

<div class="post-metadata">

**Author:** ![djstein](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/djstein/32/16692_2.png) [@djstein](https://discuss.huggingface.co/u/djstein)\
**Post date:** [June 8, 2023, 6:33pm UTC](https://discuss.huggingface.co/t/why-does-padding-max-length-cause-much-slower-model-inference/14184/2 "2023-06-08T18:33:55Z")

</div>

In the same boat. We switched added max\_length=512 and our training time went from 6 minutes to 1 hour on a 4090