Hi, at training, I’m using the forward pass and batch_decode on the logits to get the decoded output: outputs = model( input_ids, attention_mask, dec_input_ids, dec_attention_mask, labels=dec_input_ids, ) loss, logits = outputs.loss, outputs.logi…

T5 forward pass versus generate, latter outputs non-sense

deathcrush March 25, 2021, 1:06pm 9

I happened to stumble across this post, which has a similar flavour to your error. Have a look!

Topic		Replies	Views
T5 Model Generate and Model Outputs Vastly Different Beginners	2	857	August 19, 2025
T5 weird behavior between model.forward() and model.generate 🤗Transformers	0	132	March 31, 2024
T5 - model.generate() issue Beginners	2	715	March 18, 2024
The output of T5 is not consistent on multiple sequences 🤗Transformers	1	884	May 11, 2022
T5 models have non-deterministic outputs even after disabling dropout 🤗Transformers	9	303	September 15, 2024