Awesome075/flan-t5-large-wcep10-extracted-sentences

wcep-flan-t5-large

This model is a fine-tuned version of google/flan-t5-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 11.5401

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAFACTOR and the args are: No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
153.7908	0.4391	50	17.2862
138.3394	0.8782	100	14.9564
127.3862	1.3161	150	13.1359
120.2336	1.7552	200	12.1544
115.0450	2.1932	250	11.8273
112.3383	2.6323	300	11.5401

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.8.3
• Tokenizers 0.22.2