wangchanberta-hyperopt-sentiment-01

This model is a fine-tuned version of airesearch/wangchanberta-base-att-spm-uncased on the Wisesight Sentiment dataset. The model is optimized for binary sentiment classification tasks, targeting two labels: positive and negative.

It achieves the following results on the evaluation set:

• Loss: 0.3595
• Accuracy: 0.9103

Model description

This model is intended for Thai language sentiment analysis, specifically designed to classify text as either positive or negative.

Intended uses & limitations

• The model is only trained to recognize positive and negative sentiments and may not perform well on nuanced or multi-class sentiment tasks.
• The model is specialized for the Thai language and is not intended for multi-language or code-switching scenarios.

Training and evaluation data

The model is trained on the Wisesight Sentiment dataset, which is a widely-used dataset for Thai NLP tasks.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2.5692051845867925e-05
• train_batch_size: 16
• eval_batch_size: 32
• seed: 7
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.55	250	0.3128	0.8859
0.3913	1.09	500	0.2672	0.8942
0.3913	1.64	750	0.2860	0.9025
0.2172	2.19	1000	0.4044	0.9060
0.2172	2.74	1250	0.3738	0.9076

Framework versions

• Transformers 4.33.2
• Pytorch 2.0.1
• Datasets 2.14.5
• Tokenizers 0.13.3