Instructions to use InfiniAILab/OpenR1-Qwen-7B-Sparse-P50 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use InfiniAILab/OpenR1-Qwen-7B-Sparse-P50 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="InfiniAILab/OpenR1-Qwen-7B-Sparse-P50")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("InfiniAILab/OpenR1-Qwen-7B-Sparse-P50")
model = AutoModelForCausalLM.from_pretrained("InfiniAILab/OpenR1-Qwen-7B-Sparse-P50", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use InfiniAILab/OpenR1-Qwen-7B-Sparse-P50 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "InfiniAILab/OpenR1-Qwen-7B-Sparse-P50"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InfiniAILab/OpenR1-Qwen-7B-Sparse-P50",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/InfiniAILab/OpenR1-Qwen-7B-Sparse-P50

SGLang

How to use InfiniAILab/OpenR1-Qwen-7B-Sparse-P50 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "InfiniAILab/OpenR1-Qwen-7B-Sparse-P50" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InfiniAILab/OpenR1-Qwen-7B-Sparse-P50",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "InfiniAILab/OpenR1-Qwen-7B-Sparse-P50" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InfiniAILab/OpenR1-Qwen-7B-Sparse-P50",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use InfiniAILab/OpenR1-Qwen-7B-Sparse-P50 with Docker Model Runner:
```
docker model run hf.co/InfiniAILab/OpenR1-Qwen-7B-Sparse-P50
```

ZMC2019 commited on Mar 31, 2025

Commit

6fdb594

verified ·

1 Parent(s): c393ae9

Model save

Browse files

Files changed (5) hide show

README.md +58 -0
all_results.json +8 -0
generation_config.json +9 -0
train_results.json +8 -0
trainer_state.json +1540 -0

README.md ADDED Viewed

	@@ -0,0 +1,58 @@

+---
+base_model: open-r1/OpenR1-Qwen-7B
+library_name: transformers
+model_name: OpenR1-Qwen-7B-Sparse
+tags:
+- generated_from_trainer
+- trl
+- sft
+licence: license
+---
+# Model Card for OpenR1-Qwen-7B-Sparse
+This model is a fine-tuned version of [open-r1/OpenR1-Qwen-7B](https://huggingface.co/open-r1/OpenR1-Qwen-7B).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+## Quick start
+```python
+from transformers import pipeline
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="ZMC2019/OpenR1-Qwen-7B-Sparse", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+## Training procedure
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/chenzhuoming911/huggingface/runs/mharh69t)
+This model was trained with SFT.
+### Framework versions
+- TRL: 0.16.0.dev0
+- Transformers: 4.49.0
+- Pytorch: 2.5.1
+- Datasets: 3.5.0
+- Tokenizers: 0.21.1
+## Citations
+Cite TRL as:
+```bibtex
+@misc{vonwerra2022trl,
+	title        = {{TRL: Transformer Reinforcement Learning}},
+	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
+	year         = 2020,
+	journal      = {GitHub repository},
+	publisher    = {GitHub},
+	howpublished = {\url{https://github.com/huggingface/trl}}
+}
+```

all_results.json ADDED Viewed

	@@ -0,0 +1,8 @@

+{
+    "total_flos": 9.186429923093381e+17,
+    "train_loss": 0.27417567097351914,
+    "train_runtime": 29088.9913,
+    "train_samples": 93733,
+    "train_samples_per_second": 0.59,
+    "train_steps_per_second": 0.037
+}

generation_config.json ADDED Viewed

	@@ -0,0 +1,9 @@

+{
+  "bos_token_id": 151643,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "transformers_version": "4.49.0"
+}

train_results.json ADDED Viewed

	@@ -0,0 +1,8 @@

+{
+    "total_flos": 9.186429923093381e+17,
+    "train_loss": 0.27417567097351914,
+    "train_runtime": 29088.9913,
+    "train_samples": 93733,
+    "train_samples_per_second": 0.59,
+    "train_steps_per_second": 0.037
+}

trainer_state.json ADDED Viewed

	@@ -0,0 +1,1540 @@

+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 1073,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.004659832246039142,
+      "grad_norm": 0.2480848187526065,
+      "learning_rate": 9.259259259259259e-07,
+      "loss": 0.2856,
+      "step": 5
+    },
+    {
+      "epoch": 0.009319664492078284,
+      "grad_norm": 0.27846440569740516,
+      "learning_rate": 1.8518518518518519e-06,
+      "loss": 0.2743,
+      "step": 10
+    },
+    {
+      "epoch": 0.013979496738117428,
+      "grad_norm": 0.1775956763008417,
+      "learning_rate": 2.7777777777777783e-06,
+      "loss": 0.2735,
+      "step": 15
+    },
+    {
+      "epoch": 0.01863932898415657,
+      "grad_norm": 0.2097019213694641,
+      "learning_rate": 3.7037037037037037e-06,
+      "loss": 0.2783,
+      "step": 20
+    },
+    {
+      "epoch": 0.023299161230195712,
+      "grad_norm": 0.2017973119570326,
+      "learning_rate": 4.62962962962963e-06,
+      "loss": 0.2623,
+      "step": 25
+    },
+    {
+      "epoch": 0.027958993476234855,
+      "grad_norm": 0.19396546589386038,
+      "learning_rate": 5.555555555555557e-06,
+      "loss": 0.2588,
+      "step": 30
+    },
+    {
+      "epoch": 0.032618825722273995,
+      "grad_norm": 0.1833475348357745,
+      "learning_rate": 6.481481481481482e-06,
+      "loss": 0.2629,
+      "step": 35
+    },
+    {
+      "epoch": 0.03727865796831314,
+      "grad_norm": 0.19103036464826775,
+      "learning_rate": 7.4074074074074075e-06,
+      "loss": 0.2639,
+      "step": 40
+    },
+    {
+      "epoch": 0.04193849021435228,
+      "grad_norm": 0.1903046005080379,
+      "learning_rate": 8.333333333333334e-06,
+      "loss": 0.2673,
+      "step": 45
+    },
+    {
+      "epoch": 0.046598322460391424,
+      "grad_norm": 0.18460973869272648,
+      "learning_rate": 9.25925925925926e-06,
+      "loss": 0.2761,
+      "step": 50
+    },
+    {
+      "epoch": 0.05125815470643057,
+      "grad_norm": 0.19245235339515152,
+      "learning_rate": 1.0185185185185186e-05,
+      "loss": 0.2712,
+      "step": 55
+    },
+    {
+      "epoch": 0.05591798695246971,
+      "grad_norm": 0.187838588087584,
+      "learning_rate": 1.1111111111111113e-05,
+      "loss": 0.2723,
+      "step": 60
+    },
+    {
+      "epoch": 0.06057781919850885,
+      "grad_norm": 0.17781539091622867,
+      "learning_rate": 1.2037037037037039e-05,
+      "loss": 0.2667,
+      "step": 65
+    },
+    {
+      "epoch": 0.06523765144454799,
+      "grad_norm": 0.20027656085731357,
+      "learning_rate": 1.2962962962962964e-05,
+      "loss": 0.2736,
+      "step": 70
+    },
+    {
+      "epoch": 0.06989748369058714,
+      "grad_norm": 0.20349467354700868,
+      "learning_rate": 1.388888888888889e-05,
+      "loss": 0.2696,
+      "step": 75
+    },
+    {
+      "epoch": 0.07455731593662628,
+      "grad_norm": 0.20858032960565945,
+      "learning_rate": 1.4814814814814815e-05,
+      "loss": 0.2659,
+      "step": 80
+    },
+    {
+      "epoch": 0.07921714818266543,
+      "grad_norm": 0.21178392393672066,
+      "learning_rate": 1.5740740740740744e-05,
+      "loss": 0.2715,
+      "step": 85
+    },
+    {
+      "epoch": 0.08387698042870456,
+      "grad_norm": 0.19683247262126224,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 0.276,
+      "step": 90
+    },
+    {
+      "epoch": 0.08853681267474371,
+      "grad_norm": 0.19763192488794942,
+      "learning_rate": 1.7592592592592595e-05,
+      "loss": 0.2704,
+      "step": 95
+    },
+    {
+      "epoch": 0.09319664492078285,
+      "grad_norm": 0.1956972064059814,
+      "learning_rate": 1.851851851851852e-05,
+      "loss": 0.2682,
+      "step": 100
+    },
+    {
+      "epoch": 0.097856477166822,
+      "grad_norm": 0.20066192797959884,
+      "learning_rate": 1.9444444444444445e-05,
+      "loss": 0.2694,
+      "step": 105
+    },
+    {
+      "epoch": 0.10251630941286113,
+      "grad_norm": 0.22173843606276825,
+      "learning_rate": 1.995854922279793e-05,
+      "loss": 0.2691,
+      "step": 110
+    },
+    {
+      "epoch": 0.10717614165890028,
+      "grad_norm": 0.22661606789228167,
+      "learning_rate": 1.985492227979275e-05,
+      "loss": 0.2707,
+      "step": 115
+    },
+    {
+      "epoch": 0.11183597390493942,
+      "grad_norm": 0.25878304515306677,
+      "learning_rate": 1.9751295336787565e-05,
+      "loss": 0.2712,
+      "step": 120
+    },
+    {
+      "epoch": 0.11649580615097857,
+      "grad_norm": 0.20769568193046992,
+      "learning_rate": 1.9647668393782386e-05,
+      "loss": 0.2662,
+      "step": 125
+    },
+    {
+      "epoch": 0.1211556383970177,
+      "grad_norm": 0.2339523798236711,
+      "learning_rate": 1.9544041450777206e-05,
+      "loss": 0.2741,
+      "step": 130
+    },
+    {
+      "epoch": 0.12581547064305684,
+      "grad_norm": 0.21510297906435297,
+      "learning_rate": 1.9440414507772023e-05,
+      "loss": 0.2684,
+      "step": 135
+    },
+    {
+      "epoch": 0.13047530288909598,
+      "grad_norm": 0.23732427940490297,
+      "learning_rate": 1.9336787564766843e-05,
+      "loss": 0.2685,
+      "step": 140
+    },
+    {
+      "epoch": 0.13513513513513514,
+      "grad_norm": 0.2123608724012592,
+      "learning_rate": 1.923316062176166e-05,
+      "loss": 0.2808,
+      "step": 145
+    },
+    {
+      "epoch": 0.13979496738117428,
+      "grad_norm": 0.22764321452420136,
+      "learning_rate": 1.9129533678756477e-05,
+      "loss": 0.2726,
+      "step": 150
+    },
+    {
+      "epoch": 0.14445479962721341,
+      "grad_norm": 0.20310601163154524,
+      "learning_rate": 1.9025906735751297e-05,
+      "loss": 0.2852,
+      "step": 155
+    },
+    {
+      "epoch": 0.14911463187325255,
+      "grad_norm": 0.2298242877800892,
+      "learning_rate": 1.8922279792746117e-05,
+      "loss": 0.2757,
+      "step": 160
+    },
+    {
+      "epoch": 0.15377446411929171,
+      "grad_norm": 0.20978701600081326,
+      "learning_rate": 1.8818652849740934e-05,
+      "loss": 0.2733,
+      "step": 165
+    },
+    {
+      "epoch": 0.15843429636533085,
+      "grad_norm": 0.1998319044566166,
+      "learning_rate": 1.8715025906735754e-05,
+      "loss": 0.2559,
+      "step": 170
+    },
+    {
+      "epoch": 0.16309412861137,
+      "grad_norm": 0.20450079549144815,
+      "learning_rate": 1.861139896373057e-05,
+      "loss": 0.2743,
+      "step": 175
+    },
+    {
+      "epoch": 0.16775396085740912,
+      "grad_norm": 0.19585540202223178,
+      "learning_rate": 1.850777202072539e-05,
+      "loss": 0.2731,
+      "step": 180
+    },
+    {
+      "epoch": 0.1724137931034483,
+      "grad_norm": 0.20114655528014272,
+      "learning_rate": 1.8404145077720208e-05,
+      "loss": 0.2676,
+      "step": 185
+    },
+    {
+      "epoch": 0.17707362534948742,
+      "grad_norm": 0.20898866856201928,
+      "learning_rate": 1.8300518134715028e-05,
+      "loss": 0.2746,
+      "step": 190
+    },
+    {
+      "epoch": 0.18173345759552656,
+      "grad_norm": 0.23418307676215824,
+      "learning_rate": 1.8196891191709845e-05,
+      "loss": 0.2738,
+      "step": 195
+    },
+    {
+      "epoch": 0.1863932898415657,
+      "grad_norm": 0.21381181093698814,
+      "learning_rate": 1.8093264248704665e-05,
+      "loss": 0.2755,
+      "step": 200
+    },
+    {
+      "epoch": 0.19105312208760486,
+      "grad_norm": 0.1945831655656839,
+      "learning_rate": 1.7989637305699482e-05,
+      "loss": 0.2729,
+      "step": 205
+    },
+    {
+      "epoch": 0.195712954333644,
+      "grad_norm": 0.25188414879097026,
+      "learning_rate": 1.7886010362694302e-05,
+      "loss": 0.2747,
+      "step": 210
+    },
+    {
+      "epoch": 0.20037278657968313,
+      "grad_norm": 0.2034438567116008,
+      "learning_rate": 1.778238341968912e-05,
+      "loss": 0.2707,
+      "step": 215
+    },
+    {
+      "epoch": 0.20503261882572227,
+      "grad_norm": 0.19342833535436457,
+      "learning_rate": 1.767875647668394e-05,
+      "loss": 0.2761,
+      "step": 220
+    },
+    {
+      "epoch": 0.2096924510717614,
+      "grad_norm": 0.2043835725939126,
+      "learning_rate": 1.757512953367876e-05,
+      "loss": 0.2681,
+      "step": 225
+    },
+    {
+      "epoch": 0.21435228331780057,
+      "grad_norm": 0.2074491682435995,
+      "learning_rate": 1.7471502590673576e-05,
+      "loss": 0.2743,
+      "step": 230
+    },
+    {
+      "epoch": 0.2190121155638397,
+      "grad_norm": 0.19431836721424317,
+      "learning_rate": 1.7367875647668397e-05,
+      "loss": 0.2696,
+      "step": 235
+    },
+    {
+      "epoch": 0.22367194780987884,
+      "grad_norm": 0.23535008907014365,
+      "learning_rate": 1.7264248704663214e-05,
+      "loss": 0.2756,
+      "step": 240
+    },
+    {
+      "epoch": 0.22833178005591798,
+      "grad_norm": 0.22992274931044096,
+      "learning_rate": 1.716062176165803e-05,
+      "loss": 0.2747,
+      "step": 245
+    },
+    {
+      "epoch": 0.23299161230195714,
+      "grad_norm": 0.23642332032502936,
+      "learning_rate": 1.705699481865285e-05,
+      "loss": 0.2726,
+      "step": 250
+    },
+    {
+      "epoch": 0.23765144454799628,
+      "grad_norm": 0.20748084991662222,
+      "learning_rate": 1.695336787564767e-05,
+      "loss": 0.2716,
+      "step": 255
+    },
+    {
+      "epoch": 0.2423112767940354,
+      "grad_norm": 0.21945114618793227,
+      "learning_rate": 1.6849740932642488e-05,
+      "loss": 0.2771,
+      "step": 260
+    },
+    {
+      "epoch": 0.24697110904007455,
+      "grad_norm": 0.22645209579957054,
+      "learning_rate": 1.6746113989637308e-05,
+      "loss": 0.2765,
+      "step": 265
+    },
+    {
+      "epoch": 0.2516309412861137,
+      "grad_norm": 0.19551387120009264,
+      "learning_rate": 1.6642487046632125e-05,
+      "loss": 0.2723,
+      "step": 270
+    },
+    {
+      "epoch": 0.25629077353215285,
+      "grad_norm": 0.20635102879128378,
+      "learning_rate": 1.6538860103626945e-05,
+      "loss": 0.2725,
+      "step": 275
+    },
+    {
+      "epoch": 0.26095060577819196,
+      "grad_norm": 0.19198952075337267,
+      "learning_rate": 1.6435233160621765e-05,
+      "loss": 0.2709,
+      "step": 280
+    },
+    {
+      "epoch": 0.2656104380242311,
+      "grad_norm": 0.2183101530827561,
+      "learning_rate": 1.6331606217616582e-05,
+      "loss": 0.2718,
+      "step": 285
+    },
+    {
+      "epoch": 0.2702702702702703,
+      "grad_norm": 0.20425797133671955,
+      "learning_rate": 1.6227979274611402e-05,
+      "loss": 0.2788,
+      "step": 290
+    },
+    {
+      "epoch": 0.2749301025163094,
+      "grad_norm": 0.23011000314623464,
+      "learning_rate": 1.612435233160622e-05,
+      "loss": 0.2905,
+      "step": 295
+    },
+    {
+      "epoch": 0.27958993476234856,
+      "grad_norm": 0.2155480458012789,
+      "learning_rate": 1.6020725388601036e-05,
+      "loss": 0.2848,
+      "step": 300
+    },
+    {
+      "epoch": 0.2842497670083877,
+      "grad_norm": 0.2197010179321984,
+      "learning_rate": 1.5917098445595856e-05,
+      "loss": 0.2747,
+      "step": 305
+    },
+    {
+      "epoch": 0.28890959925442683,
+      "grad_norm": 0.22179807332039045,
+      "learning_rate": 1.5813471502590673e-05,
+      "loss": 0.2728,
+      "step": 310
+    },
+    {
+      "epoch": 0.293569431500466,
+      "grad_norm": 0.2120751835986257,
+      "learning_rate": 1.5709844559585493e-05,
+      "loss": 0.2787,
+      "step": 315
+    },
+    {
+      "epoch": 0.2982292637465051,
+      "grad_norm": 0.226069231034194,
+      "learning_rate": 1.5606217616580313e-05,
+      "loss": 0.2651,
+      "step": 320
+    },
+    {
+      "epoch": 0.30288909599254427,
+      "grad_norm": 0.20857741942191318,
+      "learning_rate": 1.550259067357513e-05,
+      "loss": 0.2672,
+      "step": 325
+    },
+    {
+      "epoch": 0.30754892823858343,
+      "grad_norm": 0.2027127864635231,
+      "learning_rate": 1.539896373056995e-05,
+      "loss": 0.2758,
+      "step": 330
+    },
+    {
+      "epoch": 0.31220876048462254,
+      "grad_norm": 0.21391079816437977,
+      "learning_rate": 1.5295336787564767e-05,
+      "loss": 0.2737,
+      "step": 335
+    },
+    {
+      "epoch": 0.3168685927306617,
+      "grad_norm": 0.1949931649207275,
+      "learning_rate": 1.5191709844559586e-05,
+      "loss": 0.2709,
+      "step": 340
+    },
+    {
+      "epoch": 0.32152842497670087,
+      "grad_norm": 0.203272659243948,
+      "learning_rate": 1.5088082901554406e-05,
+      "loss": 0.2744,
+      "step": 345
+    },
+    {
+      "epoch": 0.32618825722274,
+      "grad_norm": 0.23386254140039578,
+      "learning_rate": 1.4984455958549225e-05,
+      "loss": 0.2829,
+      "step": 350
+    },
+    {
+      "epoch": 0.33084808946877914,
+      "grad_norm": 0.20956086579279376,
+      "learning_rate": 1.4880829015544043e-05,
+      "loss": 0.282,
+      "step": 355
+    },
+    {
+      "epoch": 0.33550792171481825,
+      "grad_norm": 0.19915707487576464,
+      "learning_rate": 1.4777202072538862e-05,
+      "loss": 0.2728,
+      "step": 360
+    },
+    {
+      "epoch": 0.3401677539608574,
+      "grad_norm": 0.19888362106152468,
+      "learning_rate": 1.4673575129533678e-05,
+      "loss": 0.2711,
+      "step": 365
+    },
+    {
+      "epoch": 0.3448275862068966,
+      "grad_norm": 0.20565614821988315,
+      "learning_rate": 1.4569948186528497e-05,
+      "loss": 0.2771,
+      "step": 370
+    },
+    {
+      "epoch": 0.3494874184529357,
+      "grad_norm": 0.22007541774769399,
+      "learning_rate": 1.4466321243523317e-05,
+      "loss": 0.2727,
+      "step": 375
+    },
+    {
+      "epoch": 0.35414725069897485,
+      "grad_norm": 0.21285035462598179,
+      "learning_rate": 1.4362694300518136e-05,
+      "loss": 0.2813,
+      "step": 380
+    },
+    {
+      "epoch": 0.35880708294501396,
+      "grad_norm": 0.2074859169658842,
+      "learning_rate": 1.4259067357512954e-05,
+      "loss": 0.2755,
+      "step": 385
+    },
+    {
+      "epoch": 0.3634669151910531,
+      "grad_norm": 0.20518630624764556,
+      "learning_rate": 1.4155440414507773e-05,
+      "loss": 0.2751,
+      "step": 390
+    },
+    {
+      "epoch": 0.3681267474370923,
+      "grad_norm": 0.24631525396138984,
+      "learning_rate": 1.4051813471502591e-05,
+      "loss": 0.2758,
+      "step": 395
+    },
+    {
+      "epoch": 0.3727865796831314,
+      "grad_norm": 0.2072164144473051,
+      "learning_rate": 1.394818652849741e-05,
+      "loss": 0.2679,
+      "step": 400
+    },
+    {
+      "epoch": 0.37744641192917056,
+      "grad_norm": 0.20130515204148264,
+      "learning_rate": 1.384455958549223e-05,
+      "loss": 0.288,
+      "step": 405
+    },
+    {
+      "epoch": 0.3821062441752097,
+      "grad_norm": 0.21318795834604967,
+      "learning_rate": 1.3740932642487049e-05,
+      "loss": 0.2774,
+      "step": 410
+    },
+    {
+      "epoch": 0.38676607642124883,
+      "grad_norm": 0.19837831244344498,
+      "learning_rate": 1.3637305699481867e-05,
+      "loss": 0.2732,
+      "step": 415
+    },
+    {
+      "epoch": 0.391425908667288,
+      "grad_norm": 0.21706421514687568,
+      "learning_rate": 1.3533678756476684e-05,
+      "loss": 0.2805,
+      "step": 420
+    },
+    {
+      "epoch": 0.3960857409133271,
+      "grad_norm": 0.21316576729012615,
+      "learning_rate": 1.3430051813471503e-05,
+      "loss": 0.2844,
+      "step": 425
+    },
+    {
+      "epoch": 0.40074557315936626,
+      "grad_norm": 0.2014337720595473,
+      "learning_rate": 1.3326424870466321e-05,
+      "loss": 0.2828,
+      "step": 430
+    },
+    {
+      "epoch": 0.40540540540540543,
+      "grad_norm": 0.2072514307470887,
+      "learning_rate": 1.3222797927461141e-05,
+      "loss": 0.2852,
+      "step": 435
+    },
+    {
+      "epoch": 0.41006523765144454,
+      "grad_norm": 0.24227887513487076,
+      "learning_rate": 1.311917098445596e-05,
+      "loss": 0.2755,
+      "step": 440
+    },
+    {
+      "epoch": 0.4147250698974837,
+      "grad_norm": 0.20801065806848365,
+      "learning_rate": 1.3015544041450778e-05,
+      "loss": 0.2764,
+      "step": 445
+    },
+    {
+      "epoch": 0.4193849021435228,
+      "grad_norm": 0.1942439454612911,
+      "learning_rate": 1.2911917098445597e-05,
+      "loss": 0.2691,
+      "step": 450
+    },
+    {
+      "epoch": 0.424044734389562,
+      "grad_norm": 0.2076864298032534,
+      "learning_rate": 1.2808290155440415e-05,
+      "loss": 0.2771,
+      "step": 455
+    },
+    {
+      "epoch": 0.42870456663560114,
+      "grad_norm": 0.20740530807254287,
+      "learning_rate": 1.2704663212435234e-05,
+      "loss": 0.2749,
+      "step": 460
+    },
+    {
+      "epoch": 0.43336439888164024,
+      "grad_norm": 0.2226181167276175,
+      "learning_rate": 1.2601036269430054e-05,
+      "loss": 0.2726,
+      "step": 465
+    },
+    {
+      "epoch": 0.4380242311276794,
+      "grad_norm": 0.2163332314393917,
+      "learning_rate": 1.2497409326424873e-05,
+      "loss": 0.2801,
+      "step": 470
+    },
+    {
+      "epoch": 0.4426840633737186,
+      "grad_norm": 0.22697445952775339,
+      "learning_rate": 1.239378238341969e-05,
+      "loss": 0.2721,
+      "step": 475
+    },
+    {
+      "epoch": 0.4473438956197577,
+      "grad_norm": 0.21459489713651447,
+      "learning_rate": 1.2290155440414508e-05,
+      "loss": 0.2876,
+      "step": 480
+    },
+    {
+      "epoch": 0.45200372786579684,
+      "grad_norm": 0.212728336485677,
+      "learning_rate": 1.2186528497409327e-05,
+      "loss": 0.2775,
+      "step": 485
+    },
+    {
+      "epoch": 0.45666356011183595,
+      "grad_norm": 0.18956662975107208,
+      "learning_rate": 1.2082901554404145e-05,
+      "loss": 0.2717,
+      "step": 490
+    },
+    {
+      "epoch": 0.4613233923578751,
+      "grad_norm": 0.19911793563914765,
+      "learning_rate": 1.1979274611398965e-05,
+      "loss": 0.2761,
+      "step": 495
+    },
+    {
+      "epoch": 0.4659832246039143,
+      "grad_norm": 0.21638725379085677,
+      "learning_rate": 1.1875647668393784e-05,
+      "loss": 0.2876,
+      "step": 500
+    },
+    {
+      "epoch": 0.4706430568499534,
+      "grad_norm": 0.21417364115903229,
+      "learning_rate": 1.1772020725388602e-05,
+      "loss": 0.2768,
+      "step": 505
+    },
+    {
+      "epoch": 0.47530288909599255,
+      "grad_norm": 0.19378036957825795,
+      "learning_rate": 1.1668393782383421e-05,
+      "loss": 0.2757,
+      "step": 510
+    },
+    {
+      "epoch": 0.47996272134203166,
+      "grad_norm": 0.20348312570963792,
+      "learning_rate": 1.1564766839378238e-05,
+      "loss": 0.2754,
+      "step": 515
+    },
+    {
+      "epoch": 0.4846225535880708,
+      "grad_norm": 0.20078805057918098,
+      "learning_rate": 1.1461139896373056e-05,
+      "loss": 0.2801,
+      "step": 520
+    },
+    {
+      "epoch": 0.48928238583411,
+      "grad_norm": 0.22711582634492097,
+      "learning_rate": 1.1357512953367878e-05,
+      "loss": 0.2805,
+      "step": 525
+    },
+    {
+      "epoch": 0.4939422180801491,
+      "grad_norm": 0.19833314424388493,
+      "learning_rate": 1.1253886010362695e-05,
+      "loss": 0.2749,
+      "step": 530
+    },
+    {
+      "epoch": 0.49860205032618826,
+      "grad_norm": 0.20627918163276826,
+      "learning_rate": 1.1150259067357514e-05,
+      "loss": 0.2807,
+      "step": 535
+    },
+    {
+      "epoch": 0.5032618825722274,
+      "grad_norm": 0.22111449652544438,
+      "learning_rate": 1.1046632124352332e-05,
+      "loss": 0.2672,
+      "step": 540
+    },
+    {
+      "epoch": 0.5079217148182665,
+      "grad_norm": 0.2245796946555213,
+      "learning_rate": 1.094300518134715e-05,
+      "loss": 0.2745,
+      "step": 545
+    },
+    {
+      "epoch": 0.5125815470643057,
+      "grad_norm": 0.21913989463492667,
+      "learning_rate": 1.083937823834197e-05,
+      "loss": 0.2808,
+      "step": 550
+    },
+    {
+      "epoch": 0.5172413793103449,
+      "grad_norm": 0.21743987494816502,
+      "learning_rate": 1.073575129533679e-05,
+      "loss": 0.2805,
+      "step": 555
+    },
+    {
+      "epoch": 0.5219012115563839,
+      "grad_norm": 0.2223044993066891,
+      "learning_rate": 1.0632124352331608e-05,
+      "loss": 0.2875,
+      "step": 560
+    },
+    {
+      "epoch": 0.5265610438024231,
+      "grad_norm": 0.18695066177700756,
+      "learning_rate": 1.0528497409326426e-05,
+      "loss": 0.2643,
+      "step": 565
+    },
+    {
+      "epoch": 0.5312208760484622,
+      "grad_norm": 0.21093529744919742,
+      "learning_rate": 1.0424870466321243e-05,
+      "loss": 0.2699,
+      "step": 570
+    },
+    {
+      "epoch": 0.5358807082945014,
+      "grad_norm": 0.19752807407672002,
+      "learning_rate": 1.0321243523316062e-05,
+      "loss": 0.2734,
+      "step": 575
+    },
+    {
+      "epoch": 0.5405405405405406,
+      "grad_norm": 0.20088984513783537,
+      "learning_rate": 1.021761658031088e-05,
+      "loss": 0.2712,
+      "step": 580
+    },
+    {
+      "epoch": 0.5452003727865797,
+      "grad_norm": 0.20325055999916153,
+      "learning_rate": 1.01139896373057e-05,
+      "loss": 0.2774,
+      "step": 585
+    },
+    {
+      "epoch": 0.5498602050326188,
+      "grad_norm": 0.22174441086027497,
+      "learning_rate": 1.0010362694300519e-05,
+      "loss": 0.2689,
+      "step": 590
+    },
+    {
+      "epoch": 0.554520037278658,
+      "grad_norm": 0.20061271709209516,
+      "learning_rate": 9.906735751295338e-06,
+      "loss": 0.273,
+      "step": 595
+    },
+    {
+      "epoch": 0.5591798695246971,
+      "grad_norm": 0.2226596622045658,
+      "learning_rate": 9.803108808290156e-06,
+      "loss": 0.2756,
+      "step": 600
+    },
+    {
+      "epoch": 0.5638397017707363,
+      "grad_norm": 0.19697076210420275,
+      "learning_rate": 9.699481865284975e-06,
+      "loss": 0.2732,
+      "step": 605
+    },
+    {
+      "epoch": 0.5684995340167754,
+      "grad_norm": 0.22175232192708388,
+      "learning_rate": 9.595854922279793e-06,
+      "loss": 0.2812,
+      "step": 610
+    },
+    {
+      "epoch": 0.5731593662628145,
+      "grad_norm": 0.21147846013410135,
+      "learning_rate": 9.492227979274612e-06,
+      "loss": 0.2729,
+      "step": 615
+    },
+    {
+      "epoch": 0.5778191985088537,
+      "grad_norm": 0.20160869332931788,
+      "learning_rate": 9.388601036269432e-06,
+      "loss": 0.2663,
+      "step": 620
+    },
+    {
+      "epoch": 0.5824790307548928,
+      "grad_norm": 0.20814998856523514,
+      "learning_rate": 9.284974093264249e-06,
+      "loss": 0.2787,
+      "step": 625
+    },
+    {
+      "epoch": 0.587138863000932,
+      "grad_norm": 0.1873990471712869,
+      "learning_rate": 9.181347150259067e-06,
+      "loss": 0.2784,
+      "step": 630
+    },
+    {
+      "epoch": 0.5917986952469712,
+      "grad_norm": 0.21324250407573406,
+      "learning_rate": 9.077720207253888e-06,
+      "loss": 0.2791,
+      "step": 635
+    },
+    {
+      "epoch": 0.5964585274930102,
+      "grad_norm": 0.19205913704094654,
+      "learning_rate": 8.974093264248706e-06,
+      "loss": 0.2808,
+      "step": 640
+    },
+    {
+      "epoch": 0.6011183597390494,
+      "grad_norm": 0.20657296059347666,
+      "learning_rate": 8.870466321243523e-06,
+      "loss": 0.2757,
+      "step": 645
+    },
+    {
+      "epoch": 0.6057781919850885,
+      "grad_norm": 0.20355030156034296,
+      "learning_rate": 8.766839378238343e-06,
+      "loss": 0.2774,
+      "step": 650
+    },
+    {
+      "epoch": 0.6104380242311277,
+      "grad_norm": 0.210129052042712,
+      "learning_rate": 8.663212435233162e-06,
+      "loss": 0.2813,
+      "step": 655
+    },
+    {
+      "epoch": 0.6150978564771669,
+      "grad_norm": 0.20693376106919167,
+      "learning_rate": 8.55958549222798e-06,
+      "loss": 0.2654,
+      "step": 660
+    },
+    {
+      "epoch": 0.6197576887232059,
+      "grad_norm": 0.20048258208742983,
+      "learning_rate": 8.455958549222799e-06,
+      "loss": 0.2686,
+      "step": 665
+    },
+    {
+      "epoch": 0.6244175209692451,
+      "grad_norm": 0.19402957660001943,
+      "learning_rate": 8.352331606217617e-06,
+      "loss": 0.2695,
+      "step": 670
+    },
+    {
+      "epoch": 0.6290773532152842,
+      "grad_norm": 0.18920776684229643,
+      "learning_rate": 8.248704663212436e-06,
+      "loss": 0.2673,
+      "step": 675
+    },
+    {
+      "epoch": 0.6337371854613234,
+      "grad_norm": 0.21815092086301652,
+      "learning_rate": 8.145077720207254e-06,
+      "loss": 0.2788,
+      "step": 680
+    },
+    {
+      "epoch": 0.6383970177073626,
+      "grad_norm": 0.18249459905289228,
+      "learning_rate": 8.041450777202073e-06,
+      "loss": 0.2794,
+      "step": 685
+    },
+    {
+      "epoch": 0.6430568499534017,
+      "grad_norm": 0.19591809096684484,
+      "learning_rate": 7.937823834196891e-06,
+      "loss": 0.2715,
+      "step": 690
+    },
+    {
+      "epoch": 0.6477166821994408,
+      "grad_norm": 0.19514031116444766,
+      "learning_rate": 7.834196891191712e-06,
+      "loss": 0.2795,
+      "step": 695
+    },
+    {
+      "epoch": 0.65237651444548,
+      "grad_norm": 0.19204390954009534,
+      "learning_rate": 7.730569948186528e-06,
+      "loss": 0.2837,
+      "step": 700
+    },
+    {
+      "epoch": 0.6570363466915191,
+      "grad_norm": 0.1814003327176538,
+      "learning_rate": 7.626943005181348e-06,
+      "loss": 0.27,
+      "step": 705
+    },
+    {
+      "epoch": 0.6616961789375583,
+      "grad_norm": 0.20946585680553592,
+      "learning_rate": 7.523316062176167e-06,
+      "loss": 0.2704,
+      "step": 710
+    },
+    {
+      "epoch": 0.6663560111835974,
+      "grad_norm": 0.1921568693032343,
+      "learning_rate": 7.419689119170985e-06,
+      "loss": 0.2677,
+      "step": 715
+    },
+    {
+      "epoch": 0.6710158434296365,
+      "grad_norm": 0.19912449557751416,
+      "learning_rate": 7.3160621761658035e-06,
+      "loss": 0.2703,
+      "step": 720
+    },
+    {
+      "epoch": 0.6756756756756757,
+      "grad_norm": 0.20566000197011958,
+      "learning_rate": 7.212435233160623e-06,
+      "loss": 0.2766,
+      "step": 725
+    },
+    {
+      "epoch": 0.6803355079217148,
+      "grad_norm": 0.20652162303920849,
+      "learning_rate": 7.108808290155441e-06,
+      "loss": 0.2727,
+      "step": 730
+    },
+    {
+      "epoch": 0.684995340167754,
+      "grad_norm": 0.19038258519772516,
+      "learning_rate": 7.005181347150259e-06,
+      "loss": 0.2724,
+      "step": 735
+    },
+    {
+      "epoch": 0.6896551724137931,
+      "grad_norm": 0.19490385886001102,
+      "learning_rate": 6.9015544041450784e-06,
+      "loss": 0.2675,
+      "step": 740
+    },
+    {
+      "epoch": 0.6943150046598322,
+      "grad_norm": 0.18560068669657961,
+      "learning_rate": 6.797927461139897e-06,
+      "loss": 0.2702,
+      "step": 745
+    },
+    {
+      "epoch": 0.6989748369058714,
+      "grad_norm": 0.19621768587568514,
+      "learning_rate": 6.6943005181347155e-06,
+      "loss": 0.2692,
+      "step": 750
+    },
+    {
+      "epoch": 0.7036346691519105,
+      "grad_norm": 0.2112507969969134,
+      "learning_rate": 6.590673575129535e-06,
+      "loss": 0.2743,
+      "step": 755
+    },
+    {
+      "epoch": 0.7082945013979497,
+      "grad_norm": 0.18981955629490066,
+      "learning_rate": 6.487046632124353e-06,
+      "loss": 0.2723,
+      "step": 760
+    },
+    {
+      "epoch": 0.7129543336439889,
+      "grad_norm": 0.18268361672429612,
+      "learning_rate": 6.383419689119171e-06,
+      "loss": 0.2689,
+      "step": 765
+    },
+    {
+      "epoch": 0.7176141658900279,
+      "grad_norm": 0.21759778168797214,
+      "learning_rate": 6.2797927461139905e-06,
+      "loss": 0.275,
+      "step": 770
+    },
+    {
+      "epoch": 0.7222739981360671,
+      "grad_norm": 0.19607867923889052,
+      "learning_rate": 6.176165803108809e-06,
+      "loss": 0.2784,
+      "step": 775
+    },
+    {
+      "epoch": 0.7269338303821062,
+      "grad_norm": 0.17816386997362446,
+      "learning_rate": 6.0725388601036275e-06,
+      "loss": 0.267,
+      "step": 780
+    },
+    {
+      "epoch": 0.7315936626281454,
+      "grad_norm": 0.21117162602410225,
+      "learning_rate": 5.968911917098445e-06,
+      "loss": 0.2698,
+      "step": 785
+    },
+    {
+      "epoch": 0.7362534948741846,
+      "grad_norm": 0.21060183599085328,
+      "learning_rate": 5.865284974093265e-06,
+      "loss": 0.2743,
+      "step": 790
+    },
+    {
+      "epoch": 0.7409133271202236,
+      "grad_norm": 0.19400476019142068,
+      "learning_rate": 5.761658031088083e-06,
+      "loss": 0.2748,
+      "step": 795
+    },
+    {
+      "epoch": 0.7455731593662628,
+      "grad_norm": 0.20052345073381453,
+      "learning_rate": 5.658031088082902e-06,
+      "loss": 0.2736,
+      "step": 800
+    },
+    {
+      "epoch": 0.750232991612302,
+      "grad_norm": 0.20728807092698712,
+      "learning_rate": 5.554404145077721e-06,
+      "loss": 0.2743,
+      "step": 805
+    },
+    {
+      "epoch": 0.7548928238583411,
+      "grad_norm": 0.19885878147551592,
+      "learning_rate": 5.4507772020725395e-06,
+      "loss": 0.2806,
+      "step": 810
+    },
+    {
+      "epoch": 0.7595526561043803,
+      "grad_norm": 0.18112546061809712,
+      "learning_rate": 5.347150259067357e-06,
+      "loss": 0.2721,
+      "step": 815
+    },
+    {
+      "epoch": 0.7642124883504194,
+      "grad_norm": 0.18692172501667362,
+      "learning_rate": 5.243523316062177e-06,
+      "loss": 0.2772,
+      "step": 820
+    },
+    {
+      "epoch": 0.7688723205964585,
+      "grad_norm": 0.18580108350557434,
+      "learning_rate": 5.139896373056995e-06,
+      "loss": 0.2691,
+      "step": 825
+    },
+    {
+      "epoch": 0.7735321528424977,
+      "grad_norm": 0.20217401036904592,
+      "learning_rate": 5.036269430051814e-06,
+      "loss": 0.2693,
+      "step": 830
+    },
+    {
+      "epoch": 0.7781919850885368,
+      "grad_norm": 0.1943652264496117,
+      "learning_rate": 4.932642487046633e-06,
+      "loss": 0.2706,
+      "step": 835
+    },
+    {
+      "epoch": 0.782851817334576,
+      "grad_norm": 0.2198072283577463,
+      "learning_rate": 4.829015544041451e-06,
+      "loss": 0.2767,
+      "step": 840
+    },
+    {
+      "epoch": 0.7875116495806151,
+      "grad_norm": 0.18668864356982678,
+      "learning_rate": 4.72538860103627e-06,
+      "loss": 0.2841,
+      "step": 845
+    },
+    {
+      "epoch": 0.7921714818266542,
+      "grad_norm": 0.2046695245569858,
+      "learning_rate": 4.621761658031089e-06,
+      "loss": 0.2691,
+      "step": 850
+    },
+    {
+      "epoch": 0.7968313140726934,
+      "grad_norm": 0.20500691517005445,
+      "learning_rate": 4.518134715025907e-06,
+      "loss": 0.2776,
+      "step": 855
+    },
+    {
+      "epoch": 0.8014911463187325,
+      "grad_norm": 0.19167582804792827,
+      "learning_rate": 4.414507772020726e-06,
+      "loss": 0.283,
+      "step": 860
+    },
+    {
+      "epoch": 0.8061509785647717,
+      "grad_norm": 0.19270689356754245,
+      "learning_rate": 4.310880829015544e-06,
+      "loss": 0.2731,
+      "step": 865
+    },
+    {
+      "epoch": 0.8108108108108109,
+      "grad_norm": 0.20242561543530732,
+      "learning_rate": 4.207253886010363e-06,
+      "loss": 0.276,
+      "step": 870
+    },
+    {
+      "epoch": 0.8154706430568499,
+      "grad_norm": 0.18482911338240235,
+      "learning_rate": 4.103626943005182e-06,
+      "loss": 0.274,
+      "step": 875
+    },
+    {
+      "epoch": 0.8201304753028891,
+      "grad_norm": 0.19023311518057515,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 0.2744,
+      "step": 880
+    },
+    {
+      "epoch": 0.8247903075489282,
+      "grad_norm": 0.17956559274009018,
+      "learning_rate": 3.896373056994819e-06,
+      "loss": 0.2753,
+      "step": 885
+    },
+    {
+      "epoch": 0.8294501397949674,
+      "grad_norm": 0.18533025491324542,
+      "learning_rate": 3.7927461139896377e-06,
+      "loss": 0.2716,
+      "step": 890
+    },
+    {
+      "epoch": 0.8341099720410066,
+      "grad_norm": 0.18868829919298247,
+      "learning_rate": 3.6891191709844567e-06,
+      "loss": 0.285,
+      "step": 895
+    },
+    {
+      "epoch": 0.8387698042870456,
+      "grad_norm": 0.1821294998845929,
+      "learning_rate": 3.5854922279792748e-06,
+      "loss": 0.275,
+      "step": 900
+    },
+    {
+      "epoch": 0.8434296365330848,
+      "grad_norm": 0.18362042280215524,
+      "learning_rate": 3.4818652849740937e-06,
+      "loss": 0.2717,
+      "step": 905
+    },
+    {
+      "epoch": 0.848089468779124,
+      "grad_norm": 0.18125344385824224,
+      "learning_rate": 3.3782383419689123e-06,
+      "loss": 0.275,
+      "step": 910
+    },
+    {
+      "epoch": 0.8527493010251631,
+      "grad_norm": 0.20049284941029782,
+      "learning_rate": 3.274611398963731e-06,
+      "loss": 0.2797,
+      "step": 915
+    },
+    {
+      "epoch": 0.8574091332712023,
+      "grad_norm": 0.19284600312397066,
+      "learning_rate": 3.1709844559585493e-06,
+      "loss": 0.2695,
+      "step": 920
+    },
+    {
+      "epoch": 0.8620689655172413,
+      "grad_norm": 0.18752491075216704,
+      "learning_rate": 3.0673575129533683e-06,
+      "loss": 0.2664,
+      "step": 925
+    },
+    {
+      "epoch": 0.8667287977632805,
+      "grad_norm": 0.1891493054852102,
+      "learning_rate": 2.963730569948187e-06,
+      "loss": 0.2675,
+      "step": 930
+    },
+    {
+      "epoch": 0.8713886300093197,
+      "grad_norm": 0.19851493019299982,
+      "learning_rate": 2.8601036269430053e-06,
+      "loss": 0.2699,
+      "step": 935
+    },
+    {
+      "epoch": 0.8760484622553588,
+      "grad_norm": 0.19015168393839105,
+      "learning_rate": 2.7564766839378243e-06,
+      "loss": 0.2741,
+      "step": 940
+    },
+    {
+      "epoch": 0.880708294501398,
+      "grad_norm": 0.19130488968167667,
+      "learning_rate": 2.6528497409326424e-06,
+      "loss": 0.2698,
+      "step": 945
+    },
+    {
+      "epoch": 0.8853681267474371,
+      "grad_norm": 0.19002010995722263,
+      "learning_rate": 2.5492227979274614e-06,
+      "loss": 0.2818,
+      "step": 950
+    },
+    {
+      "epoch": 0.8900279589934762,
+      "grad_norm": 0.18897353773835285,
+      "learning_rate": 2.44559585492228e-06,
+      "loss": 0.2745,
+      "step": 955
+    },
+    {
+      "epoch": 0.8946877912395154,
+      "grad_norm": 0.1902154880962153,
+      "learning_rate": 2.3419689119170984e-06,
+      "loss": 0.2798,
+      "step": 960
+    },
+    {
+      "epoch": 0.8993476234855545,
+      "grad_norm": 0.20192915350132004,
+      "learning_rate": 2.2383419689119174e-06,
+      "loss": 0.2771,
+      "step": 965
+    },
+    {
+      "epoch": 0.9040074557315937,
+      "grad_norm": 0.18079604071938268,
+      "learning_rate": 2.134715025906736e-06,
+      "loss": 0.265,
+      "step": 970
+    },
+    {
+      "epoch": 0.9086672879776329,
+      "grad_norm": 0.1843476030030155,
+      "learning_rate": 2.0310880829015544e-06,
+      "loss": 0.2807,
+      "step": 975
+    },
+    {
+      "epoch": 0.9133271202236719,
+      "grad_norm": 0.2150486826165296,
+      "learning_rate": 1.9274611398963734e-06,
+      "loss": 0.2727,
+      "step": 980
+    },
+    {
+      "epoch": 0.9179869524697111,
+      "grad_norm": 0.19640451524415894,
+      "learning_rate": 1.823834196891192e-06,
+      "loss": 0.2705,
+      "step": 985
+    },
+    {
+      "epoch": 0.9226467847157502,
+      "grad_norm": 0.1881146695565526,
+      "learning_rate": 1.7202072538860104e-06,
+      "loss": 0.2689,
+      "step": 990
+    },
+    {
+      "epoch": 0.9273066169617894,
+      "grad_norm": 0.1919966422921075,
+      "learning_rate": 1.6165803108808292e-06,
+      "loss": 0.2739,
+      "step": 995
+    },
+    {
+      "epoch": 0.9319664492078286,
+      "grad_norm": 0.18379110079996336,
+      "learning_rate": 1.5129533678756477e-06,
+      "loss": 0.2726,
+      "step": 1000
+    },
+    {
+      "epoch": 0.9366262814538676,
+      "grad_norm": 0.18038958121343104,
+      "learning_rate": 1.4093264248704663e-06,
+      "loss": 0.2756,
+      "step": 1005
+    },
+    {
+      "epoch": 0.9412861136999068,
+      "grad_norm": 0.18912068564324505,
+      "learning_rate": 1.3056994818652852e-06,
+      "loss": 0.2757,
+      "step": 1010
+    },
+    {
+      "epoch": 0.9459459459459459,
+      "grad_norm": 0.19447222147289886,
+      "learning_rate": 1.2020725388601037e-06,
+      "loss": 0.2773,
+      "step": 1015
+    },
+    {
+      "epoch": 0.9506057781919851,
+      "grad_norm": 0.17799555766267627,
+      "learning_rate": 1.0984455958549225e-06,
+      "loss": 0.2736,
+      "step": 1020
+    },
+    {
+      "epoch": 0.9552656104380243,
+      "grad_norm": 0.1879289390938795,
+      "learning_rate": 9.94818652849741e-07,
+      "loss": 0.2829,
+      "step": 1025
+    },
+    {
+      "epoch": 0.9599254426840633,
+      "grad_norm": 0.18465306001205795,
+      "learning_rate": 8.911917098445596e-07,
+      "loss": 0.2712,
+      "step": 1030
+    },
+    {
+      "epoch": 0.9645852749301025,
+      "grad_norm": 0.18684975369398882,
+      "learning_rate": 7.875647668393784e-07,
+      "loss": 0.2762,
+      "step": 1035
+    },
+    {
+      "epoch": 0.9692451071761417,
+      "grad_norm": 0.18354813003667567,
+      "learning_rate": 6.839378238341969e-07,
+      "loss": 0.2723,
+      "step": 1040
+    },
+    {
+      "epoch": 0.9739049394221808,
+      "grad_norm": 0.1790853955275003,
+      "learning_rate": 5.803108808290156e-07,
+      "loss": 0.2686,
+      "step": 1045
+    },
+    {
+      "epoch": 0.97856477166822,
+      "grad_norm": 0.18958296444822564,
+      "learning_rate": 4.7668393782383424e-07,
+      "loss": 0.286,
+      "step": 1050
+    },
+    {
+      "epoch": 0.983224603914259,
+      "grad_norm": 0.1857121657317691,
+      "learning_rate": 3.730569948186528e-07,
+      "loss": 0.2674,
+      "step": 1055
+    },
+    {
+      "epoch": 0.9878844361602982,
+      "grad_norm": 0.17831325037974058,
+      "learning_rate": 2.694300518134715e-07,
+      "loss": 0.272,
+      "step": 1060
+    },
+    {
+      "epoch": 0.9925442684063374,
+      "grad_norm": 0.18156458650747095,
+      "learning_rate": 1.6580310880829015e-07,
+      "loss": 0.2746,
+      "step": 1065
+    },
+    {
+      "epoch": 0.9972041006523765,
+      "grad_norm": 0.18194645895560613,
+      "learning_rate": 6.217616580310881e-08,
+      "loss": 0.2705,
+      "step": 1070
+    },
+    {
+      "epoch": 1.0,
+      "step": 1073,
+      "total_flos": 9.186429923093381e+17,
+      "train_loss": 0.27417567097351914,
+      "train_runtime": 29088.9913,
+      "train_samples_per_second": 0.59,
+      "train_steps_per_second": 0.037
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1073,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 9.186429923093381e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}