Environment Hub
Collection
A collection of OpenEnv-spec Environments
•
11 items
•
Updated
•
21
AI & ML interests
None defined yet.
Recent Activity
sergiopaniego
updated
a
collection
5 days ago
sergiopaniego
posted
an
update
7 days ago
Post
1502
FunctionGemma Tuning Lab is a new no-code tool by
@google
that lets you fine-tune a model directly from the browser, with no coding knowledge required, using TRL behind the scenes.
blog: https://developers.googleblog.com/a-guide-to-fine-tuning-functiongemma/
try it out: google/functiongemma-tuning-lab
This example builds on a more advanced one for learning fine-tuning with SFT using TRL: https://ai.google.dev/gemma/docs/functiongemma/finetuning-with-functiongemma
blog: https://developers.googleblog.com/a-guide-to-fine-tuning-functiongemma/
try it out: google/functiongemma-tuning-lab
This example builds on a more advanced one for learning fine-tuning with SFT using TRL: https://ai.google.dev/gemma/docs/functiongemma/finetuning-with-functiongemma
- 1 reply
sergiopaniego
posted
an
update
10 days ago
Post
724
TRL v0.27.0 is out!! 🥳
It includes GDPO, the latest variant of GRPO for multi-reward RL ✨
GDPO decouples reward normalization to avoid reward collapse and improve per-reward convergence — developed by
@sliuau @SimonX et al.
Explore the paper: GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization (2601.05242)
Explore the full set of changes here:
https://github.com/huggingface/trl/releases/tag/v0.27.0
It includes GDPO, the latest variant of GRPO for multi-reward RL ✨
GDPO decouples reward normalization to avoid reward collapse and improve per-reward convergence — developed by
@sliuau @SimonX et al.
Explore the paper: GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization (2601.05242)
Explore the full set of changes here:
https://github.com/huggingface/trl/releases/tag/v0.27.0
sergiopaniego
updated
a
collection
13 days ago
sergiopaniego
posted
an
update
13 days ago
Post
2942
New REPL environment in OpenEnv available! ✨
Used in the Recursive Language Models (RLM) paper by Alex Zhang.
Ready for inference & post-training using trajectories. Handles long contexts:
> Run Python code in a sandbox
> Make recursive calls to LMs
> Explore data programmatically
> Return final result
Docs: https://meta-pytorch.org/OpenEnv/environments/repl/
Inference script: https://github.com/meta-pytorch/OpenEnv/blob/main/examples/repl_oolong_simple.py
Used in the Recursive Language Models (RLM) paper by Alex Zhang.
Ready for inference & post-training using trajectories. Handles long contexts:
> Run Python code in a sandbox
> Make recursive calls to LMs
> Explore data programmatically
> Return final result
Docs: https://meta-pytorch.org/OpenEnv/environments/repl/
Inference script: https://github.com/meta-pytorch/OpenEnv/blob/main/examples/repl_oolong_simple.py
sergiopaniego
posted
an
update
14 days ago
Post
408
Recursive Language Models (RLM) is a new interface for LLMs with cool ideas by Alex Zhang!
⚠️ LLMs struggle with long prompts → attention overload & lost info
🔄 RLMs inspect, split & call themselves on chunks, then aggregate results
✅ Handles millions of tokens, reduces noise, improves reasoning
💡 System prompt guides recursion
🎯 RLM trajectories can be used for RL training or distillation (OpenEnv+TRL!!)
We're adding it to OpenEnv (with Kashif Rasul): https://github.com/meta-pytorch/OpenEnv/pull/282
More resources:
> Paper: Recursive Language Models (2512.24601)
> Paper blog: https://alexzhang13.github.io/blog/2025/rlm/
> RLM repo: https://github.com/alexzhang13/rlm
⚠️ LLMs struggle with long prompts → attention overload & lost info
🔄 RLMs inspect, split & call themselves on chunks, then aggregate results
✅ Handles millions of tokens, reduces noise, improves reasoning
💡 System prompt guides recursion
🎯 RLM trajectories can be used for RL training or distillation (OpenEnv+TRL!!)
We're adding it to OpenEnv (with Kashif Rasul): https://github.com/meta-pytorch/OpenEnv/pull/282
More resources:
> Paper: Recursive Language Models (2512.24601)
> Paper blog: https://alexzhang13.github.io/blog/2025/rlm/
> RLM repo: https://github.com/alexzhang13/rlm
- 2 replies
burtenshaw
updated
a
Space
14 days ago
sergiopaniego
posted
an
update
18 days ago
Post
2224
New GRPO + TRL free Colab notebook out! 🔥
Fine-tune 7B+ models on T4 GPUs thanks to a ton of memory optimizations for GRPO
7B model uses only 9.2 GB VRAM (~7× reduction) 🤯
Try the notebook here 👉 https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_trl_lora_qlora.ipynb
Fine-tune 7B+ models on T4 GPUs thanks to a ton of memory optimizations for GRPO
7B model uses only 9.2 GB VRAM (~7× reduction) 🤯
Try the notebook here 👉 https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_trl_lora_qlora.ipynb
sergiopaniego
posted
an
update
24 days ago
Post
2575
The list of hands-on notebooks (some beginner-friendly!) to get started with fine-tuning using TRL keeps growing!!
• SFT
• GRPO
• Tool calling & agents
• RL environments with OpenEnv
• LLMs and VLMs
✨ Many run on FREE Colab, making it super easy to get started fast!
https://github.com/huggingface/trl/tree/main/examples/notebooks
• SFT
• GRPO
• Tool calling & agents
• RL environments with OpenEnv
• LLMs and VLMs
✨ Many run on FREE Colab, making it super easy to get started fast!
https://github.com/huggingface/trl/tree/main/examples/notebooks
sergiopaniego
posted
an
update
27 days ago
Post
471
As the year comes to an end, it’s a good moment to catch up on some of the best long-form pieces published by the
@huggingface
team.
I’ve gathered them all here if you want to read or save them for later:
https://huggingface.co/collections/sergiopaniego/research-and-long-form-blog-posts
I’ve gathered them all here if you want to read or save them for later:
https://huggingface.co/collections/sergiopaniego/research-and-long-form-blog-posts
sergiopaniego
posted
an
update
28 days ago
Post
2298
This super detailed tutorial by
@Paulescu
is pure gold 🪙 "Fine-tuning a Small Language Model for browser control with GRPO and OpenEnv"
LFM2-350M ( @LiquidAI ) + BrowserGym (OpenEnv) + GRPO (TRL) for learning browser control 🤝
https://paulabartabajo.substack.com/p/fine-tuning-lfm2-350m-for-browser
LFM2-350M ( @LiquidAI ) + BrowserGym (OpenEnv) + GRPO (TRL) for learning browser control 🤝
https://paulabartabajo.substack.com/p/fine-tuning-lfm2-350m-for-browser
qgallouedec
submitted
a
paper
to Daily Papers
about 1 month ago
sergiopaniego
posted
an
update
about 1 month ago
Post
488
if you’re on holidays 🎄 and want some reading, here are blogs I contributed to this year:
🦄 VLMs in TRL: https://huggingface.co/blog/trl-vlm-alignment
🦖 VLMs in 2025: https://huggingface.co/blog/vlms-2025
👾 tokenization in transformers v5: https://huggingface.co/blog/tokenizers
🛸 faster transformers: https://huggingface.co/blog/faster-transformers
🦄 VLMs in TRL: https://huggingface.co/blog/trl-vlm-alignment
🦖 VLMs in 2025: https://huggingface.co/blog/vlms-2025
👾 tokenization in transformers v5: https://huggingface.co/blog/tokenizers
🛸 faster transformers: https://huggingface.co/blog/faster-transformers
sergiopaniego
posted
an
update
about 1 month ago
Post
2003
The Christmas holidays are here! 🎄
Thinking about learning something new in AI?
@huggingface offers 12 FREE courses covering all the relevant topics, for every level of experience. A great challenge for the holidays (and worth saving for later 🙄)
Let’s explore them!
🧠 𝗟𝗟𝗠 𝗖𝗼𝘂𝗿𝘀𝗲: large language models with HF tools
https://huggingface.co/learn/llm-course
🤖 𝗔𝗴𝗲𝗻𝘁𝘀 𝗖𝗼𝘂𝗿𝘀𝗲: build and deploy AI agents
https://huggingface.co/learn/agents-course
🎨 𝗗𝗶𝗳𝗳𝘂𝘀𝗶𝗼𝗻 𝗖𝗼𝘂𝗿𝘀𝗲: diffusion models with 🤗 Diffusers
https://huggingface.co/learn/diffusion-course
🔊 𝗔𝘂𝗱𝗶𝗼 𝗖𝗼𝘂𝗿𝘀𝗲: transformers for audio tasks
https://huggingface.co/learn/audio-course
🎮 𝗗𝗲𝗲𝗽 𝗥𝗟 𝗖𝗼𝘂𝗿𝘀𝗲: deep reinforcement learning
https://huggingface.co/learn/deep-rl-course
👁️ 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝘆 𝗖𝗼𝗺𝗽𝘂𝘁𝗲𝗿 𝗩𝗶𝘀𝗶𝗼𝗻 𝗖𝗼𝘂𝗿𝘀𝗲: modern computer vision with HF
https://huggingface.co/learn/computer-vision-course
🦾 𝗥𝗼𝗯𝗼𝘁𝗶𝗰𝘀 𝗖𝗼𝘂𝗿𝘀𝗲 (𝗟𝗲𝗥𝗼𝗯𝗼𝘁): learning-based robotics
https://huggingface.co/learn/robotics-course
🧩 𝗠𝗖𝗣 𝗖𝗼𝘂𝗿𝘀𝗲: Model Context Protocol explained
https://huggingface.co/learn/mcp-course
🧪 𝗔 𝗦𝗺𝗼𝗹 𝗖𝗼𝘂𝗿𝘀𝗲: post-training AI models
https://huggingface.co/learn/a-smol-course
🕹️ 𝗠𝗟 𝗳𝗼𝗿 𝗚𝗮𝗺𝗲𝘀: AI in game development
https://huggingface.co/learn/ml-for-games-course
🧊 𝗠𝗟 𝗳𝗼𝗿 𝟯𝗗: machine learning for 3D data
https://huggingface.co/learn/ml-for-3d-course
📘 𝗢𝗽𝗲𝗻-𝗦𝗼𝘂𝗿𝗰𝗲 𝗔𝗜 𝗖𝗼𝗼𝗸𝗯𝗼𝗼𝗸: practical AI notebooks
https://huggingface.co/learn/cookbook
All of them can be found here: https://huggingface.co/learn
Thinking about learning something new in AI?
@huggingface offers 12 FREE courses covering all the relevant topics, for every level of experience. A great challenge for the holidays (and worth saving for later 🙄)
Let’s explore them!
🧠 𝗟𝗟𝗠 𝗖𝗼𝘂𝗿𝘀𝗲: large language models with HF tools
https://huggingface.co/learn/llm-course
🤖 𝗔𝗴𝗲𝗻𝘁𝘀 𝗖𝗼𝘂𝗿𝘀𝗲: build and deploy AI agents
https://huggingface.co/learn/agents-course
🎨 𝗗𝗶𝗳𝗳𝘂𝘀𝗶𝗼𝗻 𝗖𝗼𝘂𝗿𝘀𝗲: diffusion models with 🤗 Diffusers
https://huggingface.co/learn/diffusion-course
🔊 𝗔𝘂𝗱𝗶𝗼 𝗖𝗼𝘂𝗿𝘀𝗲: transformers for audio tasks
https://huggingface.co/learn/audio-course
🎮 𝗗𝗲𝗲𝗽 𝗥𝗟 𝗖𝗼𝘂𝗿𝘀𝗲: deep reinforcement learning
https://huggingface.co/learn/deep-rl-course
👁️ 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝘆 𝗖𝗼𝗺𝗽𝘂𝘁𝗲𝗿 𝗩𝗶𝘀𝗶𝗼𝗻 𝗖𝗼𝘂𝗿𝘀𝗲: modern computer vision with HF
https://huggingface.co/learn/computer-vision-course
🦾 𝗥𝗼𝗯𝗼𝘁𝗶𝗰𝘀 𝗖𝗼𝘂𝗿𝘀𝗲 (𝗟𝗲𝗥𝗼𝗯𝗼𝘁): learning-based robotics
https://huggingface.co/learn/robotics-course
🧩 𝗠𝗖𝗣 𝗖𝗼𝘂𝗿𝘀𝗲: Model Context Protocol explained
https://huggingface.co/learn/mcp-course
🧪 𝗔 𝗦𝗺𝗼𝗹 𝗖𝗼𝘂𝗿𝘀𝗲: post-training AI models
https://huggingface.co/learn/a-smol-course
🕹️ 𝗠𝗟 𝗳𝗼𝗿 𝗚𝗮𝗺𝗲𝘀: AI in game development
https://huggingface.co/learn/ml-for-games-course
🧊 𝗠𝗟 𝗳𝗼𝗿 𝟯𝗗: machine learning for 3D data
https://huggingface.co/learn/ml-for-3d-course
📘 𝗢𝗽𝗲𝗻-𝗦𝗼𝘂𝗿𝗰𝗲 𝗔𝗜 𝗖𝗼𝗼𝗸𝗯𝗼𝗼𝗸: practical AI notebooks
https://huggingface.co/learn/cookbook
All of them can be found here: https://huggingface.co/learn
sergiopaniego
posted
an
update
about 1 month ago
Post
1894
Google DeepMind releases FunctionGemma, a 240M model specialized in 🔧 tool calling, built for fine-tuning
TRL has day-0 support. To celebrate, we’re sharing 2 new resources:
> Colab guide to fine-tune it for 🌐 browser control with BrowserGym OpenEnv
> Standalone training script
> Colab notebook: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_functiongemma_browsergym_openenv.ipynb
> Training script: https://github.com/huggingface/trl/blob/main/examples/scripts/openenv/browsergym_llm.py (command to run it inside the script)
> More notebooks in TRL: https://huggingface.co/docs/trl/example_overview#notebooks
TRL has day-0 support. To celebrate, we’re sharing 2 new resources:
> Colab guide to fine-tune it for 🌐 browser control with BrowserGym OpenEnv
> Standalone training script
> Colab notebook: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_functiongemma_browsergym_openenv.ipynb
> Training script: https://github.com/huggingface/trl/blob/main/examples/scripts/openenv/browsergym_llm.py (command to run it inside the script)
> More notebooks in TRL: https://huggingface.co/docs/trl/example_overview#notebooks
sergiopaniego
posted
an
update
about 1 month ago
sergiopaniego
posted
an
update
about 1 month ago
Post
2142
🎄 last talk of the year about open AI and HF today at Universidad Rey Juan Carlos for undergrad students
always a pleasure to be back at my alma mater
🎅 slides: https://github.com/sergiopaniego/talks
always a pleasure to be back at my alma mater
🎅 slides: https://github.com/sergiopaniego/talks
- 1 reply