La Leaderboard

community
Activity Feed

AI & ML interests

LLM Leaderboard for Spanish Varieties and Official Languages

Recent Activity

hgalbraith  updated a dataset about 1 month ago
la-leaderboard/requests
mariagrandury  updated a dataset about 2 months ago
la-leaderboard/requests
mariagrandury  updated a dataset about 2 months ago
la-leaderboard/requests
View all activity

igorktech 
posted an update 4 days ago
view post
Post
68
📌 9,000 papers a day, 9 in 10 written by teams, yet literature review is still a solo job.

At the Nebius Build Week Hackathon (GTC Berlin) we built Pinned: a shared crime board for scientific literature. User can drop your seed papers onto a live multiplayer canvas, and an AI detective hunts down the leads your search missed. It pins each one next to your seeds with an exact quotes. Highlight passages, add sticky notes, ask questions: your team and the agent work the case together on one board!

Inside:
🧠 NVIDIA Nemotron on Nebius Token Factory (via Google ADK)
🔎 Tavily, PubMed, OpenAlex & Semantic Scholar via MCP, with exact-quote verification
🎨 Multiplayer tldraw canvas + Canvas MCP, so Claude or Cursor can join the board

We're competing in the Hugging Face Challenge: the most-liked Space by Sunday, Oct 25 wins. Try the board, watch the 2-minute pitch, and hit ❤️ in the space if you like it
Check it here nebius-build-week/pinned


albertvillanova 
posted an update 4 months ago
view post
Post
4,552
🎉 KTO is now part of the stable TRL API

As of Promote KTO to stable API, KTOTrainer and KTOConfig have graduated from trl.experimental to the stable trl API. https://github.com/huggingface/trl/pull/6175

This one closes out a long road. Over the past 6+ months, the "Align KTO with DPO" effort landed ~90 PRs methodically bringing KTO up to the standard we hold for stable trainers, one carefully-scoped change at a time:
- Feature parity with DPO: full VLM support (incl. multi-image), sync_ref_model, PEFT + Liger, ZeRO-3 + PEFT dtype fix, pad_to_multiple_of, activation offloading, IterableDataset and dict eval_dataset, remove_unused_columns, and reference-logprob precomputation at init.
- Consistency with DPO: aligned method order and signatures, tokenization, _prepare_dataset, PEFT handling, ref-model preparation for distributed training, and config layout — plus a new DataCollatorForKTO and output format. Metrics moved into _compute_loss and simplified to direct averages via the shared _metrics attribute.
- Removing legacy baggage: dropped encoder-decoder support, BOS/EOS handling, null_ref_context, generate_during_eval, model_init, preprocess_logits_for_metrics, model/ref adapter names, and several dead config knobs.
- Coverage: a full test suite mirroring DPO, text collator tests, VLM tests, and slow tests.
- The promotion itself: the experimental → stable move (#6175) and shim cleanup (#6287), handled so downstream users get a clean deprecation path.

Honestly, this has been one of the more complex tasks I've taken on since joining the team, not because any single change was hard, but because it demanded sustained consistency across a ~2,000-line trainer, with every branch, comment, and edge case kept in lockstep with DPO.

Huge thanks to everyone who reviewed along the way (especially @qgallouedec ), the incremental review cadence is exactly what kept this maintainable.

KTO now sits on equal footing with our other flagship trainers. 🚀
  • 2 replies
·