turhancan97/wroclaw-dwarves
Viewer • Updated • 8.76k • 68 • 1
🤝 Open to Collab
Turhan Can Kargın
turhancan97
AI & ML interests
Computer Vision, Deep Learning, Machine Learning, Robotics, Reinforcement Learning, Control Theory
Recent Activity
liked a model 15 days ago
mistralai/Mistral-Large-4-1T-A52B posted an update 18 days ago
New blog post: Seeing ViT Registers 🔍
I reproduced the main observation of "Vision Transformers Need Registers" (Darcet et al.) on six DINOv2 checkpoints from the Hub: Base, Large and Giant, each with and without registers.
Three findings:
1️⃣ Model size matters. In DINOv2 Giant, ~1.6% of patch tokens form a separate high-norm group. In Large the effect is weak, and in Base I found none.
2️⃣ Where you measure matters. The artifact tokens have very high norms before the final LayerNorm (hidden_states[-1]), but the lowest norms after it (last_hidden_state).
3️⃣ Registers take over. With registers, patch maps are smooth, and one register token carries a norm of ~1,346 instead.
Everything is inference-only and reproducible with one command. The post also includes practical tips for people who use DINOv2 features for dense tasks or probing.
📝 Blog: https://huggingface.co/blog/turhancan97/seeing-vit-registers
💻 Code: https://github.com/turhancan97/seeing-vit-registers upvoted an article 18 days ago
Seeing ViT Registers: Model Size, LayerNorm, and Where the High-Norm Tokens GoOrganizations
