--- title: Vision Transformers Need Registers - Fixing a Bug in DINOv2? description: Researchers from Meta AI discovered that large vision transformer models like DINOv2 develop 'attention map artifacts' — outlier high-norm tokens in background... canonical: https://daily.dev/posts/vision-transformers-need-registers---fixing-a-bug-in-dinov2--qijvq87k0 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Vision Transformers Need Registers - Fixing a Bug in DINOv2? | daily.dev og:description: Researchers from Meta AI discovered that large vision transformer models like DINOv2 develop 'attention map artifacts' — outlier high-norm tokens in background... og:url: https://daily.dev/posts/vision-transformers-need-registers---fixing-a-bug-in-dinov2--qijvq87k0 og:image: https://api.daily.dev/og/posts/qiJvq87k0.png og:image:alt: Vision Transformers Need Registers - Fixing a Bug in DINOv2? og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Vision Transformers Need Registers - Fixing a Bug in DINOv2? **[AI Papers Academy](https://daily.dev/sources/aipapersacademy)** · 9 min read · 0 upvotes · 0 comments ## Summary Researchers from Meta AI discovered that large vision transformer models like DINOv2 develop 'attention map artifacts' — outlier high-norm tokens in background regions that store global information instead of local patch data. This degrades tasks like object discovery. The fix, called 'registers', adds extra tokens to the input sequence that the model uses to store global information instead of hijacking image patch tokens. Registers are discarded at output. Results show registers nearly eliminate artifacts and improve object discovery by ~20 points for DINOv2, with modest gains in segmentation and depth estimation, though DINOv1 still outperforms DINOv2+registers on object discovery. Classification and other tasks see only marginal improvements, so the memory/latency cost may not always be justified. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Glitches in the Attention Matrix](https://daily.dev/posts/glitches-in-the-attention-matrix-yoy7bgnhx) · Towards Data Science · 3 upvotes · 0 comments - [Taming Outlier Tokens in Diffusion Transformers](https://daily.dev/posts/taming-outlier-tokens-in-diffusion-transformers-fxlik44lf) · Apple Machine Learning Research · 4 upvotes · 0 comments - [Grounding DINO: Open Vocabulary Object Detection on Videos](https://daily.dev/posts/grounding-dino-open-vocabulary-object-detection-on-videos-g6hotulhw) · PyImageSearch · 0 upvotes · 0 comments --- Tags: [#deep-learning](https://daily.dev/tags/deep-learning), [#computer-vision](https://daily.dev/tags/computer-vision) [View this post on daily.dev](https://daily.dev/posts/vision-transformers-need-registers---fixing-a-bug-in-dinov2--qijvq87k0) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Vision Transformers Need Registers - Fixing a Bug in DINOv2?","url":"https://daily.dev/posts/vision-transformers-need-registers---fixing-a-bug-in-dinov2--qijvq87k0","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/vision-transformers-need-registers---fixing-a-bug-in-dinov2--qijvq87k0"},"datePublished":"2026-04-19T08:04:45.446Z","dateModified":"2026-04-19T08:05:47.524Z","description":"Researchers from Meta AI discovered that large vision transformer models like DINOv2 develop 'attention map artifacts' — outlier high-norm tokens in background...","image":"https://i.ytimg.com/vi/AwAgxXmkda0/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/AwAgxXmkda0/sddefault.jpg","isAccessibleForFree":true,"articleSection":"AI Papers Academy","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"AI Papers Academy","logo":"https://media.daily.dev/image/upload/s--O3RGOCh9--/f_auto,q_auto/v1776585874/logos/aipapersacademy","url":"https://daily.dev/sources/aipapersacademy"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/vision-transformers-need-registers---fixing-a-bug-in-dinov2--qijvq87k0","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"deep-learning,computer-vision","timeRequired":"PT9M","video":{"@type":"VideoObject","name":"Vision Transformers Need Registers - Fixing a Bug in DINOv2?","description":"Researchers from Meta AI discovered that large vision transformer models like DINOv2 develop 'attention map artifacts' — outlier high-norm tokens in background...","thumbnailUrl":"https://i.ytimg.com/vi/AwAgxXmkda0/sddefault.jpg","uploadDate":"2026-04-19T08:04:45.446Z","duration":"PT9M","url":"https://api.daily.dev/r/qiJvq87k0","embedUrl":"https://www.youtube.com/embed/AwAgxXmkda0"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"AI Papers Academy","item":"https://daily.dev/sources/aipapersacademy"},{"@type":"ListItem","position":3,"name":"Vision Transformers Need Registers - Fixing a Bug in DINOv2?"}]} ```