--- title: Downstream Evaluations of Rotary Position Embeddings description: A head-to-head benchmark comparison of Rotary Position Embeddings (RoPE) versus GPT-style learned position embeddings, using two 1.3B parameter models trained... canonical: https://daily.dev/posts/downstream-evaluations-of-rotary-position-embeddings-btsietvad twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Downstream Evaluations of Rotary Position Embeddings | daily.dev og:description: A head-to-head benchmark comparison of Rotary Position Embeddings (RoPE) versus GPT-style learned position embeddings, using two 1.3B parameter models trained... og:url: https://daily.dev/posts/downstream-evaluations-of-rotary-position-embeddings-btsietvad og:image: https://api.daily.dev/og/posts/BtSietVAD.png og:image:alt: Downstream Evaluations of Rotary Position Embeddings og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Downstream Evaluations of Rotary Position Embeddings **[EleutherAI](https://daily.dev/sources/eleutherai)** · 3 min read · 0 upvotes · 0 comments ## Summary A head-to-head benchmark comparison of Rotary Position Embeddings (RoPE) versus GPT-style learned position embeddings, using two 1.3B parameter models trained for 100k steps on the Pile with Mesh Transformer JAX. Results across dozens of NLP tasks (lambada, hellaswag, boolq, winogrande, piqa, and many others) show RoPE generally performs slightly better or on par, with no strong consistent trend in either direction. A few tasks favor learned embeddings (triviaqa, sst, webqs), while RoPE leads on others (lambada perplexity, hellaswag, boolq, qnli). ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [RoPE, Clearly Explained](https://daily.dev/posts/rope-clearly-explained-ar0nzwulr) · Towards Data Science · 1 upvotes · 0 comments - [Rotary Position Embeddings for Long Context Length](https://daily.dev/posts/rotary-position-embeddings-for-long-context-length-n3cyl08i3) · Machine Learning Mastery · 0 upvotes · 0 comments - [DeepSeek-V3 Model: Theory, Config, and Rotary Positional Embeddings](https://daily.dev/posts/deepseek-v3-model-theory-config-and-rotary-positional-embeddings-41zyy7c6r) · PyImageSearch · 1 upvotes · 0 comments - [Hugging Face Introduces RTEB, a New Benchmark for Evaluating Retrieval Models](https://daily.dev/posts/hugging-face-introduces-rteb-a-new-benchmark-for-evaluating-retrieval-models-xufpoluug) · InfoQ · 4 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm) [View this post on daily.dev](https://daily.dev/posts/downstream-evaluations-of-rotary-position-embeddings-btsietvad) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Downstream Evaluations of Rotary Position Embeddings","url":"https://daily.dev/posts/downstream-evaluations-of-rotary-position-embeddings-btsietvad","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/downstream-evaluations-of-rotary-position-embeddings-btsietvad"},"datePublished":"2026-08-02T07:04:29.404Z","dateModified":"2026-08-02T07:31:29.763Z","description":"A head-to-head benchmark comparison of Rotary Position Embeddings (RoPE) versus GPT-style learned position embeddings, using two 1.3B parameter models trained...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7fa3fbd6e99da7e05ab42115e7e68fde?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7fa3fbd6e99da7e05ab42115e7e68fde?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"EleutherAI","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"EleutherAI","logo":"https://media.daily.dev/image/upload/s--aAcRyWGF--/f_auto,q_auto/v1785654263/logos/eleutherai","url":"https://daily.dev/sources/eleutherai"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/downstream-evaluations-of-rotary-position-embeddings-btsietvad","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"EleutherAI","item":"https://daily.dev/sources/eleutherai"},{"@type":"ListItem","position":3,"name":"Downstream Evaluations of Rotary Position Embeddings"}]} ```