--- title: Honey, I shrunk the embeddings: Matryoshka vs. PCA description: A benchmark comparison of Matryoshka Representation Learning (MRL) truncation versus PCA for reducing embedding dimensions across eight BEIR retrieval... canonical: https://daily.dev/posts/honey-i-shrunk-the-embeddings-matryoshka-vs-pca-tjrxhq3s2 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Honey, I shrunk the embeddings: Matryoshka vs. PCA | daily.dev og:description: A benchmark comparison of Matryoshka Representation Learning (MRL) truncation versus PCA for reducing embedding dimensions across eight BEIR retrieval... og:url: https://daily.dev/posts/honey-i-shrunk-the-embeddings-matryoshka-vs-pca-tjrxhq3s2 og:image: https://api.daily.dev/og/posts/TjrxhQ3S2.png og:image:alt: Honey, I shrunk the embeddings: Matryoshka vs. PCA og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Honey, I shrunk the embeddings: Matryoshka vs. PCA **[Hacker News](https://daily.dev/sources/hn)** · 12 min read · 0 upvotes · 0 comments ## Summary A benchmark comparison of Matryoshka Representation Learning (MRL) truncation versus PCA for reducing embedding dimensions across eight BEIR retrieval datasets. Testing three models (text-embedding-3-small, qwen3-embedding-8b, text-embedding-ada-002), PCA matched or outperformed MRL truncation at most compression levels — retaining 65% of quality at 32 dims versus truncation's 46% on text-embedding-3-small. PCA also proved robust to small fitting samples and out-of-domain fits, and works on any model regardless of MRL training. Combining binary quantization with PCA at 512 dims can reduce vector size to ~1% of float32 baseline while retaining 82% retrieval quality. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Is PCA better than Matryoshka embedding truncation for reducing vector dimensions in retrieval? PCA matched or outperformed MRL truncation at nearly every dimension tested across eight BEIR datasets. On OpenAI's text-embedding-3-small, PCA retained 65% of retrieval quality at 32 dimensions versus 46% for truncation. On Qwen3-Embedding-8B the gap was smaller, with MRL truncation holding a slight edge at 512 dimensions but PCA leading below 256 dimensions. _daily.dev surfaces practical benchmarks like this for engineers deciding how to shrink embedding vectors._ ### Does PCA dimensionality reduction work on embedding models that were not trained with Matryoshka Representation Learning? Yes, PCA works well even on text-embedding-ada-002, a 1,536-dimensional model never trained with MRL. Down to 128 dimensions it retains nearly the same share of quality as the MRL-trained text-embedding-3-small, with only a small gap opening at 64 and 32 dimensions (78% and 59% retained versus 82% and 65% for the MRL model). _developers comparing embedding compression options can track findings like this on daily.dev._ ### Does the sample size used to fit a PCA projection for embedding compression matter much? No, fit-sample size mattered little in testing on FiQA's 57K-document corpus: a PCA projection fit on just 1,000 documents performed almost the same as one fit on the full corpus at every tested dimension, on both text-embedding-3-small and qwen3-embedding-8b. An out-of-domain fit using 100,000 MS MARCO passages also held up at moderate compression levels. _daily.dev helps engineers weigh operational trade-offs like PCA fitting and maintenance for vector search._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 19 comments across hackernews (as of 2026-09-14). **TL;DR:** Commenters are largely receptive to the benchmarking work and unsurprised that PCA holds up well against MRL truncation, with several noting they've seen similar results independently; the main critical thread questions why PCA outperforms MRL when the original MRL paper's SVD baseline did not, and one commenter takes issue with the article's framing of search as a new problem. **Sentiment:** 65% positive · 25% mixed · 10% skeptical **The case for** - PCA comes with strong theoretical guarantees and proved surprisingly competitive against a purpose-trained technique. - PCA is model-agnostic and adds no inference latency overhead since it's applied post-embedding. - The benchmark is seen as a useful, practical contribution that fills a gap in publicly available comparisons. - Combining binary quantization with PCA offers dramatic vector-size reductions while retaining substantial retrieval quality. **The pushback** - The article's framing implies search/embeddings are a new problem, ignoring a long pre-LLM history. - It's unclear why PCA outperforms MRL truncation here when the original MRL paper's SVD baseline did not — the discrepancy isn't investigated. - Results may not be uniform across all embedding models; API-based models tested may not represent the full landscape. - PCA's O(n³) complexity makes it infeasible for very large datasets, which is where MRL's training-time approach retains appeal. **By community** - hackernews (positive): Generally enthusiastic about the practical benchmarking, with one substantive methodological question about the discrepancy with the original MRL paper's SVD baseline and a minor complaint about the article's historical framing. **Hottest debate:** Whether PCA's outperformance of MRL truncation is a genuine finding or an artifact of differing evaluation methodology compared to the original MRL paper. **Open questions** - Why does PCA outperform MRL truncation in this benchmark when the original MRL paper's SVD baseline showed degradation at low dimensions? - Would random-rotation quantization (as in TurboQuant) be complementary to PCA for text embeddings, given that text embeddings may not have structure aligned with basis vectors? - How do results generalize across a broader set of non-API embedding models? **Highlights** > Nice! Thank you for doing this. I am not surprised PCA holds up. It comes with some pretty strong theoretical guarantees. The problem with it has always been scalability. The O(n^3) complexity makes it infeasible to use for massive datasets. That is where something like Matryoshka embeddings has appeal. You trade a little bit of performance for a guarantee of training + validation set coverage. > — [pfisherman on hackernews](https://news.ycombinator.com/item?id=49232979) > The original MRL paper (https://arxiv.org/pdf/2205.13147) reported an SVD baseline, which showed comparable performance (Table 1, top-1 accuracy) at d>=256, but much degraded performance at lower dims (d \in {8,32,64}). (Though Table 2, nearest-neighbor accuracy, doesn't show degradation until d <= 16.) In your conclusion, you report that PCA won on most dimensions. Did you investigate why you found that PCA outperforms MRL when the original paper found that their SVD baseline did not? > — [jsrozner on hackernews · 1 comments](https://news.ycombinator.com/item?id=49233606) > Thanks for doing this benchmarking Dylan. I wanted to teach people PCA in my original article, but had no idea it would stack up this well against Matroyshka! Feels like a “just use logistic regression” moment :) > — [softwaredoug on hackernews · 1 comments](https://news.ycombinator.com/item?id=49232633) > Good benchmarks! > You can push this further by combining quantization with truncation or PCA. The resulting vectors can be dramatically smaller while still preserving a surprising amount of retrieval quality. Counterintuitively - quantisation can also be combined with a random rotation step before the quantisation. A random rotation spreads information across more dimensions, allowing more aggressive quantisation without losing accuracy. Ironically - almost the opposite of a PCA. I do wonder if relevant here though. It relies on the embeddings having "structure", i.e. that principal components point along basis vectors, which may not be the case with text embeddings. Source: https://research.google/blog/turboquant-redefining-ai-effici... > — [hanneshdc on hackernews · 1 comments](https://news.ycombinator.com/item?id=49232938) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49156607) · 60 points · 19 comments ## Similar posts on daily.dev - [5 Embedding Compression Techniques](https://daily.dev/posts/5-embedding-compression-techniques-xrwx2u8js) · Daily Dose of Data Science \| Avi Chawla \| Substack · 1 upvotes · 0 comments - [Scaling Vector Search: Comparing Quantization and Matryoshka Embeddings for 80% Cost Reduction](https://daily.dev/posts/scaling-vector-search-comparing-quantization-and-matryoshka-embeddings-for-80-cost-reduction-m7ol7shex) · Towards Data Science · 0 upvotes · 0 comments - [PCA: an embedding shrink-ray](https://daily.dev/posts/pca-an-embedding-shrink-ray-ayowx3qqb) · Software Doug · 0 upvotes · 0 comments --- Tags: [#data-science](https://daily.dev/tags/data-science), [#vector-search](https://daily.dev/tags/vector-search), [#embeddings](https://daily.dev/tags/embeddings) [View this post on daily.dev](https://daily.dev/posts/honey-i-shrunk-the-embeddings-matryoshka-vs-pca-tjrxhq3s2) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Honey, I shrunk the embeddings: Matryoshka vs. PCA","url":"https://daily.dev/posts/honey-i-shrunk-the-embeddings-matryoshka-vs-pca-tjrxhq3s2","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/honey-i-shrunk-the-embeddings-matryoshka-vs-pca-tjrxhq3s2"},"datePublished":"2026-08-09T16:47:57.530Z","dateModified":"2026-09-14T07:53:41.093Z","description":"A benchmark comparison of Matryoshka Representation Learning (MRL) truncation versus PCA for reducing embedding dimensions across eight BEIR retrieval...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f4d24c1e0db51d6951d9d1ddc6d48ae9?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f4d24c1e0db51d6951d9d1ddc6d48ae9?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/honey-i-shrunk-the-embeddings-matryoshka-vs-pca-tjrxhq3s2","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-science,vector-search,embeddings","timeRequired":"PT12M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Honey, I shrunk the embeddings: Matryoshka vs. PCA"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/honey-i-shrunk-the-embeddings-matryoshka-vs-pca-tjrxhq3s2#faq","mainEntity":[{"@type":"Question","name":"Is PCA better than Matryoshka embedding truncation for reducing vector dimensions in retrieval?","acceptedAnswer":{"@type":"Answer","text":"PCA matched or outperformed MRL truncation at nearly every dimension tested across eight BEIR datasets. On OpenAI's text-embedding-3-small, PCA retained 65% of retrieval quality at 32 dimensions versus 46% for truncation. On Qwen3-Embedding-8B the gap was smaller, with MRL truncation holding a slight edge at 512 dimensions but PCA leading below 256 dimensions. daily.dev surfaces practical benchmarks like this for engineers deciding how to shrink embedding vectors."}},{"@type":"Question","name":"Does PCA dimensionality reduction work on embedding models that were not trained with Matryoshka Representation Learning?","acceptedAnswer":{"@type":"Answer","text":"Yes, PCA works well even on text-embedding-ada-002, a 1,536-dimensional model never trained with MRL. Down to 128 dimensions it retains nearly the same share of quality as the MRL-trained text-embedding-3-small, with only a small gap opening at 64 and 32 dimensions (78% and 59% retained versus 82% and 65% for the MRL model). developers comparing embedding compression options can track findings like this on daily.dev."}},{"@type":"Question","name":"Does the sample size used to fit a PCA projection for embedding compression matter much?","acceptedAnswer":{"@type":"Answer","text":"No, fit-sample size mattered little in testing on FiQA's 57K-document corpus: a PCA projection fit on just 1,000 documents performed almost the same as one fit on the full corpus at every tested dimension, on both text-embedding-3-small and qwen3-embedding-8b. An out-of-domain fit using 100,000 MS MARCO passages also held up at moderate compression levels. daily.dev helps engineers weigh operational trade-offs like PCA fitting and maintenance for vector search."}}]} ```