--- title: "Explore LLM word representations using similarity analysis (part 1)" url: https://daily.dev/posts/explore-llm-word-representations-using-similarity-analysis-part-1--rm86xpu3g source_url: https://thepalindrome.org/p/explore-llm-word-representations type: article source: "The Palindrome" published: 2026-04-22T12:49:09.138Z updated: 2026-04-22T12:49:34.894Z tags: ["python", "llm", "embeddings", "bert"] reading_time: 12 upvotes: 1 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Explore LLM word representations using similarity analysis (part 1) **[The Palindrome](https://daily.dev/sources/palindrome)** · 12 min read · 1 upvotes · 0 comments ## Summary Representational Similarity Analysis (RSA) is introduced as a technique for comparing how different language models encode words internally. Rather than directly comparing embedding vectors between models like BERT and GPT-2 (which fails due to random initialization and different embedding spaces), RSA computes cosine similarities between word pairs within each model and then correlates those similarity patterns across models. The post walks through extracting embeddings from BERT-large and GPT-2-medium, demonstrating that direct correlation yields near-zero results, while RSA reveals shared semantic structure. Code is provided via GitHub/Google Colab. Part 2 will extend the analysis to transformer layers and category selectivity. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Explore LLM word representations using similarity analysis \(part 2\)](https://daily.dev/posts/explore-llm-word-representations-using-similarity-analysis-part-2--snhtoyah3) · The Palindrome · 1 upvotes · 0 comments - [How Does Cosine Similarity Work? The Math Behind LLMs Explained](https://daily.dev/posts/how-does-cosine-similarity-work-the-math-behind-llms-explained-o5bvrsjhr) · freeCodeCamp · 2 upvotes · 0 comments - [Vector embeddings & language: how models turn words into geometry](https://daily.dev/posts/vector-embeddings-language-how-models-turn-words-into-geometry-io9eekrza) · Redis · 0 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python), [#llm](https://daily.dev/tags/llm), [#embeddings](https://daily.dev/tags/embeddings), [#bert](https://daily.dev/tags/bert) [View this post on daily.dev](https://daily.dev/posts/explore-llm-word-representations-using-similarity-analysis-part-1--rm86xpu3g)