---
title: "Explore LLM word representations using similarity analysis (part 2)"
url: https://daily.dev/posts/explore-llm-word-representations-using-similarity-analysis-part-2--snhtoyah3
source_url: https://thepalindrome.org/p/explore-llm-word-representations-c57
type: article
source: "The Palindrome"
published: 2026-05-20T11:12:47.537Z
updated: 2026-05-20T11:13:12.857Z
tags: ["llm", "nlp", "pytorch"]
reading_time: 16
upvotes: 1
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Explore LLM word representations using similarity analysis (part 2)

**[The Palindrome](https://daily.dev/sources/palindrome)** · 16 min read · 1 upvotes · 0 comments

## Summary

A deep dive into Representational Similarity Analysis (RSA) applied to GPT-2-XL's internal attention matrices. Using PyTorch hook functions to extract Query, Key, and Value activations across all 48 transformer layers, the analysis shows that while Q, K, and V vectors are nearly orthogonal (near-zero direct correlations), their RSA scores remain consistently high (~0.85–0.9). Category separability analysis using Cohen's d reveals that semantic world-knowledge is encoded within each attention matrix, with within-category RSA scores consistently exceeding across-category scores across all layers. Includes runnable Google Colab code.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://thepalindrome.org/p/explore-llm-word-representations-c57>

## Similar posts on daily.dev

- [Explore LLM word representations using similarity analysis \(part 1\)](https://daily.dev/posts/explore-llm-word-representations-using-similarity-analysis-part-1--rm86xpu3g) · The Palindrome · 1 upvotes · 0 comments
- [The Soft Search That Taught Machines to Understand](https://daily.dev/posts/the-soft-search-that-taught-machines-to-understand-vswbbvpnk) · Medium · 0 upvotes · 0 comments
- [LLM Neuroanatomy II: Modern LLM Hacking and hints of a Universal Language?](https://daily.dev/posts/llm-neuroanatomy-ii-modern-llm-hacking-and-hints-of-a-universal-language--sedj667om) · Hacker News · 1 upvotes · 0 comments

---

Tags: [#llm](https://daily.dev/tags/llm), [#nlp](https://daily.dev/tags/nlp), [#pytorch](https://daily.dev/tags/pytorch)

[View this post on daily.dev](https://daily.dev/posts/explore-llm-word-representations-using-similarity-analysis-part-2--snhtoyah3)