---
title: "A Three-Phase Factual Recall Circuit in Gemma-2B and Gemma-12B-IT"
url: https://daily.dev/posts/a-three-phase-factual-recall-circuit-in-gemma-2b-and-gemma-12b-it-fxgapitlf
source_url: https://towardsdatascience.com/a-three-phase-factual-recall-circuit-in-gemma-2b-and-gemma-12b-it
type: article
source: "Towards Data Science"
published: 2026-06-24T15:37:00.402Z
updated: 2026-06-26T10:20:49.449Z
tags: ["llm", "gemma"]
reading_time: 9
upvotes: 1
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# A Three-Phase Factual Recall Circuit in Gemma-2B and Gemma-12B-IT

**[Towards Data Science](https://daily.dev/sources/tds)** · 9 min read · 1 upvotes · 0 comments

## Summary

An empirical mechanistic interpretability study of factual recall in Gemma-2B and Gemma-12B-IT using activation patching. The research identifies a consistent three-phase circuit: Phase 1 (Storage) encodes facts as directions in the residual stream at the entity token position in early-to-mid layers, with the residual stream contributing 40× more causally than attention outputs. Phase 2 (Routing) moves the signal to the final token position via distributed attention heads with no single dominant head. Phase 3 (Readout) retrieves the answer in late layers without additional computation. The pattern replicates proportionally at the 12B scale, with routing becoming even more distributed. The study also highlights tokenizer-induced dataset drift as a methodological concern for cross-model comparisons, and proposes path patching and SAE analysis as natural next steps.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://towardsdatascience.com/a-three-phase-factual-recall-circuit-in-gemma-2b-and-gemma-12b-it>

## Similar posts on daily.dev

- [Attention Is Bayesian Inference](https://daily.dev/posts/attention-is-bayesian-inference-y68katwus) · Hacker News · 1 upvotes · 0 comments
- [T5Gemma 2: The next generation of encoder-decoder models](https://daily.dev/posts/t5gemma-2-the-next-generation-of-encoder-decoder-models-nyxpiksgp) · Hacker News · 1 upvotes · 0 comments

---

Tags: [#llm](https://daily.dev/tags/llm), [#gemma](https://daily.dev/tags/gemma)

[View this post on daily.dev](https://daily.dev/posts/a-three-phase-factual-recall-circuit-in-gemma-2b-and-gemma-12b-it-fxgapitlf)