LLM Neuroanatomy: How I Topped the AI Leaderboard Without Changing a Single Weight
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
A researcher topped the HuggingFace Open LLM Leaderboard in mid-2024 without training or modifying any weights. The method, called RYS (Repeat Yourself), works by identifying a specific block of middle transformer layers in a 72B model and duplicating them. Starting from two observations — that LLMs can reason in Base64 and that the Goliath-120B frankenmerge worked despite violating distribution assumptions — the author hypothesized that transformers have a functional anatomy: early layers encode input, late layers decode output, and middle layers contain discrete reasoning circuits. A 'brain scanner' was built by sweeping all possible layer duplication configurations and scoring them with two orthogonal probes: hard math guessing and emotional quotient benchmarks. The optimal config for Qwen2-72B was duplicating 7 specific middle layers, yielding +17.72% on MuSR and +8.16% on MATH with no weight changes. Heatmaps revealed that only circuit-sized blocks improve performance — single-layer duplication fails — suggesting middle layers form indivisible multi-step reasoning pipelines. Descendants of this model still dominate the leaderboard as of early 2026.
