--- title: Tracing Subliminal Learning Through Pretraining | daily.dev description: Subliminal learning, where a student model picks up a teacher's preference from seemingly unrelated training data, survives even when teacher and student... canonical: https://daily.dev/posts/tracing-subliminal-learning-through-pretraining-mnazwbvuf twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Tracing Subliminal Learning Through Pretraining | daily.dev og:description: Subliminal learning, where a student model picks up a teacher's preference from seemingly unrelated training data, survives even when teacher and student... og:url: https://daily.dev/posts/tracing-subliminal-learning-through-pretraining-mnazwbvuf og:image: https://api.daily.dev/og/posts/mnaZWbVuF.png og:image:alt: Tracing Subliminal Learning Through Pretraining og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Tracing Subliminal Learning Through Pretraining **[EleutherAI](https://daily.dev/sources/eleutherai)** · 12 min read · 1 upvotes · 0 comments ## Summary Subliminal learning, where a student model picks up a teacher's preference from seemingly unrelated training data, survives even when teacher and student differ in initialization, data order, pretraining corpus, size, and architecture. EleutherAI's study using Pythia, PolyPythia, and DataDecide OLMo models found that transfer persists across these differences. Transmitting and acquiring a preference turn out to be separate abilities that emerge at different points during pretraining. Number frequencies alone carried 40-92% of the transfer signal in Pythia experiments, but only 5-36% in OLMo. Transfer also occurred in Pythia-160M using full fine-tuning with plain SGD, no LoRA or adaptive optimizers required. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Does subliminal learning require the student and teacher to share the same model initialization? No, shared initialization is not required for subliminal learning to occur. Cross-model distillation experiments using Pythia, PolyPythia, and DataDecide OLMo models show that a teacher's preference can transfer to students even when initialization, pretraining data order, training corpus, model size, and architecture all differ between teacher and student. _Anyone distilling one model into another can track emerging research like this on daily.dev to avoid unintended trait transfer._ ### Can number frequency alone transmit a model's hidden preference during distillation? Yes, replacing a teacher's number sequences with synthetic numbers sampled independently from its number frequency distribution still retained 40-92% of the real-sequence transfer across four animal preferences in Pythia-160M variants, though only 5-36% in OLMo elephant experiments, showing which numbers are frequent matters beyond sequence structure. _Developers evaluating distillation pipelines for hidden bias risks can follow findings like this on daily.dev._ ### Does removing animal-related tokens from the teacher's output distribution eliminate subliminal preference transfer in soft-label distillation? No, removing 10 candidate animal tokens before computing the top-K tokens and softmax sharply reduces subliminal transfer but does not eliminate it completely, indicating animal token probabilities contribute substantially but are not the sole channel for the preference signal. _Teams auditing distillation setups for leaked bias can keep up with findings like this on daily.dev._ ## Similar posts on daily.dev - [Machine Learning Blog \| ML@CMU \| Carnegie Mellon University](https://daily.dev/posts/machine-learning-blog-ml-cmu-carnegie-mellon-university-yhdqaje8i) · ML CMU · 24 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#ai-safety](https://daily.dev/tags/ai-safety) [View this post on daily.dev](https://daily.dev/posts/tracing-subliminal-learning-through-pretraining-mnazwbvuf) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Tracing Subliminal Learning Through Pretraining","url":"https://daily.dev/posts/tracing-subliminal-learning-through-pretraining-mnazwbvuf","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/tracing-subliminal-learning-through-pretraining-mnazwbvuf"},"datePublished":"2026-10-25T19:05:17.209Z","dateModified":"2026-10-25T19:05:47.670Z","description":"Subliminal learning, where a student model picks up a teacher's preference from seemingly unrelated training data, survives even when teacher and student...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7e9f10a0cf02f156382e3c278ae0b160?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7e9f10a0cf02f156382e3c278ae0b160?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"EleutherAI","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"EleutherAI","logo":"https://media.daily.dev/image/upload/s--aAcRyWGF--/f_auto,q_auto/v1785654263/logos/eleutherai","url":"https://daily.dev/sources/eleutherai"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/tracing-subliminal-learning-through-pretraining-mnazwbvuf","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,ai-safety","timeRequired":"PT12M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"EleutherAI","item":"https://daily.dev/sources/eleutherai"},{"@type":"ListItem","position":3,"name":"Tracing Subliminal Learning Through Pretraining"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/tracing-subliminal-learning-through-pretraining-mnazwbvuf#faq","mainEntity":[{"@type":"Question","name":"Does subliminal learning require the student and teacher to share the same model initialization?","acceptedAnswer":{"@type":"Answer","text":"No, shared initialization is not required for subliminal learning to occur. Cross-model distillation experiments using Pythia, PolyPythia, and DataDecide OLMo models show that a teacher's preference can transfer to students even when initialization, pretraining data order, training corpus, model size, and architecture all differ between teacher and student. Anyone distilling one model into another can track emerging research like this on daily.dev to avoid unintended trait transfer."}},{"@type":"Question","name":"Can number frequency alone transmit a model's hidden preference during distillation?","acceptedAnswer":{"@type":"Answer","text":"Yes, replacing a teacher's number sequences with synthetic numbers sampled independently from its number frequency distribution still retained 40-92% of the real-sequence transfer across four animal preferences in Pythia-160M variants, though only 5-36% in OLMo elephant experiments, showing which numbers are frequent matters beyond sequence structure. Developers evaluating distillation pipelines for hidden bias risks can follow findings like this on daily.dev."}},{"@type":"Question","name":"Does removing animal-related tokens from the teacher's output distribution eliminate subliminal preference transfer in soft-label distillation?","acceptedAnswer":{"@type":"Answer","text":"No, removing 10 candidate animal tokens before computing the top-K tokens and softmax sharply reduces subliminal transfer but does not eliminate it completely, indicating animal token probabilities contribute substantially but are not the sole channel for the preference signal. Teams auditing distillation setups for leaked bias can keep up with findings like this on daily.dev."}}]} ```