--- title: Scaling Compute on Context — Jack Morris, Engram | daily.dev description: A researcher from the startup Engram frames a research problem he calls 'scaling compute on context' — how to give a pre-trained language model deep,... canonical: https://daily.dev/posts/scaling-compute-on-context-jack-morris-engram-vp68wy9vk twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Scaling Compute on Context — Jack Morris, Engram | daily.dev og:description: A researcher from the startup Engram frames a research problem he calls 'scaling compute on context' — how to give a pre-trained language model deep,... og:url: https://daily.dev/posts/scaling-compute-on-context-jack-morris-engram-vp68wy9vk og:image: https://api.daily.dev/og/posts/VP68Wy9vK.png og:image:alt: Scaling Compute on Context — Jack Morris, Engram og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Scaling Compute on Context — Jack Morris, Engram **[AI Engineer](https://daily.dev/sources/aidotengineer)** · 19 min read · 0 upvotes · 0 comments ## Summary A researcher from the startup Engram frames a research problem he calls 'scaling compute on context' — how to give a pre-trained language model deep, personalized knowledge of a fixed private dataset (emails, meeting transcripts, company data) rather than just public information. He walks through failed and partial approaches: naive next-token fine-tuning (collapses generation quality), KV-cache compaction, on-policy distillation and self-study (generating Q&A pairs from data), synthetic-data continued pre-training, and unsupervised RL environments. Each hits a plateau because the data set is fixed and finite. The proposed missing ingredient is self-improvement, similar to AlphaGo, where the model generates progressively harder training signal from its own outputs to keep deepening its internal model of the data over time, avoiding the plateau seen in early experiments. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why doesn't fine-tuning an LLM directly on a company's private documents work well for teaching it that data? Training a model directly on a fixed private dataset via next-token prediction until loss is near zero causes the model's generation quality to collapse. The model memorizes the data perfectly but loses normal generalization properties, so it can only answer questions that are verbatim encoded in the training data with their answers, which rarely matches real usage. _Teams weighing fine-tuning versus other personalization approaches for private data can track emerging techniques like this via daily.dev._ ### What is on-policy distillation and how is it used to teach a model new private knowledge? On-policy distillation is a technique where text is shown to a model and the model is trained to behave as though that text were present in its context window, effectively simulating in-context learning through gradient updates. The model is updated throughout training (the 'on-policy' part). A related method, self-study from the cartridges approach, generates question-answer pairs conditioned on the private data and trains the model to answer as if it had seen that data in context. _Anyone comparing distillation-based approaches to continual learning can follow this line of research through daily.dev._ ### Why do synthetic-data approaches to teaching an LLM new private knowledge eventually plateau? Any method that crafts a fixed synthetic dataset from private data and trains on it hits an upper bound once the model has learned everything in that synthetic set, producing a plateauing training curve similar to hitting a data wall. The unsolved piece is generating progressively harder training signal as the model improves, mirroring how AlphaGo made its own training problems harder through self-play, so the model keeps deepening its knowledge instead of saturating. _Researchers tackling this self-improvement plateau in continual learning can keep up with new approaches via daily.dev._ ## Similar posts on daily.dev - [A smarter way for large language models to think about hard problems](https://daily.dev/posts/a-smarter-way-for-large-language-models-to-think-about-hard-problems-d6aqwfq0t) · MIT News · 1 upvotes · 0 comments - [Ilya Sutskever, Yann LeCun and the End of “Just Add GPUs” — abZ Global](https://daily.dev/posts/ilya-sutskever-yann-lecun-and-the-end-of-just-add-gpus-abz-global-h14kbzejk) · Hacker News · 2 upvotes · 0 comments - [Scaling Laws, Carefully](https://daily.dev/posts/scaling-laws-carefully-wt9mkgtn8) · Lil’Log · 1 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#deep-learning](https://daily.dev/tags/deep-learning), [#reinforcement-learning](https://daily.dev/tags/reinforcement-learning) [View this post on daily.dev](https://daily.dev/posts/scaling-compute-on-context-jack-morris-engram-vp68wy9vk) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Scaling Compute on Context — Jack Morris, Engram","url":"https://daily.dev/posts/scaling-compute-on-context-jack-morris-engram-vp68wy9vk","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/scaling-compute-on-context-jack-morris-engram-vp68wy9vk"},"datePublished":"2026-08-12T15:54:21.098Z","dateModified":"2026-09-14T07:54:22.973Z","description":"A researcher from the startup Engram frames a research problem he calls 'scaling compute on context' — how to give a pre-trained language model deep,...","image":"https://i.ytimg.com/vi/WiqDvX6isc4/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/WiqDvX6isc4/sddefault.jpg","isAccessibleForFree":true,"articleSection":"AI Engineer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"AI Engineer","logo":"https://media.daily.dev/image/upload/s--u5PucxNT--/f_auto/v1724338940/logos/aidotengineer","url":"https://daily.dev/sources/aidotengineer"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/scaling-compute-on-context-jack-morris-engram-vp68wy9vk","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,deep-learning,reinforcement-learning","timeRequired":"PT19M","video":{"@type":"VideoObject","name":"Scaling Compute on Context — Jack Morris, Engram","description":"A researcher from the startup Engram frames a research problem he calls 'scaling compute on context' — how to give a pre-trained language model deep,...","thumbnailUrl":"https://i.ytimg.com/vi/WiqDvX6isc4/sddefault.jpg","uploadDate":"2026-08-12T15:54:21.098Z","duration":"PT19M","url":"https://api.daily.dev/r/VP68Wy9vK","embedUrl":"https://www.youtube.com/embed/WiqDvX6isc4"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"AI Engineer","item":"https://daily.dev/sources/aidotengineer"},{"@type":"ListItem","position":3,"name":"Scaling Compute on Context — Jack Morris, Engram"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/scaling-compute-on-context-jack-morris-engram-vp68wy9vk#faq","mainEntity":[{"@type":"Question","name":"Why doesn't fine-tuning an LLM directly on a company's private documents work well for teaching it that data?","acceptedAnswer":{"@type":"Answer","text":"Training a model directly on a fixed private dataset via next-token prediction until loss is near zero causes the model's generation quality to collapse. The model memorizes the data perfectly but loses normal generalization properties, so it can only answer questions that are verbatim encoded in the training data with their answers, which rarely matches real usage. Teams weighing fine-tuning versus other personalization approaches for private data can track emerging techniques like this via daily.dev."}},{"@type":"Question","name":"What is on-policy distillation and how is it used to teach a model new private knowledge?","acceptedAnswer":{"@type":"Answer","text":"On-policy distillation is a technique where text is shown to a model and the model is trained to behave as though that text were present in its context window, effectively simulating in-context learning through gradient updates. The model is updated throughout training (the 'on-policy' part). A related method, self-study from the cartridges approach, generates question-answer pairs conditioned on the private data and trains the model to answer as if it had seen that data in context. Anyone comparing distillation-based approaches to continual learning can follow this line of research through daily.dev."}},{"@type":"Question","name":"Why do synthetic-data approaches to teaching an LLM new private knowledge eventually plateau?","acceptedAnswer":{"@type":"Answer","text":"Any method that crafts a fixed synthetic dataset from private data and trains on it hits an upper bound once the model has learned everything in that synthetic set, producing a plateauing training curve similar to hitting a data wall. The unsolved piece is generating progressively harder training signal as the model improves, mirroring how AlphaGo made its own training problems harder through self-play, so the model keeps deepening its knowledge instead of saturating. Researchers tackling this self-improvement plateau in continual learning can keep up with new approaches via daily.dev."}}]} ```