--- title: GitHub - swellweb/reame: Reame — CPU-first LLM inference... description: Reame is a CPU-first LLM inference server built on llama.cpp, designed for cheap hardware like free-tier VMs and 2-core ARM boxes. Its core thesis is to never... canonical: https://daily.dev/posts/github---swellweb-reame-reame-cpu-first-llm-inference-server-on-llama-cpp-disk-kv-cache-self-re-w9j1c37bo twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: GitHub - swellweb/reame: Reame — CPU-first LLM inference server on llama.cpp: disk KV cache, self-regulating speculation, generation archive, interleaved multi-user, the Conclave. Your hardware, your | daily.dev og:description: Reame is a CPU-first LLM inference server built on llama.cpp, designed for cheap hardware like free-tier VMs and 2-core ARM boxes. Its core thesis is to never... og:url: https://daily.dev/posts/github---swellweb-reame-reame-cpu-first-llm-inference-server-on-llama-cpp-disk-kv-cache-self-re-w9j1c37bo og:image: https://api.daily.dev/og/posts/W9j1C37BO.png og:image:alt: GitHub - swellweb/reame: Reame — CPU-first LLM inference server on llama.cpp: disk KV cache, self-regulating speculation, generation archive, interleaved multi-user, the Conclave. Your hardware, your og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # GitHub - swellweb/reame: Reame — CPU-first LLM inference server on llama.cpp: disk KV cache, self-regulating speculation, generation archive, interleaved multi-user, the Conclave. Your hardware, your **[Hacker News](https://daily.dev/sources/hn)** · 11 min read · 1 upvotes · 0 comments ## Summary Reame is a CPU-first LLM inference server built on llama.cpp, designed for cheap hardware like free-tier VMs and 2-core ARM boxes. Its core thesis is to never compute the same thing twice on a CPU. Key features include a persistent shared-prefix KV cache stored on disk (survives restarts), a generation archive (Palimpsest) that drafts future responses from past outputs, self-regulating speculative decoding that measures its own effectiveness and disables itself when counterproductive, a consensus mechanism (The Conclave) that runs N candidates in one interleaved batch for improved accuracy, and interleaved multi-user serving. It exposes an OpenAI-compatible REST API and ships with 210 test cases. Benchmarks show real speedups: 4.8× end-to-end with warm disk cache, 3.2× with speculative decoding, and 2.3× with archive speculation. The project explicitly positions itself against Ollama, arguing that Reame gets faster the longer it runs while Ollama treats every request as new. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is the Conclave best-of-N feature in Reame and how does it improve LLM answer accuracy without a bigger model? The Conclave is Reame's consensus mechanism invoked with --best-of N, which generates N candidate answers to the same prompt in one interleaved batch by cloning a single prefilled KV cache instead of prefilling N times, then elects a winner by majority vote and stops stragglers once consensus is reached. Measured results show roughly +0.5 to +2 extra correct answers per quiz using a 1.5B model with best-of-5, at about 2.5x wall time rather than 5x, though it never lets a 1.5B model outperform a 3B on hard reasoning. _Teams weighing consensus decoding against bigger models can compare real benchmarks like this on daily.dev._ ### How much speedup does a persistent disk-based KV cache give for repeated LLM prompts on shared CPU hardware? A shared Contabo VPS running TinyLlama 1.1B showed a 4.8x end-to-end speedup comparing a warm disk cache against a cold one, because shared prompt prefixes are snapshotted to disk with zstd compression and checksums and reused across different prompts, restarts, and processes rather than recomputed each time. _Anyone sizing self-hosted inference on cheap CPU boxes can track caching benchmarks like this on daily.dev._ ### Does speculative decoding always speed up LLM inference on shared or oversubscribed CPU servers? No, on heavily oversubscribed shared vCPUs a draft model runs as slowly as its target, making speculative decoding counter-productive rather than helpful. Reame addresses this by measuring acceptance rate and draft economics at runtime and automatically disabling speculation when it stops paying off, rather than assuming speculation always helps on CPU hardware. _Developers tuning speculative decoding on constrained hardware can follow findings like this on daily.dev._ ## Similar posts on daily.dev - [My self-hosted LLMs are a lot more than just a chat replacement – here's how they boost my productivity](https://daily.dev/posts/my-self-hosted-llms-are-a-lot-more-than-just-a-chat-replacement-here-s-how-they-boost-my-productiv-dqzrp8yzq) · XDA Developers · 0 upvotes · 0 comments - [GitHub - lyogavin/airllm: AirLLM 70B inference with single 4GB GPU](https://daily.dev/posts/github---lyogavin-airllm-airllm-70b-inference-with-single-4gb-gpu-s8ukkihfl) · Hacker News · 3 upvotes · 1 comments --- Tags: [#c](https://daily.dev/tags/c), [#computing](https://daily.dev/tags/computing), [#ai-inference](https://daily.dev/tags/ai-inference) [View this post on daily.dev](https://daily.dev/posts/github---swellweb-reame-reame-cpu-first-llm-inference-server-on-llama-cpp-disk-kv-cache-self-re-w9j1c37bo) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"GitHub - swellweb/reame: Reame — CPU-first LLM inference server on llama.cpp: disk KV cache, self-regulating speculation, generation archive, interleaved multi-user, the Conclave. Your hardware, your ","url":"https://daily.dev/posts/github---swellweb-reame-reame-cpu-first-llm-inference-server-on-llama-cpp-disk-kv-cache-self-re-w9j1c37bo","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/github---swellweb-reame-reame-cpu-first-llm-inference-server-on-llama-cpp-disk-kv-cache-self-re-w9j1c37bo"},"datePublished":"2026-07-12T00:07:46.293Z","dateModified":"2026-09-14T07:37:04.734Z","description":"Reame is a CPU-first LLM inference server built on llama.cpp, designed for cheap hardware like free-tier VMs and 2-core ARM boxes. Its core thesis is to never...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3eaa9fddec644bee7c81a91aeb0aaa60?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3eaa9fddec644bee7c81a91aeb0aaa60?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/github---swellweb-reame-reame-cpu-first-llm-inference-server-on-llama-cpp-disk-kv-cache-self-re-w9j1c37bo","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"c,computing,ai-inference","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"GitHub - swellweb/reame: Reame — CPU-first LLM inference server on llama.cpp: disk KV cache, self-regulating speculation, generation archive, interleaved multi-user, the Conclave. Your hardware, your "}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/github---swellweb-reame-reame-cpu-first-llm-inference-server-on-llama-cpp-disk-kv-cache-self-re-w9j1c37bo#faq","mainEntity":[{"@type":"Question","name":"What is the Conclave best-of-N feature in Reame and how does it improve LLM answer accuracy without a bigger model?","acceptedAnswer":{"@type":"Answer","text":"The Conclave is Reame's consensus mechanism invoked with --best-of N, which generates N candidate answers to the same prompt in one interleaved batch by cloning a single prefilled KV cache instead of prefilling N times, then elects a winner by majority vote and stops stragglers once consensus is reached. Measured results show roughly +0.5 to +2 extra correct answers per quiz using a 1.5B model with best-of-5, at about 2.5x wall time rather than 5x, though it never lets a 1.5B model outperform a 3B on hard reasoning. Teams weighing consensus decoding against bigger models can compare real benchmarks like this on daily.dev."}},{"@type":"Question","name":"How much speedup does a persistent disk-based KV cache give for repeated LLM prompts on shared CPU hardware?","acceptedAnswer":{"@type":"Answer","text":"A shared Contabo VPS running TinyLlama 1.1B showed a 4.8x end-to-end speedup comparing a warm disk cache against a cold one, because shared prompt prefixes are snapshotted to disk with zstd compression and checksums and reused across different prompts, restarts, and processes rather than recomputed each time. Anyone sizing self-hosted inference on cheap CPU boxes can track caching benchmarks like this on daily.dev."}},{"@type":"Question","name":"Does speculative decoding always speed up LLM inference on shared or oversubscribed CPU servers?","acceptedAnswer":{"@type":"Answer","text":"No, on heavily oversubscribed shared vCPUs a draft model runs as slowly as its target, making speculative decoding counter-productive rather than helpful. Reame addresses this by measuring acceptance rate and draft economics at runtime and automatically disabling speculation when it stops paying off, rather than assuming speculation always helps on CPU hardware. Developers tuning speculative decoding on constrained hardware can follow findings like this on daily.dev."}}]} ```