--- title: tokens too cheap to meter | daily.dev description: AI token costs are dropping by roughly 2.5 orders of magnitude per year, driven by GPU efficiency doubling every two years, model cost-per-task falling due to... canonical: https://daily.dev/posts/tokens-too-cheap-to-meter-wekdxgrid twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: tokens too cheap to meter | daily.dev og:description: AI token costs are dropping by roughly 2.5 orders of magnitude per year, driven by GPU efficiency doubling every two years, model cost-per-task falling due to... og:url: https://daily.dev/posts/tokens-too-cheap-to-meter-wekdxgrid og:image: https://api.daily.dev/og/posts/WEkdxgRId.png og:image:alt: tokens too cheap to meter og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # tokens too cheap to meter **[Lobsters](https://daily.dev/sources/lobsters)** · 16 min read · 0 upvotes · 0 comments ## Summary AI token costs are dropping by roughly 2.5 orders of magnitude per year, driven by GPU efficiency doubling every two years, model cost-per-task falling due to better training and Mixture-of-Experts architectures, inference engine gains from vLLM and NVIDIA/Intel MLPerf improvements, and Mamba-hybrid architectures cutting RAM needs 5x for local models. Specialized non-generative classifiers like TypeSafe AI's Jev push cost down another 1-2 orders of magnitude, enabling tools like jgrep that embed cheap AI calls into pipelines. The piece argues that once tokens become cheaper than tool calls (comparing costs to grep, HTML parsing, and cargo build), models will get embedded directly into infrastructure, while OpenAI and Anthropic stay ahead on quality even as commodity volume gets cheap, and speculates on downstream effects for cybersecurity, compute providers, software moats, and job markets. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much cheaper did AI model inference get per task between 2025 and 2026 according to cost-vs-intelligence benchmarks? Cost per task got roughly two orders of magnitude cheaper. Comparing the intelligence-versus-cost-per-task pareto frontier from artificialanalysis.ai across 2025 and 2026, the quality axis stayed roughly similar while the cost axis dropped about 100x, even as models continued getting smarter on top of that. _Anyone budgeting for LLM usage can track pricing shifts like this through daily.dev before locking in a vendor._ ### What is TypeSafe AI's Jev model and how is it priced compared to typical LLMs? Jev is a non-generative classifier model from TypeSafe AI that only outputs a probability between 0 and 100% rather than free text, launched as their flagship System One product. It prices input tokens at $0.042 per million tokens (versus $0.20-$10 for typical LLMs) and output tokens are free, making it roughly two orders of magnitude cheaper than generative models for yes/no style tasks. _Developers weighing classifier models against general LLMs for cheap filtering tasks can follow pricing news like this on daily.dev._ ### How much less RAM does a Mamba-hybrid model like Nemotron-H-47B need compared to a Transformer model of similar quality? Nemotron-H-47B can hold over a million tokens in 32 GB of VRAM when quantized to 4-bit weights, while a comparable-quality Llama-3.1 60B transformer model needs almost 120 GB for the same token count, roughly a 4x difference in RAM requirements. This comes from Mamba's lossy-summary memory approach versus transformers remembering every input in full. _Engineers sizing local LLM hardware can compare architecture tradeoffs like this via daily.dev before buying GPUs._ ## Community take How the wider developer community reacted, aggregated from 3 discussions and 13 comments across lobsters, hackernews (as of 2026-09-23). **TL;DR:** Discussion centers heavily on skepticism about whether token cost declines are real or subsidized, with some evidence offered (open-weight self-hosting, third-party inference providers, benchmark data) but no consensus reached; a side debate questions why tool execution costs wouldn't fall just as fast as inference costs. **Sentiment:** 15% positive · 35% mixed · 50% skeptical **The case for** - Open-weight models can now be self-hosted, letting people calculate real unsubsidized token costs themselves. - Third-party inference providers serving open-source models without huge investment appear to be profitable, suggesting real cost declines. - Benchmark data showing cost-per-intelligence dropping aligns with some users' own experience. - Hardware is simply getting more efficient per watt over time, which is unsurprising and well-documented. **The pushback** - It's unclear whether the price per token is genuinely falling or just being subsidized by AI labs racing to displace human labor. - We have little real insight into what inference actually costs frontier labs like OpenAI and Anthropic. - Skepticism that cost-per-intelligence benchmarks reflect true economics rather than marketing claims. - Doubt that a tool's own execution cost wouldn't improve on a similar efficiency curve, undercutting the premise that tokens will become cheaper than tool calls. **By community** - lobsters (mixed): A long thread debates whether falling token prices are genuine efficiency gains or lab subsidies, with some pointing to self-hosting and third-party providers as evidence, while others remain unconvinced; a separate thread questions the premise that tools won't get cheaper too. - hackernews (mixed): No comments were available to characterize the discussion. **Hottest debate:** Whether the touted drop in token cost-per-intelligence reflects real economics or is being masked by lab subsidies. **Open questions** - Is inference pricing from major labs actually profitable or subsidized relative to true compute cost? - Will tool execution costs (e.g., grep, parsers) fall at a similar rate to inference costs, undermining the premise that models will out-cheapen tools? **Highlights** > This isn't what "subsidized" means. I think we can make some useful analysis about how much inference actually costs some providers, but I *don't* think it's unreasonable to say that we have very little idea how much inference costs OpenAI and Anthropic. It's entirely possible that they are asking customers for less than it costs them to serve those customers; indeed, since their goal is literally displacing all human knowledge workers, as they have stated, it would be kind of irrational _not_ to do this. > — [mtset on lobsters · 1 points](https://lobste.rs/s/n4vfwm/tokens_too_cheap_meter#c_gdhvps) > The author gives an example of using [jgrep](https://github.com/keltokhy/jgrep) rather than grep. If I wanted to search a corpus of text for references to sphynx cats, I (or an LLM) could write a regex to search that corpus with grep. But if I want it to also catch references like "sphinx cats", "them bald cats", or "那种没有毛的猫的品种", then I'm suddenly less confident in this regex. Maybe I could search the corpus with a tool that uses a parser? Even if I can get it right, that's a lot of parsing! And it's spending as much compute parsing a book about breeds of cats as it as it is books about maths. Or I could use jgrep, and it's efficiency in part comes from knowing that it needn't focus too hard in documents that are very unlikely to have references to sphynx cats, like physics papers, or lobster.rs comments. Of course, it'll miss some, but maybe that's okay for the usecase (and maybe it'll miss fewer than the regex or the parser, and spend less compute and energy too). > — [tuxes on lobsters · 4 points](https://lobste.rs/s/n4vfwm/tokens_too_cheap_meter#c_uvg6ux) > > once models are cheaper than tools Why wouldn’t a tool’s execution get cheaper on a similar curve to inference costs? > — [mysteriouspants on lobsters · 4 points, 1 comments](https://lobste.rs/s/n4vfwm/tokens_too_cheap_meter#c_dqoovq) > I guess my real question is how do we know the cost per intelligence isn't being heavily subsidized by the labs? > — [roflmaoqwerty on lobsters · 3 points, 5 comments](https://lobste.rs/s/n4vfwm/tokens_too_cheap_meter#c_ynluyw) **Source threads** - [lobsters](https://lobste.rs/s/n4vfwm/tokens_too_cheap_meter) · 26 points · 13 comments - [hackernews](https://news.ycombinator.com/item?id=49803198) · 5 points · 0 comments - [hackernews](https://news.ycombinator.com/item?id=49810244) · 4 points · 0 comments ## Similar posts on daily.dev - [AI token prices fell 98% but enterprise bills tripled](https://daily.dev/posts/ai-token-prices-fell-98-but-enterprise-bills-tripled-ebg7lcnvk) · The Next Web · 0 upvotes · 0 comments --- Tags: [#local-ai](https://daily.dev/tags/local-ai), [#vllm](https://daily.dev/tags/vllm), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts) [View this post on daily.dev](https://daily.dev/posts/tokens-too-cheap-to-meter-wekdxgrid) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"tokens too cheap to meter","url":"https://daily.dev/posts/tokens-too-cheap-to-meter-wekdxgrid","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/tokens-too-cheap-to-meter-wekdxgrid"},"datePublished":"2026-09-23T08:02:18.099Z","dateModified":"2026-09-23T16:03:06.922Z","description":"AI token costs are dropping by roughly 2.5 orders of magnitude per year, driven by GPU efficiency doubling every two years, model cost-per-task falling due to...","image":"https://media.daily.dev/image/upload/s--1KxV4ohY--/f_auto/v1722860400/public/Placeholder%2007","thumbnailUrl":"https://media.daily.dev/image/upload/s--1KxV4ohY--/f_auto/v1722860400/public/Placeholder%2007","isAccessibleForFree":true,"articleSection":"Lobsters","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Lobsters","logo":"https://media.daily.dev/image/upload/s--tl8v_Fku--/f_auto,t_logo/v1698841318/logos/lobste.jpg","url":"https://daily.dev/sources/lobsters"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/tokens-too-cheap-to-meter-wekdxgrid","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"local-ai,vllm,mixture-of-experts","timeRequired":"PT16M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Lobsters","item":"https://daily.dev/sources/lobsters"},{"@type":"ListItem","position":3,"name":"tokens too cheap to meter"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/tokens-too-cheap-to-meter-wekdxgrid#faq","mainEntity":[{"@type":"Question","name":"How much cheaper did AI model inference get per task between 2025 and 2026 according to cost-vs-intelligence benchmarks?","acceptedAnswer":{"@type":"Answer","text":"Cost per task got roughly two orders of magnitude cheaper. Comparing the intelligence-versus-cost-per-task pareto frontier from artificialanalysis.ai across 2025 and 2026, the quality axis stayed roughly similar while the cost axis dropped about 100x, even as models continued getting smarter on top of that. Anyone budgeting for LLM usage can track pricing shifts like this through daily.dev before locking in a vendor."}},{"@type":"Question","name":"What is TypeSafe AI's Jev model and how is it priced compared to typical LLMs?","acceptedAnswer":{"@type":"Answer","text":"Jev is a non-generative classifier model from TypeSafe AI that only outputs a probability between 0 and 100% rather than free text, launched as their flagship System One product. It prices input tokens at $0.042 per million tokens (versus $0.20-$10 for typical LLMs) and output tokens are free, making it roughly two orders of magnitude cheaper than generative models for yes/no style tasks. Developers weighing classifier models against general LLMs for cheap filtering tasks can follow pricing news like this on daily.dev."}},{"@type":"Question","name":"How much less RAM does a Mamba-hybrid model like Nemotron-H-47B need compared to a Transformer model of similar quality?","acceptedAnswer":{"@type":"Answer","text":"Nemotron-H-47B can hold over a million tokens in 32 GB of VRAM when quantized to 4-bit weights, while a comparable-quality Llama-3.1 60B transformer model needs almost 120 GB for the same token count, roughly a 4x difference in RAM requirements. This comes from Mamba's lossy-summary memory approach versus transformers remembering every input in full. Engineers sizing local LLM hardware can compare architecture tradeoffs like this via daily.dev before buying GPUs."}}]} ```