<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/not-everything-needs-an-llm-zrpcrlmei" -->

---
title: Not Everything Needs an LLM | daily.dev
description: Decision models like Jev offer a cheaper, faster alternative to LLMs for simple yes/no or multiple-choice checks, returning a typed answer with a calibrated...
canonical: https://daily.dev/posts/not-everything-needs-an-llm-zrpcrlmei
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: Not Everything Needs an LLM | daily.dev
og:description: Decision models like Jev offer a cheaper, faster alternative to LLMs for simple yes/no or multiple-choice checks, returning a typed answer with a calibrated...
og:url: https://daily.dev/posts/not-everything-needs-an-llm-zrpcrlmei
og:image: https://api.daily.dev/og/posts/zrpcRlMEi.png
og:image:alt: Not Everything Needs an LLM
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Not Everything Needs an LLM

**[Hacker News](https://daily.dev/sources/hn)** · 8 min read · 0 upvotes · 0 comments

## Summary

Decision models like Jev offer a cheaper, faster alternative to LLMs for simple yes/no or multiple-choice checks, returning a typed answer with a calibrated probability instead of free-form prose. PostHog uses them to gate expensive LLM calls in its Replay Vision PII scrubber, where 98.7% of summaries skip the LLM entirely, with thresholds set at 0.4 for costly false negatives and 0.85+ for a Slack bot avoiding unnecessary replies. They're also used as model routers and to rerank search results, improving ranking quality by about 25% on 20 test queries. Multi-step reasoning flooded the model server and blocked other products' calls for hours, so the source advises leaving that to LLMs, while the inconclusive 0.52 AUC came from Jev ranking PR review findings.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://newsletter.posthog.com/p/decision-models>

## Questions this post answers

### How much cheaper and faster are decision models like Jev compared to frontier LLMs?

Decision models can be up to 444.6 times cheaper and 193.6 times faster than frontier LLMs, according to TypeSafe AI, the maker of the Jev decision model. This speed and cost gap is why PostHog uses them to gate whether an expensive LLM call is even needed, rather than sending every request straight to a model like Gemini.

_Teams weighing LLM costs against latency can track real-world benchmarks like this on daily.dev._

### How do you decide the probability threshold for a decision model check?

Set the threshold based on which type of mistake costs more. PostHog uses a low 0.4 threshold for a PII-detection check because a false negative (storing personal data as clean) is worse than a false positive (an extra LLM call), but uses a high 0.85+ threshold for a Slack bot answering questions, where a false positive (replying unnecessarily) is the costlier mistake.

_Developers tuning AI pipeline thresholds can find practical calibration examples like this on daily.dev._

### Can decision models handle multi-step reasoning tasks?

Not well. PostHog tried chaining a decision model through two sequential questions to rank a 'Highlights' feed of session recordings, sending requests of up to 32 sessions and 12-38k tokens each, which flooded the model server and blocked other products' calls for hours. The conclusion was that multi-step reasoning should be left to LLMs or handled by preprocessing data before it reaches the decision model.

_Anyone designing AI pipelines can weigh trade-offs like this between decision models and LLMs on daily.dev._

## Community take

How the wider developer community reacted, aggregated from 1 discussion and 69 comments across hackernews (as of 2026-11-18).

**TL;DR:** Discussion largely agrees that lightweight decision models are a sensible way to gate expensive LLM calls, but many push back on whether 'decision model' is meaningfully different from an LLM, question the PostHog PII use case, and debate whether people over-reach for LLMs in general.

**Sentiment:** 25% positive · 40% mixed · 35% skeptical

**The case for**

- Decision models give a guaranteed structured output (can't go off the rails) unlike free-form LLMs.
- Several agree many LLM use cases could be replaced with cheaper, faster decision models.
- Gating expensive LLM calls with a cheap pre-check is seen as a sound general pattern.

**The pushback**

- Some argue a 'decision model' is really just an LLM with constrained output, so the framing/naming is misleading or not novel.
- Using this approach to detect PII in customer data is called out as a risky or inappropriate use case.
- Skepticism that decision models used as model routers would lack information about newer models' capabilities.
- Concern that decision model rerankers are a strange tradeoff versus simpler, already-calibrated rerankers.
- Criticism that the 0.85 calibrated probability threshold is treated as precise when its own accuracy is unverified.
- Several comments criticize non-technical PMs/managers for reaching for LLMs to solve trivially simple problems like sorting or JSON mapping.

**By community**

- hackernews (mixed): People broadly endorse cheap decision models for gating LLM calls, but there's notable pushback on terminology, the PII use case, and whether the core idea is even new.

**Hottest debate:** Whether a 'decision model' like Jev is fundamentally different from an LLM or just an LLM with constrained/non-generative output dressed up with new branding.

**Open questions**

- How was the 0.85 calibrated probability threshold itself validated as accurate rather than arbitrary?
- How would a decision model used as a model router account for newer models it has no information about?
- Would a local/on-prem decision model be better suited for PII detection than a hosted one?

**Highlights**

> Sure it's an LLM, it's just neither generative nor instruction-tuned, like BERT (also an LLM), which gave them a reason to market it as not being one. Though modern "LLMs" are also modeling a lot more than just the language so they're kind of hard to define. Not much point in being pedantic, let them call anything anything and see what sticks.
> — [orbital-decay on hackernews](https://news.ycombinator.com/item?id=50539552)

> I am so skeptical about these decision model rerankers. Training a reranker has never been easier, and many of them already return calibrated scores. Decision models are often larger, slower, non colocated… it just seems like a strange tradeoff to make.
> — [nostrebored on hackernews](https://news.ycombinator.com/item?id=50542116)

> > A probability score of 0.85 still means ~150 wrong answers per 1,000 checks, so plan ahead for false positives and false negatives. But that's only true if we believe the value of 0.85 in itself is correct!? For all we know it could be 0.75 or 0.9 no? If were to make calls to something like JEV and know that the probability the answer is correct is, say, 0.85193847098713, wouldn't that be even more amazing than JEV and LLMs themselves!? I'm not being facetious here: how do we know it's 0.85 and neither 0.75 nor 0.9? And if 0.85 is "correct", why stop there? Why not 0.85193847098713? Any explanation is very welcome.
> — [TacticalCoder on hackernews · 1 comments](https://news.ycombinator.com/item?id=50541184)

> The approach of using decision models to gate more expensive work.. I mean, yeah. But using protecting PII as an example use case is a bit bonkers?
> — [Rapzid on hackernews · 1 comments](https://news.ycombinator.com/item?id=50539292)

> Using these decision models as a model router sounds really bad to me the models will most likely be newer than the decision model and thus the decision model will have no information about the capabilities of the model other than the short blurb you feed it.
> — [throwawayffffas on hackernews](https://news.ycombinator.com/item?id=50540197)

**Source threads**

- [hackernews](https://news.ycombinator.com/item?id=50535740) · 144 points · 69 comments

## Similar posts on daily.dev

- [Jev: System One — Best Partner to LLMs](https://daily.dev/posts/jev-system-one-best-partner-to-llms-8kfcdbklg) · Medium · 2 upvotes · 0 comments
- [Bag of Decisions Reranker](https://daily.dev/posts/bag-of-decisions-reranker-rhw6chlak) · Hacker News · 1 upvotes · 0 comments

---

Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#llm](https://daily.dev/tags/llm), [#ai-gateway](https://daily.dev/tags/ai-gateway)

[View this post on daily.dev](https://daily.dev/posts/not-everything-needs-an-llm-zrpcrlmei)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"TechArticle","headline":"Not Everything Needs an LLM","url":"https://daily.dev/posts/not-everything-needs-an-llm-zrpcrlmei","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/not-everything-needs-an-llm-zrpcrlmei"},"datePublished":"2026-11-17T23:36:24.248Z","dateModified":"2026-11-18T10:27:12.258Z","description":"Decision models like Jev offer a cheaper, faster alternative to LLMs for simple yes/no or multiple-choice checks, returning a typed answer with a calibrated...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d02d4609a27386fe48021c4a0555a357?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d02d4609a27386fe48021c4a0555a357?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/not-everything-needs-an-llm-zrpcrlmei","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,llm,ai-gateway","timeRequired":"PT8M"}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Not Everything Needs an LLM"}]}
{"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/not-everything-needs-an-llm-zrpcrlmei#faq","mainEntity":[{"@type":"Question","name":"How much cheaper and faster are decision models like Jev compared to frontier LLMs?","acceptedAnswer":{"@type":"Answer","text":"Decision models can be up to 444.6 times cheaper and 193.6 times faster than frontier LLMs, according to TypeSafe AI, the maker of the Jev decision model. This speed and cost gap is why PostHog uses them to gate whether an expensive LLM call is even needed, rather than sending every request straight to a model like Gemini. Teams weighing LLM costs against latency can track real-world benchmarks like this on daily.dev."}},{"@type":"Question","name":"How do you decide the probability threshold for a decision model check?","acceptedAnswer":{"@type":"Answer","text":"Set the threshold based on which type of mistake costs more. PostHog uses a low 0.4 threshold for a PII-detection check because a false negative (storing personal data as clean) is worse than a false positive (an extra LLM call), but uses a high 0.85+ threshold for a Slack bot answering questions, where a false positive (replying unnecessarily) is the costlier mistake. Developers tuning AI pipeline thresholds can find practical calibration examples like this on daily.dev."}},{"@type":"Question","name":"Can decision models handle multi-step reasoning tasks?","acceptedAnswer":{"@type":"Answer","text":"Not well. PostHog tried chaining a decision model through two sequential questions to rank a 'Highlights' feed of session recordings, sending requests of up to 32 sessions and 12-38k tokens each, which flooded the model server and blocked other products' calls for hours. The conclusion was that multi-step reasoning should be left to LLMs or handled by preprocessing data before it reaches the decision model. Anyone designing AI pipelines can weigh trade-offs like this between decision models and LLMs on daily.dev."}}]}
```

