<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/perplexity-and-intelligence-2iufjba8x" -->

---
title: Perplexity and Intelligence | daily.dev
description: OpenAI proposes perplexity, or cross-entropy loss, as a shared industry metric for tracking AI progress and setting safety limits. The post publishes OpenAI's...
canonical: https://daily.dev/posts/perplexity-and-intelligence-2iufjba8x
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: Perplexity and Intelligence | daily.dev
og:description: OpenAI proposes perplexity, or cross-entropy loss, as a shared industry metric for tracking AI progress and setting safety limits. The post publishes OpenAI's...
og:url: https://daily.dev/posts/perplexity-and-intelligence-2iufjba8x
og:image: https://api.daily.dev/og/posts/2IUFJBA8x.png
og:image:alt: Perplexity and Intelligence
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Perplexity and Intelligence

**[OpenAI](https://daily.dev/sources/openai)** · 8 min read · 0 upvotes · 0 comments

## Summary

OpenAI proposes perplexity, or cross-entropy loss, as a shared industry metric for tracking AI progress and setting safety limits. The post publishes OpenAI's internal perplexity numbers: GPT-3.5 scored a log-perplexity of about 0.99 and enabled fluent conversation, GPT-4 reached about 0.91 and enabled productive chain-of-thought reasoning, and losses below 0.8 brought transformative capabilities in cybersecurity and math. At log-perplexity ~0.76 the model began producing novel math results. The author expects further transformative capabilities around the 0.68-0.7 log-perplexity range and proposes industry-wide 'common limits' tied to perplexity until better AI monitoring tools exist, with a similar avalanche of pretraining advances conceivably possible as soon as 2027.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://openai.com/index/perplexity-and-intelligence>

## Questions this post answers

### What was GPT-4's log-perplexity and what capability did it unlock?

GPT-4 reached a log-perplexity of about 0.91 on OpenAI's main evaluation dataset. This loss level corresponded to the model becoming proficient at productively utilizing chain-of-thought reasoning, which motivated OpenAI's subsequent development of dedicated reasoning models built around eliciting that latent capability.

_Developers tracking what model upgrades actually unlock can follow these capability thresholds on daily.dev._

### What log-perplexity level did GPT-3.5 reach and what could it do?

GPT-3.5 reached a log-perplexity of approximately 0.99 on OpenAI's current main evaluation dataset. At this level the model became capable of fluent conversation, which formed the technical basis for the original ChatGPT release.

_Anyone comparing model generations for their own projects can track these benchmarks on daily.dev._

### Why is perplexity useful for comparing progress across different AI model developers?

Perplexity measures how well a model predicts the next token on a held-out dataset, with lower perplexity meaning better predictions, and it has remained a consistent metric for tracking pretraining progress over years of model development. It is proposed as a shared benchmark because advances on it correlate with the emergence of new, often unpredictable capabilities, though comparison across labs is complicated by different evaluation datasets and training data mixtures.

_Teams evaluating which AI models to adopt can follow this kind of cross-lab benchmarking discussion on daily.dev._

## Community take

How the wider developer community reacted, aggregated from 1 discussion and 177 comments across hackernews (as of 2026-11-22).

**TL;DR:** Discussion is split between those who find the perplexity-as-progress-metric framing informative and those who view it as self-serving marketing with cherry-picked, causally dubious correlations between a narrow codebase-based metric and broad emergent capabilities.

**Sentiment:** 20% positive · 45% mixed · 35% skeptical

**The case for**

- Some argue cross-entropy/log-loss is a theoretically clean, information-theoretic measure of prediction success that's hard to beat with alternatives.
- Several note the trend does roughly track their own subjective sense of capability jumps (e.g., post-o1, post-December 2025).
- One commenter working at OpenAI defends the metric as empirically useful even if imperfect and not claimed to be perfect.

**The pushback**

- Many argue correlation between perplexity and specific capabilities like cybersecurity or novel math doesn't establish causation.
- Several question why OpenAI measures held-out perplexity on its own internal codebase rather than a broad, representative dataset, making cross-company comparison unreliable.
- Commenters note the metric ignores RL/RLVR and test-time scaling, which many believe drove most recent capability gains rather than pretraining loss alone.
- Some dismiss the post outright as disguised advertising/marketing rather than genuine research insight.
- Critics compare the capability milestones chart to plotting unrelated axes together (e.g., GHz vs. StarCraft wins), calling it pseudo-scientific.

**By community**

- hackernews (heated): Heavy technical back-and-forth between people defending perplexity as a meaningful, informative metric and others calling the post marketing dressed up as science, with no clear majority.

**Hottest debate:** Whether the observed correlation between falling pretraining perplexity and emergent capabilities reflects real causation or is a selectively framed marketing narrative.

**Open questions**

- Would the capability-vs-perplexity relationship hold if measured on a broad, representative dataset instead of OpenAI's internal codebase?
- How much of recent capability gains come from pretraining loss improvements versus RL/RLVR and test-time compute, which aren't captured by this metric?
- Can different companies' perplexity numbers be meaningfully compared given they're measured on different, undisclosed datasets?

**Highlights**

> This is just an ad. Why is it top #1 of the front page? Just some guys on a payroll claiming that an arbitrary set of milestones is a curve towards a greater magic goal that means that some valuation is actually true. Why would you read this
> — [hypfer on hackernews · 2 comments](https://news.ycombinator.com/item?id=50602171)

> I work at OpenAI. The correlation between perplexity and the quality isn’t 1, but neither is it 0. It makes sense that as a model gets better at learning pretraining data, it also gets better at generically learning whatever you’re trying to teach it afterwards. The relationship is imperfect and empirical, and if it breaks down in the future, we’ll of course adapt to different measures instead of dogmatically clinging to the old paradigm. But so far, it’s still informative.
> — [tedsanders on hackernews · 1 comments](https://news.ycombinator.com/item?id=50602183)

> > I assumed that we had moved beyond cross-entropy loss being important with the increasing use of RL From an information theory perspective, cross-entropy loss (log loss) seems to me to be about as pure as it gets in terms of capturing the notion of whether prediction "succeeded". Even Solomonoff Induction minimizes log loss. Would be curious to hear others' perspectives though on why another function might be better for universal prediction purposes.
> — [Xcelerate on hackernews · 1 comments](https://news.ycombinator.com/item?id=50602522)

> I wouldn't say perplexity is great metric because GPT-4 and o3 have similar perplexity on this chart but o3 was a massive step improvement. Pachocki has an incentive to push perplexity as one of the creators of the original LLM paradigm but I would say we have kind of moved on from that.
> — [qphe95 on hackernews · 3 comments](https://news.ycombinator.com/item?id=50602026)

> The chart appears to say that "GPT-6.2 Astra" is the model that had solved the millennium prize problem (Navier-Stokes). I haven't seen this claim elsewhere, it's a big deal. Anyone from OpenAI here, or in the know, who can confirm?
> — [pelican0 on hackernews · 3 comments](https://news.ycombinator.com/item?id=50602738)

**Source threads**

- [hackernews](https://news.ycombinator.com/item?id=50601525) · 224 points · 177 comments

---

Tags: [#llm](https://daily.dev/tags/llm), [#openai](https://daily.dev/tags/openai), [#ai-safety](https://daily.dev/tags/ai-safety), [#perplexity](https://daily.dev/tags/perplexity)

[View this post on daily.dev](https://daily.dev/posts/perplexity-and-intelligence-2iufjba8x)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"TechArticle","headline":"Perplexity and Intelligence","url":"https://daily.dev/posts/perplexity-and-intelligence-2iufjba8x","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/perplexity-and-intelligence-2iufjba8x"},"datePublished":"2026-11-22T17:54:26.717Z","dateModified":"2026-11-22T21:16:06.932Z","description":"OpenAI proposes perplexity, or cross-entropy loss, as a shared industry metric for tracking AI progress and setting safety limits. The post publishes OpenAI's...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b92dbf4eacac3adce92c222ebc216393?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b92dbf4eacac3adce92c222ebc216393?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"OpenAI","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"OpenAI","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/984b3632316f4afaa3d5213d5c64fe65","url":"https://daily.dev/sources/openai"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/perplexity-and-intelligence-2iufjba8x","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,openai,ai-safety,perplexity","timeRequired":"PT8M"}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"OpenAI","item":"https://daily.dev/sources/openai"},{"@type":"ListItem","position":3,"name":"Perplexity and Intelligence"}]}
{"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/perplexity-and-intelligence-2iufjba8x#faq","mainEntity":[{"@type":"Question","name":"What was GPT-4's log-perplexity and what capability did it unlock?","acceptedAnswer":{"@type":"Answer","text":"GPT-4 reached a log-perplexity of about 0.91 on OpenAI's main evaluation dataset. This loss level corresponded to the model becoming proficient at productively utilizing chain-of-thought reasoning, which motivated OpenAI's subsequent development of dedicated reasoning models built around eliciting that latent capability. Developers tracking what model upgrades actually unlock can follow these capability thresholds on daily.dev."}},{"@type":"Question","name":"What log-perplexity level did GPT-3.5 reach and what could it do?","acceptedAnswer":{"@type":"Answer","text":"GPT-3.5 reached a log-perplexity of approximately 0.99 on OpenAI's current main evaluation dataset. At this level the model became capable of fluent conversation, which formed the technical basis for the original ChatGPT release. Anyone comparing model generations for their own projects can track these benchmarks on daily.dev."}},{"@type":"Question","name":"Why is perplexity useful for comparing progress across different AI model developers?","acceptedAnswer":{"@type":"Answer","text":"Perplexity measures how well a model predicts the next token on a held-out dataset, with lower perplexity meaning better predictions, and it has remained a consistent metric for tracking pretraining progress over years of model development. It is proposed as a shared benchmark because advances on it correlate with the emergence of new, often unpredictable capabilities, though comparison across labs is complicated by different evaluation datasets and training data mixtures. Teams evaluating which AI models to adopt can follow this kind of cross-lab benchmarking discussion on daily.dev."}}]}
```

