--- title: The Entropy of a Markov Chain | daily.dev description: An exploration of how entropy, originally defined by Clausius in thermodynamics, can be applied to Markov chains using Boltzmann's statistical approach.... canonical: https://daily.dev/posts/the-entropy-of-a-markov-chain-mehuhcdab twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: The Entropy of a Markov Chain | daily.dev og:description: An exploration of how entropy, originally defined by Clausius in thermodynamics, can be applied to Markov chains using Boltzmann's statistical approach.... og:url: https://daily.dev/posts/the-entropy-of-a-markov-chain-mehuhcdab og:image: https://api.daily.dev/og/posts/meHuHcdAb.png og:image:alt: The Entropy of a Markov Chain og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The Entropy of a Markov Chain **[Hacker News](https://daily.dev/sources/hn)** · 7 min read · 0 upvotes · 0 comments ## Summary An exploration of how entropy, originally defined by Clausius in thermodynamics, can be applied to Markov chains using Boltzmann's statistical approach. Starting from the classic Boltzmann entropy formula (S = k·ln W), the post works through a concrete example with a magnetic spin system to show how to count microstates. It then extends this counting argument to Dyson's toy model of a cell — a Markov chain with three states (empty, active, inactive) — demonstrating how equilibrium probabilities allow entropy to be computed via the multinomial coefficient. The post promises a follow-up on how entropy evolves based on graph topology. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community take How the wider developer community reacted, aggregated from 2 discussions and 9 comments across hackernews (as of 2026-08-05). **TL;DR:** Commenters engage substantively with the math but find the article underspecified — the actual entropy formula for a Markov chain is never clearly stated, prompting readers to work it out themselves in the thread. **Sentiment:** 30% positive · 50% mixed · 20% skeptical **The case for** - The topic connects thermodynamic entropy to Markov chains in a way that sparks genuine technical curiosity. - The connection to stochastic thermodynamics and eigenvalue analysis is noted as a rich area worth exploring. **The pushback** - The article never explicitly states how to compute the entropy of the example Markov chain, leaving readers to search elsewhere. - A labeling error in the example's edge transitions (np/qp swapped) is flagged. - The dependence between states in a Markov chain complicates naive entropy counting, a subtlety the post doesn't address. - The distinction between entropy rate and equilibrium (thermodynamic) entropy is left unresolved by the article. **By community** - hackernews (mixed): Readers find the topic interesting but are frustrated that the central question — how to actually compute the entropy — is buried or unanswered, and they end up deriving and debating the correct formula themselves in the comments. **Hottest debate:** Whether the natural definition of 'entropy of a Markov chain' should be the entropy rate of the transition matrix or the Shannon entropy of the stationary distribution. **Open questions** - What is the entropy of the specific Markov chain example given in the post? - How is the entropy rate related to the entropy of the stationary distribution? - How does entropy evolution relate to the second eigenvalue of the transition matrix? **Highlights** > So how does one calculate the entropy of a Markov Chain? Is it actually specified? If so, it seems buried. The Markov chain provided as an example has the edge labels swapped (np should be qp and qp should be np). Regardless, what is the entropy of the example provided? The problem with Markov chains is that states are dependent, so simply cataloguing states now violates the basic entropy calculation as neighboring states are now dependent on each other. If the Markov chain is ergodic then maybe you can talk about the entropy of the stationary distribution? Then it's just $\sum p_i lg(p_i)$ of the stationary distribution probabilities? The article alludes to how entropy evolves. In the context of ergodic Markov chains, this is related to the size of the second eigenvalue? > — [abetusk on hackernews · 3 comments](https://news.ycombinator.com/item?id=49187896) > OK, no, I'm wrong. The entropy of a Markov chain with stationary distribution v is [0]: -\sum v_i p_{i,j} \log(p_{i,j}) That is, the "entropy" of the transition matrix modified by the stationary distribution. [0] https://math.stackexchange.com/questions/1040972/entropy-of-... > — [abetusk on hackernews · 1 comments](https://news.ycombinator.com/item?id=49188437) > I'm not sure I had a well formed question. I saw the post title, didn't get a clarification in the article and started searching. The entropy rate seems like a pretty natural definition of "entropy of a Markov chain", no? It's not exactly this but it's similar to "start at state i, end on state j (maybe in n steps?), what is the number of bits I need to send over the wire to tell you what path was taken". What does the entropy of the raw stationary distribution give you? Is the entropy rate related to the entropy of the stationary distribution (the thermodynamic entropy?)? > — [abetusk on hackernews](https://news.ycombinator.com/item?id=49188810) > Find the equilibrium probabilities from first eigenvector, use definition of entropy like you did. If it's not ergodic then you get subsystems with their own entropies. Second eigenvalue iirc basically characterizes dominant relaxation time. This is all if you mean the equilibrium entropy of the underlying system, not the entropy rate. > — [chermi on hackernews](https://news.ycombinator.com/item?id=49188611) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49183017) · 94 points · 9 comments - [hackernews](https://news.ycombinator.com/item?id=49087297) · 1 points · 0 comments ## Similar posts on daily.dev - [Uncovering entanglement entropy near Gross-Neveu criticality by a high-efficiency fermionic quantum Monte Carlo scanning](https://daily.dev/posts/uncovering-entanglement-entropy-near-gross-neveu-criticality-by-a-high-efficiency-fermionic-quantum--vxqw0ptqg) · Nature · 0 upvotes · 0 comments - [How I learned to stop worrying and…no, I’ve always adored entropy](https://daily.dev/posts/how-i-learned-to-stop-worrying-and-no-i-ve-always-adored-entropy-kflv66w90) · Quantum Frontiers · 0 upvotes · 2 comments - [Mathematical proof unites two puzzling phenomena in spin glass physics](https://daily.dev/posts/mathematical-proof-unites-two-puzzling-phenomena-in-spin-glass-physics-dvq0eweet) · Phys.org · 0 upvotes · 0 comments - [Markov Chains: nuclear algorithm used by Quants to crush 87% of Polymarket traders](https://daily.dev/posts/markov-chains-nuclear-algorithm-used-by-quants-to-crush-87-of-polymarket-traders-czdce9mxh) · Coins Bench · 2 upvotes · 1 comments --- Tags: [#statistics](https://daily.dev/tags/statistics) [View this post on daily.dev](https://daily.dev/posts/the-entropy-of-a-markov-chain-mehuhcdab) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"The Entropy of a Markov Chain","url":"https://daily.dev/posts/the-entropy-of-a-markov-chain-mehuhcdab","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/the-entropy-of-a-markov-chain-mehuhcdab"},"datePublished":"2026-08-05T16:09:03.017Z","dateModified":"2026-08-05T22:55:09.043Z","description":"An exploration of how entropy, originally defined by Clausius in thermodynamics, can be applied to Markov chains using Boltzmann's statistical approach....","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/91dbf75df14fd74aa50d5038903b8322?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/91dbf75df14fd74aa50d5038903b8322?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/the-entropy-of-a-markov-chain-mehuhcdab","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"statistics","timeRequired":"PT7M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"The Entropy of a Markov Chain"}]} ```