--- title: A Tale of Dynamic Programming (2022) | daily.dev description: An in-depth mathematical and computational exploration of dynamic programming, tracing how the Bellman principle of optimality unifies seemingly disparate... canonical: https://daily.dev/posts/a-tale-of-dynamic-programming-2022--demkthpmp twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: A Tale of Dynamic Programming (2022) | daily.dev og:description: An in-depth mathematical and computational exploration of dynamic programming, tracing how the Bellman principle of optimality unifies seemingly disparate... og:url: https://daily.dev/posts/a-tale-of-dynamic-programming-2022--demkthpmp og:image: https://api.daily.dev/og/posts/deMKTHpMP.png og:image:alt: A Tale of Dynamic Programming (2022) og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # A Tale of Dynamic Programming (2022) **[Hacker News](https://daily.dev/sources/hn)** · 41 min read · 0 upvotes · 0 comments ## Summary An in-depth mathematical and computational exploration of dynamic programming, tracing how the Bellman principle of optimality unifies seemingly disparate algorithms: shortest-path search, value iteration, policy iteration, backward induction, and stochastic dynamic programming for Markov Decision Processes. The piece builds up the theory of decision processes and value functions, connects it to fixed-point theory via the Banach Fixed Point Theorem, and provides Julia-like pseudocode implementations of value iteration, in-place value iteration, policy iteration, and backwards induction, including worked examples like shortest path in graphs, the Fibonacci recurrence, and Atari game-playing via reinforcement learning. An appendix proves the Bellman operator is a monotone contraction, guaranteeing existence and uniqueness of the optimal value function. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is the difference between value iteration and policy iteration in dynamic programming? Value iteration repeatedly applies the Bellman operator to a value function until convergence, with policy calculations left implicit until the end, converging linearly. Policy iteration alternates between policy evaluation and policy improvement, always producing a strictly better policy at each step, and is guaranteed to converge to the optimal policy in a finite number of steps since there are only finitely many policies. _daily.dev surfaces deep dives like this for engineers comparing dynamic programming algorithms before choosing one._ ### Why does backward induction converge in a single pass for finite horizon decision processes? Backward induction exploits the acyclic structure of finite horizon problems by traversing states in topological order, from terminal states backward. Since terminal states have zero cost and every other state only transitions to states with a lower topological index, the value at each state is already optimal by the time it's computed, so a single sweep suffices instead of iterating to convergence like standard value iteration. _engineers optimizing acyclic decision problems track algorithmic tricks like this via daily.dev._ ### What guarantees the existence and uniqueness of the optimal value function in a Markov Decision Process? The Banach Fixed Point Theorem guarantees existence and uniqueness whenever the discount factor is strictly less than 1, because the Bellman operator is a contraction with Lipschitz constant equal to that discount factor over the space of bounded continuous value functions. This means iterating the operator from any starting value function converges linearly to a unique optimal value function. _daily.dev helps developers dig into the theory behind reinforcement learning and MDP convergence guarantees._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 14 comments across hackernews (as of 2026-08-13). **TL;DR:** People largely enjoyed the article as a clear, refreshing refresher on dynamic programming and Bellman's principle, though several got tangled up trying to paraphrase the optimality principle in plain English, and one commenter pushed back on a proof step that dropped stated assumptions. **Sentiment:** 55% positive · 35% mixed · 10% skeptical **The case for** - Several found it a well-written, refreshing non-AI technical read that reminded them why they liked the topic. - Some appreciated the historical trivia about why the term 'dynamic programming' was chosen. **The pushback** - Multiple people struggled to translate the formal statement of the optimality principle into intuitive language, going back and forth without landing on a satisfying phrasing. - One commenter argued a proof step was sloppy because it invoked the Bellman operator's fixed point without restating the boundedness assumptions on c, A, and T. **By community** - hackernews (mixed): Enjoyment of the article's clarity and history was mixed with confusion over stating the optimality principle intuitively and a critique of rigor in one proof step. **Hottest debate:** Whether the Bellman optimality principle can be restated simply without losing its actual content, versus just sounding circular. **Open questions** - Is there a clearer, non-circular way to phrase the principle of optimality for newcomers? - What are concrete practical (non-academic) applications of Banach fixed point theory in this context? **Highlights** > There's got to be a better way of saying this. "A policy is good if it keeps being good after the first time you use it" is the gist that I'm getting, but that's way too simplistic to actually tell you any of the value of dynamic programming. > — [jazzypants on hackernews](https://news.ycombinator.com/item?id=49283955) > >Proof: It all follows from applying the Banach fixed point theorem to the Bellman operator. I disagree. When people hate mathematicians, it's because of things like this. Why? Because it doesn't follow exclusively from the things listed. It also follows from the fact that c, T, A are bounded by the problem definition. Variable names in math confer no meaning, you can choose any variable name, hence you could have chosen an optimization problem that has no unique solution, simply out of spite. Now the counter to that is that the introduction covered the restrictions on c, A and T but why drop them in the proof? > — [imtringued on hackernews](https://news.ycombinator.com/item?id=49282730) > [0]: > ... I thought dynamic programming was a good name. It was something not even a Congressman could object to. So I used it as an umbrella for my activities. [0] https://en.wikipedia.org/wiki/Dynamic_programming#History_of... > — [abetusk on hackernews](https://news.ycombinator.com/item?id=49281534) > Agreed, very cool article. > I always struggled to understand practical applications. Me too...and specifically with non-smooth dynamical systems. I always felt some better knowledge or intuition on my part could lead to understanding the practical application. It's one reason I left academia... wtf this is cool, but wtf is this useful for? The closest practical application I can fathom in this theoretical realm is optimization. And welp, since that underlies all AI, that's pretty important. Oh and cryptography. > — [kooi on hackernews](https://news.ycombinator.com/item?id=49279718) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49232312) · 81 points · 14 comments ## Similar posts on daily.dev - [Bellman Equations and Dynamic Programming in RL](https://daily.dev/posts/bellman-equations-and-dynamic-programming-in-rl-vuacrm8zo) · Daily Dose of Data Science \| Avi Chawla \| Substack · 3 upvotes · 0 comments - [Bellman Equation: A Comprehensive Guide](https://daily.dev/posts/bellman-equation-a-comprehensive-guide-nfsuexrh9) · Medium · 0 upvotes · 0 comments - [Hamilton-Jacobi-Bellman Equation: Reinforcement Learning and Diffusion Models](https://daily.dev/posts/hamilton-jacobi-bellman-equation-reinforcement-learning-and-diffusion-models-um6oohsnn) · Hacker News · 0 upvotes · 0 comments - [Learn Dynamic Programming Through Dynamic Visuals](https://daily.dev/posts/learn-dynamic-programming-through-dynamic-visuals-rgocrr6yk) · freeCodeCamp · 307 upvotes · 1 comments --- Tags: [#algorithms](https://daily.dev/tags/algorithms), [#reinforcement-learning](https://daily.dev/tags/reinforcement-learning), [#julia](https://daily.dev/tags/julia), [#dynamic-programming](https://daily.dev/tags/dynamic-programming) [View this post on daily.dev](https://daily.dev/posts/a-tale-of-dynamic-programming-2022--demkthpmp) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"A Tale of Dynamic Programming (2022)","url":"https://daily.dev/posts/a-tale-of-dynamic-programming-2022--demkthpmp","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/a-tale-of-dynamic-programming-2022--demkthpmp"},"datePublished":"2026-08-13T00:35:20.862Z","dateModified":"2026-08-13T12:06:05.787Z","description":"An in-depth mathematical and computational exploration of dynamic programming, tracing how the Bellman principle of optimality unifies seemingly disparate...","image":"https://media.daily.dev/image/upload/s--qPvKM23u--/f_auto/v1722860399/public/Placeholder%2009","thumbnailUrl":"https://media.daily.dev/image/upload/s--qPvKM23u--/f_auto/v1722860399/public/Placeholder%2009","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/a-tale-of-dynamic-programming-2022--demkthpmp","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"algorithms,reinforcement-learning,julia,dynamic-programming","timeRequired":"PT41M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"A Tale of Dynamic Programming (2022)"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/a-tale-of-dynamic-programming-2022--demkthpmp#faq","mainEntity":[{"@type":"Question","name":"What is the difference between value iteration and policy iteration in dynamic programming?","acceptedAnswer":{"@type":"Answer","text":"Value iteration repeatedly applies the Bellman operator to a value function until convergence, with policy calculations left implicit until the end, converging linearly. Policy iteration alternates between policy evaluation and policy improvement, always producing a strictly better policy at each step, and is guaranteed to converge to the optimal policy in a finite number of steps since there are only finitely many policies. daily.dev surfaces deep dives like this for engineers comparing dynamic programming algorithms before choosing one."}},{"@type":"Question","name":"Why does backward induction converge in a single pass for finite horizon decision processes?","acceptedAnswer":{"@type":"Answer","text":"Backward induction exploits the acyclic structure of finite horizon problems by traversing states in topological order, from terminal states backward. Since terminal states have zero cost and every other state only transitions to states with a lower topological index, the value at each state is already optimal by the time it's computed, so a single sweep suffices instead of iterating to convergence like standard value iteration. engineers optimizing acyclic decision problems track algorithmic tricks like this via daily.dev."}},{"@type":"Question","name":"What guarantees the existence and uniqueness of the optimal value function in a Markov Decision Process?","acceptedAnswer":{"@type":"Answer","text":"The Banach Fixed Point Theorem guarantees existence and uniqueness whenever the discount factor is strictly less than 1, because the Bellman operator is a contraction with Lipschitz constant equal to that discount factor over the space of bounded continuous value functions. This means iterating the operator from any starting value function converges linearly to a unique optimal value function. daily.dev helps developers dig into the theory behind reinforcement learning and MDP convergence guarantees."}}]} ```