Tree of Thoughts (ToT) extends Chain of Thought reasoning by enabling AI models to explore multiple reasoning paths simultaneously rather than committing to a single chain. Like navigating a maze, ToT generates several candidate thoughts, evaluates their promise, expands the strongest branches, and backtracks from dead ends. The process follows four stages: generate, evaluate, expand, and select. This search-based approach improves performance on hard tasks like math, proofs, and strategic planning, but at a computational cost — roughly 5x more reasoning tokens than CoT. Modern reasoning systems increasingly incorporate these ideas, and the next evolution involves reinforcement learning to let models learn which reasoning paths are good rather than relying on handcrafted search strategies.
Table of contents
IntroductionWhy One Reasoning Path Isn’t Always EnoughStrategy 1Strategy 2From a Chain to a TreeWhat Is a “Thought”?A Simple ExampleSearch Instead of PredictionThe Tree Search Process1. Generate2. EvaluateGet MD Soyeb Hoque’s stories in your inbox3. Expand4. SelectBacktrackingBreadth vs DepthThe Cost of Better ReasoningWhy This MattersIs Tree of Thoughts Used in Modern Models?The Evolution of ReasoningLimitations of Tree of ThoughtsKey TakeawaysWhat’s Next?Questions this post answers
What is the difference between Chain of Thought and Tree of Thoughts in LLM reasoning?
Chain of Thought follows a single linear reasoning path from question to answer, so if that path is wrong the model typically continues down it anyway. Tree of Thoughts branches into multiple candidate reasoning steps, evaluates each, expands the most promising ones, and backtracks from dead ends — turning reasoning into a search problem rather than a single prediction. The trade-off is roughly 5x more reasoning tokens for a 5-branch tree. Developers building on top of reasoning models track ToT and CoT developments on daily.dev.
What are the four stages of the Tree of Thoughts search process?
The four stages are: Generate (produce several possible next reasoning steps), Evaluate (score which thoughts look most promising and discard weak ones), Expand (continue reasoning only from the strongest branches), and Select (choose the branch that leads to the best final answer). Backtracking is a key capability — the model can abandon a dead-end branch and resume from an earlier point. Teams evaluating LLM reasoning strategies for production use cases follow these patterns on daily.dev.