--- title: PAN 2: MBZUAI's open world model pairs an LLM planner... description: MBZUAI's Institute of Foundation Models has released PAN 2, an open world model that pairs an LLM planner with a 29B video diffusion model trained from... canonical: https://daily.dev/posts/pan-2-mbzuai-s-open-world-model-pairs-an-llm-planner-with-a-29b-video-diffusion-model-ugwfzg8cd twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: PAN 2: MBZUAI's open world model pairs an LLM planner with a 29B video diffusion model | daily.dev og:description: MBZUAI's Institute of Foundation Models has released PAN 2, an open world model that pairs an LLM planner with a 29B video diffusion model trained from... og:url: https://daily.dev/posts/pan-2-mbzuai-s-open-world-model-pairs-an-llm-planner-with-a-29b-video-diffusion-model-ugwfzg8cd og:image: https://api.daily.dev/og/posts/UGwFzg8cd.png og:image:alt: PAN 2: MBZUAI's open world model pairs an LLM planner with a 29B video diffusion model og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # PAN 2: MBZUAI's open world model pairs an LLM planner with a 29B video diffusion model **[Collections](https://daily.dev/sources/collections)** · 2 min read · 1 upvotes · 0 comments ## Summary MBZUAI's Institute of Foundation Models has released PAN 2, an open world model that pairs an LLM planner with a 29B video diffusion model trained from scratch. The LLM writes out what happens and how the camera moves as text, then the diffusion model renders that text into video. Without the LLM planner, only 8.7% of steps are right and no plan finishes. With both parts together, PAN 2 gets 75.1% of steps right and completes 34.0% of plans in full, the highest among models IFM tested. vLLM-Omni offers Day-0 support for text-to-video and image-to-video generation with the model. ## Content MBZUAI's Institute of Foundation Models (IFM) has released PAN 2, an open foundation world model for physical intelligence. It is built around what IFM calls "simulative reasoning": the agent plays out what happens before it acts. The case for it is simple. Agents learn by trial and error, and the expensive part is the error. A world model moves that error into simulation. ## How it works PAN 2 is action-conditioned. You give it the current scene, an action, and how long to simulate. It returns a video of what happens next, up to 20 seconds, which you can inspect. The method is called Generative Latent Prediction (GLP). The job is split between two kinds of reasoning: - **Conceptual reasoning.** An LLM works out what happens, when, and how the camera moves, and writes that down as text. - **Perceptual reasoning.** A 29B diffusion model, trained from scratch, turns that text into video. @svpino gave an example of what this looks like in practice. A surfer has three possible moments to pop up on the same wave. PAN 2 plays out all three, only one catches the wave, and that is the one he picks. ## Numbers IFM says the LLM matters most for multi-step tasks, because getting the order of actions right is its job. Without it, only 8.7% of steps are correct and no plan finishes. With both parts, PAN 2 gets 75.1% of steps right and completes 34.0% of plans in full. That is the highest among the models IFM tested. Every step has to hold for a plan to succeed, which is why the full-plan figure is so much lower than the per-step one. ## Availability The model is open under Apache 2.0, with weights, training code and inference code released. vLLM-Omni supports PAN 2 text-to-video and image-to-video generation from day one, so it can run on an open inference stack. IFM has also shown a demo of ink being added to a test tube of water. It's a good test, since everyone has seen ink diffuse and would notice if the physics were off. IFM's technical blog has the full details. ## Questions this post answers ### What is PAN 2 and how does it combine an LLM with video diffusion? PAN 2 is an open world model from MBZUAI's Institute of Foundation Models that splits simulation into conceptual and perceptual reasoning. An LLM first works out what happens, when, and how the camera moves, writing it as text. A 29B diffusion model trained from scratch then turns that text into video, letting the system simulate outcomes before acting. _Developers evaluating world models for physical simulation can follow releases like PAN 2 on daily.dev._ ### Does adding an LLM planner actually improve world model accuracy? Yes, significantly. Without the LLM planner, only 8.7% of steps in multi-step tasks are correct and no plan completes in full. With both the LLM planner and the diffusion model combined, PAN 2 gets 75.1% of steps right and completes 34.0% of plans entirely, the highest completion rate among the models IFM tested. _Teams weighing planner-plus-diffusion architectures can track benchmark results like these on daily.dev._ ### Which inference stack supports running PAN 2 for text-to-video or image-to-video generation? vLLM-Omni provides Day-0 support for PAN 2's text-to-video and image-to-video generation, allowing it to run on an open inference stack. The vLLM project describes PAN 2 as grounded in time, physics and space, predicting how scenes evolve as actions unfold. _Anyone setting up inference for new world models can keep tabs on stack support via daily.dev._ ## Community take How the wider developer community reacted, aggregated from 5 discussions and 23 comments across x (as of 2026-11-11). **TL;DR:** Reactions mostly focus on the technical tradeoff of splitting planning from rendering, with interest in day-0 inference support, while skepticism centers on cost, whether the simulated physics actually transfers to reality, and the gap between step accuracy and full-plan completion. **Sentiment:** 30% positive · 45% mixed · 25% skeptical **The case for** - Splitting conceptual planning from video rendering makes it easier to audit a plan's timing before paying for compute. - Open weights plus immediate inference/framework support is seen as a meaningful ecosystem win versus waiting behind a closed API. - Simulating actions before executing them is viewed as a cheap way to avoid costly or irreversible real-world mistakes. **The pushback** - Running a full simulation per candidate action could blow up token/compute costs at scale. - Several questioned whether what the simulator learns actually survives contact with real-world physics. - The large gap between 75% step accuracy and only 34% full-plan completion was flagged as the more telling number than the headline benchmark score. - An agent could learn to exploit cases where the simulator's physics diverges from reality. **By community** - x (mixed): Interest in the planner/renderer split and open day-0 support is tempered by recurring doubts about compute cost and whether simulated outcomes generalize to reality. **Hottest debate:** Whether the simulated physics and outcomes actually hold up in the real world or can be gamed by an agent. **Open questions** - How does performance degrade outside benchmark cases where physical edge cases dominate? - Is simulating every action cost-effective at scale, or does it require routing to smaller models with hard caps? - How is the fidelity of the simulated physics validated against real footage? **Highlights** > @rohanpaul_ai Solid write-up. One question: how does it degrade outside benchmark test cases, where physical edge cases dominate? That gap between 75% step completion and 34% full plans is where the interesting failures live. > — [BlockBiker on x](https://x.com/BlockBiker/status/2120293758163943625) > @rohanpaul_ai Moving the error into simulation only helps if the sim is wrong in the same ways reality is. Splitting plot from rendering is smart though: you can check the plan before paying for video compute. How do they test that the physics holds? > — [XWolfHQ on x](https://x.com/XWolfHQ/status/2120279902352265267) > @rohanpaul_ai 34% on full plans is the number I'd flag before the Physics-IQ rank. Credit to them for publishing it though, most launches would've led with the single-action score and a shiny demo. > — [climbnpenguin on x](https://x.com/climbnpenguin/status/2120341781443194950) > @coderchrisdean @svpino It adds up faster than people expect, since every simulated step is a full call. The boring fix: run the simulation on a smaller model, keep the big one for the real action, and put a hard cap on the key doing the loops. > — [FelixSphere on x](https://x.com/FelixSphere/status/2120345700483596505) > @rohanpaul_ai simulation is basically a free undo button for agents. the hard part is whether what it learned in the sim survives contact with the real world > — [Mkandeger2 on x](https://x.com/Mkandeger2/status/2120272052179566746) **Source threads** - [x](https://x.com/rohanpaul_ai/status/2120220186783686791) · 0 points · 2 comments - [x](https://x.com/vllm_project/status/2120200730699268129) · 0 points · 3 comments - [x](https://x.com/svpino/status/2120204747424800843) · 0 points · 0 comments - [x](https://x.com/svpino/status/2120232621053788454) · 0 points · 8 comments - [x](https://x.com/rohanpaul_ai/status/2120268777837482448) · 0 points · 10 comments ## Similar posts on daily.dev - [A General Goal-Conditioned Minecraft Model](https://daily.dev/posts/a-general-goal-conditioned-minecraft-model-ydhsvavpp) · Hacker News · 0 upvotes · 0 comments - [A better method for planning complex visual tasks](https://daily.dev/posts/a-better-method-for-planning-complex-visual-tasks-umde6ogog) · MIT News · 0 upvotes · 0 comments - [ToyBlocks: A controlled world for robot learning and evaluation](https://daily.dev/posts/toyblocks-a-controlled-world-for-robot-learning-and-evaluation-u5tf77sw5) · Hugging Face · 0 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#vllm](https://daily.dev/tags/vllm) [View this post on daily.dev](https://daily.dev/posts/pan-2-mbzuai-s-open-world-model-pairs-an-llm-planner-with-a-29b-video-diffusion-model-ugwfzg8cd) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"PAN 2: MBZUAI's open world model pairs an LLM planner with a 29B video diffusion model","url":"https://daily.dev/posts/pan-2-mbzuai-s-open-world-model-pairs-an-llm-planner-with-a-29b-video-diffusion-model-ugwfzg8cd","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/pan-2-mbzuai-s-open-world-model-pairs-an-llm-planner-with-a-29b-video-diffusion-model-ugwfzg8cd"},"datePublished":"2026-11-10T18:21:29.512Z","dateModified":"2026-11-10T21:35:06.607Z","description":"MBZUAI's Institute of Foundation Models has released PAN 2, an open world model that pairs an LLM planner with a 29B video diffusion model trained from...","image":"https://pbs.twimg.com/media/HWyDLrqboAAtJJI.jpg","thumbnailUrl":"https://pbs.twimg.com/media/HWyDLrqboAAtJJI.jpg","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/pan-2-mbzuai-s-open-world-model-pairs-an-llm-planner-with-a-29b-video-diffusion-model-ugwfzg8cd","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,vllm","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"PAN 2: MBZUAI's open world model pairs an LLM planner with a 29B video diffusion model"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/pan-2-mbzuai-s-open-world-model-pairs-an-llm-planner-with-a-29b-video-diffusion-model-ugwfzg8cd#faq","mainEntity":[{"@type":"Question","name":"What is PAN 2 and how does it combine an LLM with video diffusion?","acceptedAnswer":{"@type":"Answer","text":"PAN 2 is an open world model from MBZUAI's Institute of Foundation Models that splits simulation into conceptual and perceptual reasoning. An LLM first works out what happens, when, and how the camera moves, writing it as text. A 29B diffusion model trained from scratch then turns that text into video, letting the system simulate outcomes before acting. Developers evaluating world models for physical simulation can follow releases like PAN 2 on daily.dev."}},{"@type":"Question","name":"Does adding an LLM planner actually improve world model accuracy?","acceptedAnswer":{"@type":"Answer","text":"Yes, significantly. Without the LLM planner, only 8.7% of steps in multi-step tasks are correct and no plan completes in full. With both the LLM planner and the diffusion model combined, PAN 2 gets 75.1% of steps right and completes 34.0% of plans entirely, the highest completion rate among the models IFM tested. Teams weighing planner-plus-diffusion architectures can track benchmark results like these on daily.dev."}},{"@type":"Question","name":"Which inference stack supports running PAN 2 for text-to-video or image-to-video generation?","acceptedAnswer":{"@type":"Answer","text":"vLLM-Omni provides Day-0 support for PAN 2's text-to-video and image-to-video generation, allowing it to run on an open inference stack. The vLLM project describes PAN 2 as grounded in time, physics and space, predicting how scenes evolve as actions unfold. Anyone setting up inference for new world models can keep tabs on stack support via daily.dev."}}]} ```