--- title: "RL Doesn't Work on Slurm" url: https://daily.dev/posts/rl-doesn-t-work-on-slurm-jl6qocokr source_url: https://blog.skypilot.co/rl-doesnt-work-on-slurm type: article source: "SkyPilot" published: 2026-05-21T21:44:02.001Z updated: 2026-05-21T21:44:27.968Z tags: ["kubernetes", "reinforcement-learning", "vllm"] reading_time: 12 upvotes: 0 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # RL Doesn't Work on Slurm **[SkyPilot](https://daily.dev/sources/skypilot)** · 12 min read · 0 upvotes · 0 comments ## Summary Online reinforcement learning for LLMs (GRPO, PPO, RLHF) requires multiple cooperating services — policy/actor, rollout engine, reward model, reference model — that must coordinate on every iteration. Slurm's batch scheduling model fundamentally cannot express this: it lacks service discovery, enforces job time limits, has no component-level health checking, and offers no dynamic scaling. Every major RL framework (OpenRLHF, veRL, NeMo RL, TRL) has independently built the same glue code to bootstrap Ray clusters on top of Slurm, papering over its limitations. Real-world teams including Meta, H Company, and Danijar Hafner's lab have all hit this wall. SkyPilot Job Groups are presented as a solution: a multi-document YAML format where each RL component (trainer, rollout, reward, buffer) is a separate task with its own resource requirements, stable DNS-based service discovery, and independent failure recovery. Slurm remains strong for homogeneous pretraining and SFT batch jobs, but online RL has moved beyond its design space. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [SkyPilot Job Groups: Run RL on Heterogenous Hardware](https://daily.dev/posts/skypilot-job-groups-run-rl-on-heterogenous-hardware-komwla8ap) · SkyPilot · 0 upvotes · 0 comments - [Migrating from Slurm to Kubernetes](https://daily.dev/posts/migrating-from-slurm-to-kubernetes-tgt7479nl) · SkyPilot · 0 upvotes · 0 comments - [Run high-scale RL for LLMs on GKE](https://daily.dev/posts/run-high-scale-rl-for-llms-on-gke-oa4xx0efv) · Google Cloud · 0 upvotes · 0 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes), [#reinforcement-learning](https://daily.dev/tags/reinforcement-learning), [#vllm](https://daily.dev/tags/vllm) [View this post on daily.dev](https://daily.dev/posts/rl-doesn-t-work-on-slurm-jl6qocokr)