--- title: Introducing co-operative time-slicing for RL in llm-d description: Co-operative time-slicing in the llm-d project addresses the structural GPU idle-time problem in reinforcement learning post-training for LLMs. By treating... canonical: https://daily.dev/posts/introducing-co-operative-time-slicing-for-rl-in-llm-d-y3fjprmne twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Introducing co-operative time-slicing for RL in llm-d | daily.dev og:description: Co-operative time-slicing in the llm-d project addresses the structural GPU idle-time problem in reinforcement learning post-training for LLMs. By treating... og:url: https://daily.dev/posts/introducing-co-operative-time-slicing-for-rl-in-llm-d-y3fjprmne og:image: https://api.daily.dev/og/posts/y3FjPRMNe.png og:image:alt: Introducing co-operative time-slicing for RL in llm-d og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Introducing co-operative time-slicing for RL in llm-d **[Google Cloud](https://daily.dev/sources/gcp)** · 9 min read · 2 upvotes · 0 comments ## Summary Co-operative time-slicing in the llm-d project addresses the structural GPU idle-time problem in reinforcement learning post-training for LLMs. By treating discrete RL phases (sampling rollouts and gradient training) as schedulable entities, the system interleaves independent RL jobs onto shared physical hardware via checkpoint/restore swaps. The architecture has three layers: a workload-scoped client library exposing acquire()/yield() gRPC APIs, a cluster-scoped orchestrator managing a lock queue, and a node-scoped snapshot agent (DaemonSet) that serializes GPU state to host DRAM. Initial benchmarks show aggregate accelerator duty cycles improving from ~40% to ~70% without impacting model convergence. The full stack — Snapshot Agent, Accelerator Orchestrator, and Python client libraries — is being released today with roadmap items including faster checkpoint backends, automated scheduling, and TPU support. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is co-operative time-slicing in llm-d and how does it improve GPU utilization for RL training? Co-operative time-slicing is a feature in the llm-d project that treats discrete reinforcement learning steps, like sampling rollouts and gradient training, as schedulable entities so independent RL jobs can be interleaved onto shared physical accelerators. Initial benchmarks show it raises aggregate accelerator duty cycles from a roughly 40% baseline up to 70%, without affecting model convergence or accuracy. _Teams fighting idle GPU time during RL post-training can track infrastructure releases like this one on daily.dev._ ### How does the time-slicing swap between two RL jobs work without causing out-of-memory errors on the GPU? A swap works as a checkpoint/restore: when one job yields, its entire device state is checkpointed out of accelerator memory into host DRAM, and the next job's previously saved state is restored in its place. Since only one job's state occupies the accelerator at any time, steps alternate safely without framework-level interference or OOM faults; the first backend implementing this is cuda-checkpoint. _Engineers debugging GPU memory contention during multi-job scheduling can follow implementation details like this via daily.dev._ ### What APIs does a developer use to integrate their RL training loop with the llm-d time-slice orchestrator? Developers wrap accelerator-touching phases with two gRPC calls exposed by a time-slice client library: acquire() to request exclusive accelerator access and yield() to release it. The underlying ML framework, such as PyTorch FSDP or vLLM, along with the CUDA context and memory allocations, runs unmodified, so integration mainly involves annotating training and sampling phases with these calls. _Developers wiring RL pipelines into new scheduling APIs often compare integration approaches surfaced on daily.dev._ ## Similar posts on daily.dev - [Keep the Tokens Flowing: Lessons from 16 Open-Source RL Libraries](https://daily.dev/posts/keep-the-tokens-flowing-lessons-from-16-open-source-rl-libraries-dsuwyaezv) · Hugging Face · 1 upvotes · 0 comments - [GPU Time-Slicing for Concurrent LLM Agents on Kubernetes](https://daily.dev/posts/gpu-time-slicing-for-concurrent-llm-agents-on-kubernetes-guzcjf1ji) · Towards Data Science · 0 upvotes · 0 comments - [Reinforcement Learning is an Infrastructure Problem](https://daily.dev/posts/reinforcement-learning-is-an-infrastructure-problem-j5npoud2t) · Modal · 0 upvotes · 0 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes), [#reinforcement-learning](https://daily.dev/tags/reinforcement-learning) [View this post on daily.dev](https://daily.dev/posts/introducing-co-operative-time-slicing-for-rl-in-llm-d-y3fjprmne) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Introducing co-operative time-slicing for RL in llm-d","url":"https://daily.dev/posts/introducing-co-operative-time-slicing-for-rl-in-llm-d-y3fjprmne","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/introducing-co-operative-time-slicing-for-rl-in-llm-d-y3fjprmne"},"datePublished":"2026-07-23T17:18:31.888Z","dateModified":"2026-09-14T06:14:25.358Z","description":"Co-operative time-slicing in the llm-d project addresses the structural GPU idle-time problem in reinforcement learning post-training for LLMs. By treating...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/71a924f2b121958fb9f73a189cb1663d?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/71a924f2b121958fb9f73a189cb1663d?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Google Cloud","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Google Cloud","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/gcp","url":"https://daily.dev/sources/gcp"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/introducing-co-operative-time-slicing-for-rl-in-llm-d-y3fjprmne","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"kubernetes,reinforcement-learning","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Google Cloud","item":"https://daily.dev/sources/gcp"},{"@type":"ListItem","position":3,"name":"Introducing co-operative time-slicing for RL in llm-d"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/introducing-co-operative-time-slicing-for-rl-in-llm-d-y3fjprmne#faq","mainEntity":[{"@type":"Question","name":"What is co-operative time-slicing in llm-d and how does it improve GPU utilization for RL training?","acceptedAnswer":{"@type":"Answer","text":"Co-operative time-slicing is a feature in the llm-d project that treats discrete reinforcement learning steps, like sampling rollouts and gradient training, as schedulable entities so independent RL jobs can be interleaved onto shared physical accelerators. Initial benchmarks show it raises aggregate accelerator duty cycles from a roughly 40% baseline up to 70%, without affecting model convergence or accuracy. Teams fighting idle GPU time during RL post-training can track infrastructure releases like this one on daily.dev."}},{"@type":"Question","name":"How does the time-slicing swap between two RL jobs work without causing out-of-memory errors on the GPU?","acceptedAnswer":{"@type":"Answer","text":"A swap works as a checkpoint/restore: when one job yields, its entire device state is checkpointed out of accelerator memory into host DRAM, and the next job's previously saved state is restored in its place. Since only one job's state occupies the accelerator at any time, steps alternate safely without framework-level interference or OOM faults; the first backend implementing this is cuda-checkpoint. Engineers debugging GPU memory contention during multi-job scheduling can follow implementation details like this via daily.dev."}},{"@type":"Question","name":"What APIs does a developer use to integrate their RL training loop with the llm-d time-slice orchestrator?","acceptedAnswer":{"@type":"Answer","text":"Developers wrap accelerator-touching phases with two gRPC calls exposed by a time-slice client library: acquire() to request exclusive accelerator access and yield() to release it. The underlying ML framework, such as PyTorch FSDP or vLLM, along with the CUDA context and memory allocations, runs unmodified, so integration mainly involves annotating training and sampling phases with these calls. Developers wiring RL pipelines into new scheduling APIs often compare integration approaches surfaced on daily.dev."}}]} ```