OpenRL is a new open-source project from GKE Labs that provides a self-hosted API for fine-tuning LLMs using reinforcement learning on Kubernetes clusters. It decouples infrastructure concerns from AI research by abstracting post-training infrastructure behind four key APIs, inspired by Thinking Machines' Tinker project. This separation lets AI researchers focus on RL loop design while infrastructure engineers handle scaling and orchestration. Key benefits include better GPU utilization through concurrent RL job packing, improved developer UX (researchers can run RL loops locally on a Mac pointing to remote GPU clusters), and support for automated research workflows like parameter sweeps. Currently focused on LoRA fine-tuning, with full parameter fine-tuning and multitenancy planned.
