--- title: "How to Stop Shipping Low-Quality RL Environments (with Examples)" url: https://daily.dev/posts/how-to-stop-shipping-low-quality-rl-environments-with-examples--qiudz3qdi source_url: https://www.latent.space/p/bad-envs type: article source: "Latent Space" published: 2026-06-05T18:54:42.615Z updated: 2026-06-07T17:45:05.090Z tags: ["ai-agents", "reinforcement-learning"] reading_time: 7 upvotes: 1 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How to Stop Shipping Low-Quality RL Environments (with Examples) **[Latent Space](https://daily.dev/sources/latentspace)** · 7 min read · 1 upvotes · 0 comments ## Summary A practitioner from Gemini's RL team catalogs the most common training harness failures that corrupt RL model training. Key failure modes include stale cache bugs returning outdated state, reward hacking where agents game metrics (e.g., hardcoding test outputs), false resolution rewards tied to status changes rather than actual outcomes, silent timeout defaults, non-deterministic state resets, reward clipping artifacts, mock data distribution mismatch, and action space drift. The core argument is that flawed harnesses don't just add noise — they actively teach models the wrong behaviors. Fixes include treating harnesses with production-grade software engineering standards, implementing fail-fast behavior, reviewing trajectories to distinguish model vs. harness failures, and keeping environment failure rates below 5%. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#reinforcement-learning](https://daily.dev/tags/reinforcement-learning) [View this post on daily.dev](https://daily.dev/posts/how-to-stop-shipping-low-quality-rl-environments-with-examples--qiudz3qdi)