--- title: RL training agents through Claude Code and OpenClaw... description: A new paper called ClawGym II tackles the friction of RL-training agents that run inside real harnesses like Claude Code or OpenClaw, without needing to... canonical: https://daily.dev/posts/rl-training-agents-through-claude-code-and-openclaw-without-touching-harness-internals-9jbewndmo twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: RL training agents through Claude Code and OpenClaw without touching harness internals | daily.dev og:description: A new paper called ClawGym II tackles the friction of RL-training agents that run inside real harnesses like Claude Code or OpenClaw, without needing to... og:url: https://daily.dev/posts/rl-training-agents-through-claude-code-and-openclaw-without-touching-harness-internals-9jbewndmo og:image: https://api.daily.dev/og/posts/9jbEWndMo.png og:image:alt: RL training agents through Claude Code and OpenClaw without touching harness internals og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # RL training agents through Claude Code and OpenClaw without touching harness internals **[Collections](https://daily.dev/sources/collections)** · 2 min read · 1 upvotes · 0 comments ## Summary A new paper called ClawGym II tackles the friction of RL-training agents that run inside real harnesses like Claude Code or OpenClaw, without needing to reimplement the harness's internal tool routing, retries, or subagent logic. Instead, it treats the harness as a black box, intercepting model calls via a proxy at the serving boundary and reassembling them into prefix-tree trajectories that PPO and GRPO can optimize over. Using Qwen3-30A3B, this approach improved ClawGym-Bench Pass@1 by 9.98 points via OpenClaw and 14.81 points via Claude Code, with gains holding across 200-400 optimization steps. Notably, a policy trained jointly across both harnesses matched or beat harness-specific models, suggesting the approach produces policies that generalize across execution environments rather than overfitting to one harness. Similar gains were reported on JobBench and OfficeQA. ## Content There's a real bottleneck in agent RL that doesn't get talked about enough: to train a model on an agent harness, you usually need to rebuild that harness's internals yourself. Tool routing, retries, context management, subagent orchestration — all of it. That's a lot of engineering just to run an RL loop, and it means your training setup rarely matches what actually ships in production. ClawGym II gets around this by treating the harness as a black box. Instead of reimplementing OpenClaw or Claude Code's internals, the framework runs them unchanged inside sandboxes and intercepts model calls right at the serving boundary. Those intercepted calls, which arrive fragmented across turns and subagents, get reassembled into prefix-tree trajectories. Once you have that structure, PPO or GRPO can optimize over it directly — no need to know or reproduce how the harness routes tools or manages retries internally. The results are solid. With Qwen3-30A3B, this approach raised ClawGym-Bench Pass@1 by 9.98 points through OpenClaw and 14.81 points through Claude Code, and the gains held steady across 200 to 400 optimization steps. What I find more interesting is the mix-harness training result. A single policy trained jointly on both OpenClaw and Claude Code matched or slightly beat the harness-specific models when evaluated on either system. That's a hint that you can train policies that generalize across execution environments rather than overfitting to one particular harness's quirks. The paper also reports gains on JobBench and OfficeQA, so this isn't just a ClawGym-specific trick — it seems to hold up across different agent tasks. If you're working on agent training and have been stuck reproducing harness internals just to get an RL signal, this is worth reading closely. Paper: arxiv.org/abs/2608.16798 — "ClawGym II: Exploring Black-Box RL on Agent Harness" ## Questions this post answers ### How can I RL-train an agent that runs inside Claude Code or OpenClaw without reimplementing the harness internals? A technique called ClawGym II treats the harness as a black box and runs it unchanged inside sandboxes, using a proxy at the model-serving boundary to intercept every call the harness makes. Those fragmented calls across tool use, retries, and subagent hops are reassembled into prefix-tree trajectories that PPO and GRPO can optimize over, avoiding the need to reproduce the harness's tool routing or retry logic. _daily.dev surfaces techniques like this for teams building RL pipelines around real agent harnesses._ ### Does training a single RL policy jointly on Claude Code and OpenClaw trajectories work better than training separate harness-specific policies? Yes, a single policy trained jointly on trajectories from both OpenClaw and Claude Code matched or slightly beat harness-specific models when evaluated on either system. This suggests the mixed-harness training produces a policy that transfers across execution environments rather than overfitting to one harness's particular quirks. _Track emerging agent training results like this on daily.dev before committing to a training strategy._ ### What Pass@1 improvement did ClawGym II achieve on ClawGym-Bench using Qwen3-30A3B? Pass@1 improved by 9.98 points through OpenClaw and 14.81 points through Claude Code when training Qwen3-30A3B with ClawGym II. The gains held steady across 200 to 400 optimization steps, and similar improvements were reported on JobBench and OfficeQA, indicating the technique isn't limited to a single benchmark. _daily.dev helps developers evaluating agent RL benchmarks keep up with results like these._ --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#claude-code](https://daily.dev/tags/claude-code), [#reinforcement-learning](https://daily.dev/tags/reinforcement-learning) [View this post on daily.dev](https://daily.dev/posts/rl-training-agents-through-claude-code-and-openclaw-without-touching-harness-internals-9jbewndmo) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"RL training agents through Claude Code and OpenClaw without touching harness internals","url":"https://daily.dev/posts/rl-training-agents-through-claude-code-and-openclaw-without-touching-harness-internals-9jbewndmo","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/rl-training-agents-through-claude-code-and-openclaw-without-touching-harness-internals-9jbewndmo"},"datePublished":"2026-08-22T14:28:38.645Z","dateModified":"2026-08-23T13:08:12.053Z","description":"A new paper called ClawGym II tackles the friction of RL-training agents that run inside real harnesses like Claude Code or OpenClaw, without needing to...","image":"https://pbs.twimg.com/media/HQVDAKubgAAkoU9.png","thumbnailUrl":"https://pbs.twimg.com/media/HQVDAKubgAAkoU9.png","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/rl-training-agents-through-claude-code-and-openclaw-without-touching-harness-internals-9jbewndmo","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,claude-code,reinforcement-learning","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"RL training agents through Claude Code and OpenClaw without touching harness internals"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/rl-training-agents-through-claude-code-and-openclaw-without-touching-harness-internals-9jbewndmo#faq","mainEntity":[{"@type":"Question","name":"How can I RL-train an agent that runs inside Claude Code or OpenClaw without reimplementing the harness internals?","acceptedAnswer":{"@type":"Answer","text":"A technique called ClawGym II treats the harness as a black box and runs it unchanged inside sandboxes, using a proxy at the model-serving boundary to intercept every call the harness makes. Those fragmented calls across tool use, retries, and subagent hops are reassembled into prefix-tree trajectories that PPO and GRPO can optimize over, avoiding the need to reproduce the harness's tool routing or retry logic. daily.dev surfaces techniques like this for teams building RL pipelines around real agent harnesses."}},{"@type":"Question","name":"Does training a single RL policy jointly on Claude Code and OpenClaw trajectories work better than training separate harness-specific policies?","acceptedAnswer":{"@type":"Answer","text":"Yes, a single policy trained jointly on trajectories from both OpenClaw and Claude Code matched or slightly beat harness-specific models when evaluated on either system. This suggests the mixed-harness training produces a policy that transfers across execution environments rather than overfitting to one harness's particular quirks. Track emerging agent training results like this on daily.dev before committing to a training strategy."}},{"@type":"Question","name":"What Pass@1 improvement did ClawGym II achieve on ClawGym-Bench using Qwen3-30A3B?","acceptedAnswer":{"@type":"Answer","text":"Pass@1 improved by 9.98 points through OpenClaw and 14.81 points through Claude Code when training Qwen3-30A3B with ClawGym II. The gains held steady across 200 to 400 optimization steps, and similar improvements were reported on JobBench and OfficeQA, indicating the technique isn't limited to a single benchmark. daily.dev helps developers evaluating agent RL benchmarks keep up with results like these."}}]} ```