The Reward Signal Problem for Agents

Part 11 of a Reinforcement Learning course tackles the reward signal problem in agentic RL. For tasks like math and code, verifiers provide clear rewards, but free-form tasks (RAG, summarization, customer support) have no verifier. The current best solution is LLM-as-a-judge, where a capable model scores agent outputs and those scores feed into GRPO as advantages. Research shows strong judge models agree with human preferences at over 80%. The post also covers Constitutional AI, hand-written reward function limitations, known judge biases, and includes a hands-on implementation of an LLM-as-judge scorer on a RAG task. A sponsored section demonstrates building agents with AWS Strands Agents SDK using a model-driven loop and MCP integration.