A joint study by LangChain Labs and Harvey explores how to reduce the cost of LLM-based verifiers used in legal agent evaluation and RL post-training. Two main strategies are tested: batching multiple rubric criteria into a single LLM call (reducing token usage by ~10x) and substituting frontier models with cheaper alternatives like DeepSeek. Experiments ran across 2,348 rubric criteria from 40 legal tasks. DeepSeek emerged as a strong approximation of Opus at 60–1000x lower cost, while Haiku showed unacceptably high false-pass rates for legal use. Prompt tuning via an auto-research loop further reduced DeepSeek's false-pass rate. The findings suggest open models with batched verification can make large-scale legal agent evaluation and RL post-training practically feasible.
