A practical playbook for tuning the scaffolding around Nemotron 3 Ultra (an open NVIDIA model) rather than the model weights themselves. By iterating on system prompts, tool descriptions, and middleware using a trace-driven eval loop, the authors brought Nemotron's best-run score on the Deep Agents suite to 0.86 — nearly matching Claude Opus 4.8's best of 0.87 — at roughly 10x lower cost (~$4.48 vs ~$43.48 per full suite run). Key findings: targeted single-purpose prompt blocks outperform broad rewrites; context injection at the point of need beats standing system-prompt rules; middleware enforces reliability that prompting can't guarantee; and harness tuning has a hard ceiling where model post-training must take over.
Table of contents
The harness is the part you controlStart with evalsThe two layersWhat didn't workWhat the tuning bought usWhere the lines areFurther reading4 Impressions