When adopting coding agents, teams debate whether to use AGENTS.md (a single always-loaded markdown file) or skills (modular, on-demand instruction bundles). The real issue isn't the format — it's whether agent steering actually changes behavior. Key findings from evals: instructions have a compliance curve that degrades as context fills, phrasing and placement matter significantly, and more instructions don't linearly improve adherence. Vercel's head-to-head eval found AGENTS.md achieved 100% pass rate vs. 79% for skills, though results vary by setup. The recommended approach is building a reproducible eval loop with fixed fixtures, observable assertions, a runner, and a checker — then integrating it into CI so config changes and model upgrades are gated on measured pass rates, not intuition.
Table of contents
AGENTS.md vs. skills: the short answerWhat the evals show about AGENTS.md vs. skillsHow to test agent config with a reproducible eval loopSteering agents in a delivery pipelineThe format is downstream of the feedback loopQuestions this post answers
What is the difference between AGENTS.md and skills for configuring a coding agent?
AGENTS.md is a single markdown file loaded at the start of every agent session, best for small, stable, always-applicable guidance like build commands and repo conventions. Skills are modular instruction bundles the agent loads on demand only when a task matches, suited for large, situational, or procedural guidance like release checklists or migration steps. Most teams end up using both. Teams choosing between these approaches for their agent setup track real-world eval results on daily.dev.
How did AGENTS.md compare to skills in Vercel's agent eval benchmark?
In Vercel's head-to-head eval, AGENTS.md achieved a 100% pass rate while the skills-based approach topped out at 79%. The gap was partly because the agent sometimes chose not to load the relevant skill at all. Community discussion noted the result is specific to Vercel's tasks, model, and setup, so teams should run their own evals rather than inheriting this conclusion. Developers benchmarking agent configurations find comparative eval results like these on daily.dev.
How do I build a reproducible eval loop to test coding agent configuration?
A workable eval loop has four parts: fixtures (representative tasks pinned to a known repo state and prompt), a behavior to assert (observable, testable directives like 'did lint run before commit'), a runner (script that feeds fixtures to the agent and captures commands and diffs), and a checker (code that inspects output and returns pass or fail). Run each fixture multiple times and track pass rate to account for run-to-run variance. Engineers building agent feedback loops find practical patterns like this on daily.dev.