unknown

some shoutouts to ppl with great content on deeply examining your agent design over auto-accepting slop, 
has shaped my current thinking on how to intentionally design every eval

@HamelHusain - "look at your data" 
@dexhorthy - "you can't outsource the thinking"
@masondrxy - no quote, we just spend a lot of time reading agent outputs and he's incredibly thoughtful on how our design decisions affect the output.

all the above are immediately important for eval design.  
you can't just ask Claude to pick 3 benchmarks, run them, and say the agent is good

every single task uniquely measures the behavior we want our agent to have and we should spend time reading tasks and the traces

worth checking out their content if you're on an eval journey or reach out here :)

<p>some shoutouts to ppl with great content on deeply examining your agent design over auto-accepting slop, 
has shaped my current thinking on how to intentionally design every eval

@HamelHusain - &quot;look at your data&quot; 
@dexhorthy - &quot;you can't outsource the thinking&quot;
@masondrxy - no quote, we just spend a lot of time reading agent outputs and he's incredibly thoughtful on how our design decisions affect the output.

all the above are immediately important for eval design.  
you can't just ask Claude to pick 3 benchmarks, run them, and say the agent is good

every single task uniquely measures the behavior we want our agent to have and we should spend time reading tasks and the traces

worth checking out their content if you're on an eval journey or reach out here :)</p>

Harrison Chase

A retweet sharing shoutouts to people producing quality content on agent design, emphasizing thoughtful evaluation over blindly accepting AI-generated output.