--- title: How Ora benchmarks every major AI agent on Vercel description: Ora built a platform to benchmark AI agents like Claude Code, ChatGPT, Gemini, Hermes, OpenClaw, and Vercel's eve framework against live customer websites,... canonical: https://daily.dev/posts/how-ora-benchmarks-every-major-ai-agent-on-vercel-a6guigdva twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How Ora benchmarks every major AI agent on Vercel | daily.dev og:description: Ora built a platform to benchmark AI agents like Claude Code, ChatGPT, Gemini, Hermes, OpenClaw, and Vercel's eve framework against live customer websites,... og:url: https://daily.dev/posts/how-ora-benchmarks-every-major-ai-agent-on-vercel-a6guigdva og:image: https://api.daily.dev/og/posts/a6GUiGdVa.png og:image:alt: How Ora benchmarks every major AI agent on Vercel og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How Ora benchmarks every major AI agent on Vercel **[Vercel](https://daily.dev/sources/vercel)** · 6 min read · 3 upvotes · 0 comments ## Summary Ora built a platform to benchmark AI agents like Claude Code, ChatGPT, Gemini, Hermes, OpenClaw, and Vercel's eve framework against live customer websites, measuring where agents fail during signup, integration, and payment flows. Everything runs on Vercel, including separate agent runtimes for each harness since none share infrastructure expectations. When Vercel launched eve, Ora benchmarked it under the same conditions as competitors, finding eve took 7% fewer steps, achieved 2x native success rate, and found 9% more valid endpoints compared to Claude Code running the same models. A prompt-caching fix after benchmark feedback cut eve's cost by roughly 15%. Ora subsequently rebuilt its own agent infrastructure on eve, citing eve's Next.js-like configuration and a sandbox override feature that let Ora's tracing instrumentation work inside eve's execution environment. The 16-person engineering team ships hundreds of commits daily, largely through coding agents managing Vercel infrastructure directly. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How does Vercel's eve agent framework compare to Claude Code in benchmark tests? In head-to-head testing on hundreds of real website journeys using the same models, eve needed 7% fewer steps to complete tasks, achieved twice the native success rate (completing tasks on the customer's own site rather than falling back to web search), and found 9% more valid, callable endpoints than Claude Code, according to results published by benchmarking company Ora. _Developers comparing agent frameworks can follow real-world benchmark results like this on daily.dev._ ### What is the sandbox override feature in Vercel's eve agent framework? It is a feature that lets developers swap out eve's default sandbox (the isolated environment where an agent executes tools and touches files) for a custom environment. Ora used it to route eve's agent execution into its own instrumented tracing environment, so eve agents could be recorded and traced the same way as every other harness it benchmarks. _Engineers building custom agent runtimes can track framework features like this via daily.dev._ ### What issue did benchmarking uncover in Vercel's eve agent framework, and how was it fixed? Benchmarking against Claude Code surfaced a prompt-caching issue in eve. After the eve team shipped a fix, the next round of benchmark results measured roughly 15% lower total cost for completing the same agent tasks. _Teams debugging agent cost and latency issues can follow fixes like this on daily.dev._ ## Similar posts on daily.dev - [Vercel launches eve, an open-source framework that treats agents as directories](https://daily.dev/posts/vercel-launches-eve-an-open-source-framework-that-treats-agents-as-directories-8jfizvhze) · The New Stack · 1 upvotes · 0 comments - [Vercel Introduces Eve, an Open-Source Framework for Building AI Agents](https://daily.dev/posts/vercel-introduces-eve-an-open-source-framework-for-building-ai-agents-ic7xwvqxp) · InfoQ · 2 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#claude-code](https://daily.dev/tags/claude-code), [#vercel](https://daily.dev/tags/vercel) [View this post on daily.dev](https://daily.dev/posts/how-ora-benchmarks-every-major-ai-agent-on-vercel-a6guigdva) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How Ora benchmarks every major AI agent on Vercel","url":"https://daily.dev/posts/how-ora-benchmarks-every-major-ai-agent-on-vercel-a6guigdva","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-ora-benchmarks-every-major-ai-agent-on-vercel-a6guigdva"},"datePublished":"2026-08-21T19:17:58.929Z","dateModified":"2026-09-14T06:42:45.499Z","description":"Ora built a platform to benchmark AI agents like Claude Code, ChatGPT, Gemini, Hermes, OpenClaw, and Vercel's eve framework against live customer websites,...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7f8d75775bb40b125e315a38eb3f23ab?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7f8d75775bb40b125e315a38eb3f23ab?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Vercel","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Vercel","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/04cca7235cf84db092f24b7ba427b3e1","url":"https://daily.dev/sources/vercel"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-ora-benchmarks-every-major-ai-agent-on-vercel-a6guigdva","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,claude-code,vercel","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Vercel","item":"https://daily.dev/sources/vercel"},{"@type":"ListItem","position":3,"name":"How Ora benchmarks every major AI agent on Vercel"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-ora-benchmarks-every-major-ai-agent-on-vercel-a6guigdva#faq","mainEntity":[{"@type":"Question","name":"How does Vercel's eve agent framework compare to Claude Code in benchmark tests?","acceptedAnswer":{"@type":"Answer","text":"In head-to-head testing on hundreds of real website journeys using the same models, eve needed 7% fewer steps to complete tasks, achieved twice the native success rate (completing tasks on the customer's own site rather than falling back to web search), and found 9% more valid, callable endpoints than Claude Code, according to results published by benchmarking company Ora. Developers comparing agent frameworks can follow real-world benchmark results like this on daily.dev."}},{"@type":"Question","name":"What is the sandbox override feature in Vercel's eve agent framework?","acceptedAnswer":{"@type":"Answer","text":"It is a feature that lets developers swap out eve's default sandbox (the isolated environment where an agent executes tools and touches files) for a custom environment. Ora used it to route eve's agent execution into its own instrumented tracing environment, so eve agents could be recorded and traced the same way as every other harness it benchmarks. Engineers building custom agent runtimes can track framework features like this via daily.dev."}},{"@type":"Question","name":"What issue did benchmarking uncover in Vercel's eve agent framework, and how was it fixed?","acceptedAnswer":{"@type":"Answer","text":"Benchmarking against Claude Code surfaced a prompt-caching issue in eve. After the eve team shipped a fix, the next round of benchmark results measured roughly 15% lower total cost for completing the same agent tasks. Teams debugging agent cost and latency issues can follow fixes like this on daily.dev."}}]} ```