--- title: More models, more choice: Comparing 11 different AI models description: Netlify partnered with OpenRouter to let Agent Runners use a wide range of AI models, including Claude Opus 5 and Sonnet 5, GPT 5.6 (Sol/Terra/Luna tiers),... canonical: https://daily.dev/posts/more-models-more-choice-comparing-11-different-ai-models-uqrz8i7ag twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: More models, more choice: Comparing 11 different AI models | daily.dev og:description: Netlify partnered with OpenRouter to let Agent Runners use a wide range of AI models, including Claude Opus 5 and Sonnet 5, GPT 5.6 (Sol/Terra/Luna tiers),... og:url: https://daily.dev/posts/more-models-more-choice-comparing-11-different-ai-models-uqrz8i7ag og:image: https://api.daily.dev/og/posts/uQrZ8I7Ag.png og:image:alt: More models, more choice: Comparing 11 different AI models og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # More models, more choice: Comparing 11 different AI models **[Hacker News](https://daily.dev/sources/hn)** · 15 min read · 1 upvotes · 0 comments ## Summary Netlify partnered with OpenRouter to let Agent Runners use a wide range of AI models, including Claude Opus 5 and Sonnet 5, GPT 5.6 (Sol/Terra/Luna tiers), Gemini 3.6 Flash and 3.1 Pro, Kimi K3 and K2.7 Code, GLM 5.2, and DeepSeek V4 (Pro and Flash). Using an internal evaluation tool called AXIS, Netlify ran the same three prompts across all models and compared credit usage and output quality, starting with a simple coffee shop website scenario. Results varied widely: Claude Opus 5 produced the most polished designs but sometimes used over 1,000 credits per run, while DeepSeek V4 Flash 0731 produced usable results for under 3 credits on average. Open-weight models like Kimi and GLM showed mixed design quality but very low cost, and the post promises follow-up coverage of more complex app-building scenarios where platform-specific functionality matters more than visual design. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Which AI models are available through Netlify's Agent Runners now? Netlify's Agent Runners now supports Claude Agent, OpenAI Codex, and Gemini CLI as before, plus a new OpenCode agent to drive additional models via an OpenRouter partnership, including Kimi K3, GLM 5.2, and DeepSeek V4. This expands the agent runner selection beyond the original three provider-specific coding agents to any model available on OpenRouter. _Developers weighing which coding agent and model to build with can track platform updates like this on daily.dev._ ### How much do Claude Opus 5 and DeepSeek V4 Flash differ in credit cost for generating a simple website? Claude Opus 5 averaged 519 credits across three runs building a one-page coffee shop site, with one run spiking to 1,055 credits, while DeepSeek V4 Flash 0731 averaged just 2.4 credits for comparable output. Opus produced more polished, detailed designs, but DeepSeek V4 Flash's cheapest run was described as resembling mid-tier closed-model quality despite the far lower cost. _Anyone budgeting AI credits across models can compare cost-versus-quality trade-offs like this via daily.dev._ ### Does GLM 5.2 support image inputs for AI-assisted web design? No, GLM 5.2 is a text-only model that cannot accept image inputs, meaning it cannot use a reference screenshot to guide visual design the way Kimi models can. This limits its usefulness for design-inspiration workflows even though it produces varied results at very low credit cost. _Developers picking a model based on multimodal capability can weigh details like this on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 77 comments across hackernews (as of 2026-09-13). **TL;DR:** The comparison sparked more discussion about how similar all the models' outputs looked and whether the coffee-shop prompt is even a meaningful benchmark than about which model actually won, with several commenters preferring the cheaper, plainer outputs over the flashier expensive ones. **Sentiment:** 20% positive · 45% mixed · 35% skeptical **The case for** - Some found the cost-vs-output comparison genuinely useful and more concrete than typical benchmark posts. - Several liked that cheaper/open models (Kimi, GLM, DeepSeek) produced more scannable, less cluttered results for finding practical info like hours and address. - One appreciated that the piece was more enjoyable to read than typical benchmark posts. **The pushback** - Many pointed out that all the models' outputs looked strikingly similar, calling into question the value of the comparison. - Several argued a one-shot, low-detail prompt with only three runs per model is not representative of real-world development work or serious model evaluation. - Some questioned whether a coffee shop even needs a custom website at all, sparking a long tangential debate. - Concerns were raised about the lack of variance/statistical rigor (sample size, run count) undermining confidence in the results. - Some felt evals like this fail to capture real agentic coding work, being closer to toy examples than genuine dev benchmarks. **By community** - hackernews (mixed): Discussion split between interest in the cost/output comparison and skepticism about whether the coffee-shop prompt and methodology are meaningful, with a long tangent debating whether coffee shops even need websites. **Hottest debate:** Whether a simple coffee shop even needs its own website versus relying on Google/Maps listings. **Open questions** - How would results differ with more runs per model given the observed variance from run to run? - How would the models perform on more complex, real-world constrained tasks rather than an open-ended one-shot prompt? - Would results differ meaningfully if follow-up/iterative prompts were used instead of a single one-shot generation? **Highlights** > It's also kind of pointless. Why does a coffee shop need a website anyway? Nobody is saying, "man I would love to go to this coffee shop but I can't find their website". If all you want is "opening hours, the address, a short menu and a photo" there are easier and cheaper ways to do that. > — [tantalor on hackernews · 3 comments](https://news.ycombinator.com/item?id=49285845) > Am I wrong or are these evaluations, while interesting, not really meaningful for anyone doing serious development work? I'm asking because I personally only use AI with specific and detailed instructions, building my projects piece-by-piece. I mostly don't look at the low level code and some of it I don't understand as much as I'd like, but I very much give much more technical instructions than a simple, two sentence prompt. So, it seems to me this "oneshot from a simple prompt" eval is fairly meaningless when it comes to model evaluation itself, as it is in no way representative of real world application. This would be more something for "vibe coders", people with little to no programming background wanting a website? > — [Systemerror7A69 on hackernews · 10 comments](https://news.ycombinator.com/item?id=49285824) > Approaching this from the perspective of a potential customer and not a designer - I find that I actually like the smaller/open model output quite a bit more. Kimi, GLM, and Deepseek all absolutely run away with the "Can I quickly read the menu and find the address" challenge. Most of the rest of the pages are stylistic, but hard to parse. If I were in a car on a mobile phone trying to find the address of the place to meet a friend for coffee... I don't want a bunch of fluff and stylistic design that makes it hard to parse the information on the site. > — [horsawlarway on hackernews](https://news.ycombinator.com/item?id=49285842) > > Building ad-hoc evals is trivial these days. I highly doubt that you have solved it. Writing proper evals and benchmarks for real-world scenarios is far from trivial. Benchmarking an agent essentially means freezing, at the very minimum: - the model - the model's configuration (e.g. effort, permissions, provider) - the dataset (e.g. a git repository at a specific sha) - the code running the agent itself (you can build your own harness, trivial, but you still need to ship it as a single executable, froze in time. benchmarking against closed source runtime like claude code is quite useless, they change too frequently and in ways you cannot directly inspect). - the tools at agent's disposal. Even a slightly different implementation of tool X (e.g. grep or readfile or sed) has an impact. In general this implies also freezing a very specific container image. In my personal benchmarks I provide a specific list of tools that come with the executable, there's no possibility of interacting with the outside world besides the provided apis, the agent bundles its own tools. And even then: there's significant noise coming from the LLM providers themselves which noticeably change the models behaviour, I don't know whether this is because they optimize some settings or change the inference over time, etc. And, last but not least, the output of LLMs is non deterministic. Also, the LLM as judge presents essentially the same non-deterministic problems, has to be benchmarked itself thoroughly, and writing quality rubrics or "golden answers/outputs" is just difficult. One of the metrics I consistently try to emphasize is to avoid the "shotgun vomit dump" of information. So answers that get right to the point in plain terms avoiding dumps of information filled of jargon on top of the user are rated differently. In short: its far from trivial to benchmark models on real-world agentic work taken from your personal or professional projects. And even creating the test cases themselves is hard. No: you cannot take the output of some "sota" and use it as gold standard. This is a very crap approach. It's the sloppiest solution to the problem, in the very sense of slop: plausible, average, lacking any creativity or out of the box thinking, the things that make the real difference in complex software development. The very point of creating these benchmarks is to find which configuration/model/tools/harness (skills/mcps/documentation/agents.md, etc) works better. And it only works if you create these benchmarks yourself from genuinely difficult non-trivial work and find a solution that is better by most metrics implementation-wise, albeit you could settle on the implementation solving a series of cases and edge cases. > — [epolanski on hackernews](https://news.ycombinator.com/item?id=49286042) > I've been doing a lot of benchmarking for a long time now with a number of local models for the purposes of spam filtering. The major observation is that there is a lot of variance in model performance. This should not be surprising, as these are probabilistic machines based on random numbers, so your performance will vary from run to run. But this also means that any sort of evaluation of benchmark with a sample size of 1 is essentially worthless for the purposes of model comparison. In my benchmarks, I started insisting on having at least 5 runs. This also makes me very suspicious of some of the posted benchmarks that compare models. If the results don't say how many benchmark runs were performed and what the variance was, there is really no basis for comparison. You are just guessing. > — [jwr on hackernews · 1 comments](https://news.ycombinator.com/item?id=49285776) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49285327) · 171 points · 77 comments ## Similar posts on daily.dev - [How to Run OpenClaw with Open-Source Models](https://daily.dev/posts/how-to-run-openclaw-with-open-source-models-e58kjjusl) · Towards Data Science · 1 upvotes · 0 comments - [Why Your Best Model Is Two Models: Routing Between Kimi K3 and Claude](https://daily.dev/posts/why-your-best-model-is-two-models-routing-between-kimi-k3-and-claude-ejhl4kv4j) · DigitalOcean Community · 0 upvotes · 0 comments - [Open-source coding agents like OpenCode, Cline, and Aider are solving a huge headache for developers](https://daily.dev/posts/open-source-coding-agents-like-opencode-cline-and-aider-are-solving-a-huge-headache-for-developers-77xyw8rmy) · The New Stack · 4 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents), [#vibe-coding](https://daily.dev/tags/vibe-coding), [#netlify](https://daily.dev/tags/netlify), [#openrouter](https://daily.dev/tags/openrouter) [View this post on daily.dev](https://daily.dev/posts/more-models-more-choice-comparing-11-different-ai-models-uqrz8i7ag) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"More models, more choice: Comparing 11 different AI models","url":"https://daily.dev/posts/more-models-more-choice-comparing-11-different-ai-models-uqrz8i7ag","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/more-models-more-choice-comparing-11-different-ai-models-uqrz8i7ag"},"datePublished":"2026-08-13T13:51:00.141Z","dateModified":"2026-09-13T19:44:22.620Z","description":"Netlify partnered with OpenRouter to let Agent Runners use a wide range of AI models, including Claude Opus 5 and Sonnet 5, GPT 5.6 (Sol/Terra/Luna tiers),...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e43a7efaee9c9e67e2b0b72536d3732d?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e43a7efaee9c9e67e2b0b72536d3732d?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/more-models-more-choice-comparing-11-different-ai-models-uqrz8i7ag","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,ai-agents,vibe-coding,netlify,openrouter","timeRequired":"PT15M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"More models, more choice: Comparing 11 different AI models"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/more-models-more-choice-comparing-11-different-ai-models-uqrz8i7ag#faq","mainEntity":[{"@type":"Question","name":"Which AI models are available through Netlify's Agent Runners now?","acceptedAnswer":{"@type":"Answer","text":"Netlify's Agent Runners now supports Claude Agent, OpenAI Codex, and Gemini CLI as before, plus a new OpenCode agent to drive additional models via an OpenRouter partnership, including Kimi K3, GLM 5.2, and DeepSeek V4. This expands the agent runner selection beyond the original three provider-specific coding agents to any model available on OpenRouter. Developers weighing which coding agent and model to build with can track platform updates like this on daily.dev."}},{"@type":"Question","name":"How much do Claude Opus 5 and DeepSeek V4 Flash differ in credit cost for generating a simple website?","acceptedAnswer":{"@type":"Answer","text":"Claude Opus 5 averaged 519 credits across three runs building a one-page coffee shop site, with one run spiking to 1,055 credits, while DeepSeek V4 Flash 0731 averaged just 2.4 credits for comparable output. Opus produced more polished, detailed designs, but DeepSeek V4 Flash's cheapest run was described as resembling mid-tier closed-model quality despite the far lower cost. Anyone budgeting AI credits across models can compare cost-versus-quality trade-offs like this via daily.dev."}},{"@type":"Question","name":"Does GLM 5.2 support image inputs for AI-assisted web design?","acceptedAnswer":{"@type":"Answer","text":"No, GLM 5.2 is a text-only model that cannot accept image inputs, meaning it cannot use a reference screenshot to guide visual design the way Kimi models can. This limits its usefulness for design-inspiration workflows even though it produces varied results at very low credit cost. Developers picking a model based on multimodal capability can weigh details like this on daily.dev."}}]} ```