--- title: 11 LLM Evaluation Methods | daily.dev description: A visual breakdown of 11 LLM evaluation methods every AI engineer should know. Covers classic metrics like BLEU and ROUGE, semantic similarity via BERTScore,... canonical: https://daily.dev/posts/11-llm-evaluation-methods-pph5ntaze twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: 11 LLM Evaluation Methods | daily.dev og:description: A visual breakdown of 11 LLM evaluation methods every AI engineer should know. Covers classic metrics like BLEU and ROUGE, semantic similarity via BERTScore,... og:url: https://daily.dev/posts/11-llm-evaluation-methods-pph5ntaze og:image: https://api.daily.dev/og/posts/PpH5NTaze.png og:image:alt: 11 LLM Evaluation Methods og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # 11 LLM Evaluation Methods **[Daily Dose of Data Science \| Avi Chawla \| Substack](https://daily.dev/sources/dailydoseofds)** · 8 min read · 1 upvotes · 0 comments ## Summary A visual breakdown of 11 LLM evaluation methods every AI engineer should know. Covers classic metrics like BLEU and ROUGE, semantic similarity via BERTScore, LLM-as-Judge and LLM juries for subjective criteria, DAG-based rubric scoring, trajectory accuracy for agents, multi-turn conversation evaluation, and safety classifiers. Each method is explained with its appropriate use case, key pitfalls, and practical guidance on when to apply it. Also briefly covers Kimi K3's delta attention mechanism for handling million-token contexts with linear memory cost. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why does a correct LLM answer sometimes score near zero on BLEU? BLEU measures n-gram overlap between the output and a reference text, so a correct paraphrase that uses different wording than the reference can score close to zero even though the meaning is right. This is why BLEU works well for translation, where reference wording is close to the only acceptable phrasing, but fails for open-ended generation. _Anyone picking evaluation metrics for a generation pipeline can compare tradeoffs like this through daily.dev._ ### What is delta attention in Kimi K3 and how does it avoid a growing KV cache? Delta attention replaces the standard growing list of key-value pairs with one fixed-size matrix that behaves like a lookup table. For each token it reads the matrix's current guess for a key, compares it to the desired value, and writes only the difference (the delta), correcting old associations instead of appending new ones, which gives linear rather than quadratic cost, at the price of only approximate recall of any single token. _Engineers tracking long-context attention tricks like this can follow the space on daily.dev._ ### Why is LLM-as-judge evaluation biased toward longer answers and how can that be controlled? Judge models reliably favor longer outputs regardless of actual quality, and they can also favor whichever answer appears first in the comparison. Controlling for this requires randomizing answer order across evaluation runs and explicitly controlling for length, otherwise pairwise comparisons between models, prompts, or retrieval configs will be skewed toward verbosity rather than correctness. _Teams designing LLM evaluation pipelines can dig into pitfalls like judge bias on daily.dev._ ## Similar posts on daily.dev - [Everything You Need to Know About LLM Evaluation Metrics](https://daily.dev/posts/everything-you-need-to-know-about-llm-evaluation-metrics-rqy5cls0k) · Machine Learning Mastery · 0 upvotes · 0 comments - [Best Practices and Methods for LLM Evaluation](https://daily.dev/posts/best-practices-and-methods-for-llm-evaluation-x3yfemf90) · databricks · 1 upvotes · 0 comments - [LLM evaluation metrics explained: how to measure AI system quality](https://daily.dev/posts/llm-evaluation-metrics-explained-how-to-measure-ai-system-quality-ynrvblcpe) · Netguru · 0 upvotes · 0 comments - [Choosing Effective LLM Evaluation Strategies for AI Teams](https://daily.dev/posts/choosing-effective-llm-evaluation-strategies-for-ai-teams-zboitrqp2) · PromptLayer Blog · 0 upvotes · 0 comments - [29 LLM Evaluation Concepts Every Engineer Needs to Know](https://daily.dev/posts/29-llm-evaluation-concepts-every-engineer-needs-to-know-4ux61ldzn) · System Design Newsletter · 3 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents), [#llmops](https://daily.dev/tags/llmops) [View this post on daily.dev](https://daily.dev/posts/11-llm-evaluation-methods-pph5ntaze) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"11 LLM Evaluation Methods","url":"https://daily.dev/posts/11-llm-evaluation-methods-pph5ntaze","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/11-llm-evaluation-methods-pph5ntaze"},"datePublished":"2026-07-24T22:08:05.117Z","dateModified":"2026-09-13T19:16:59.853Z","description":"A visual breakdown of 11 LLM evaluation methods every AI engineer should know. Covers classic metrics like BLEU and ROUGE, semantic similarity via BERTScore,...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/85f05ae85cac514e0b49e4a5652d9cac?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/85f05ae85cac514e0b49e4a5652d9cac?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Daily Dose of Data Science | Avi Chawla | Substack","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Daily Dose of Data Science | Avi Chawla | Substack","logo":"https://media.daily.dev/image/upload/s--4IHQgTOw--/f_auto/v1710503712/logos/dailydoseofds","url":"https://daily.dev/sources/dailydoseofds"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/11-llm-evaluation-methods-pph5ntaze","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,ai-agents,llmops","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Daily Dose of Data Science | Avi Chawla | Substack","item":"https://daily.dev/sources/dailydoseofds"},{"@type":"ListItem","position":3,"name":"11 LLM Evaluation Methods"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/11-llm-evaluation-methods-pph5ntaze#faq","mainEntity":[{"@type":"Question","name":"Why does a correct LLM answer sometimes score near zero on BLEU?","acceptedAnswer":{"@type":"Answer","text":"BLEU measures n-gram overlap between the output and a reference text, so a correct paraphrase that uses different wording than the reference can score close to zero even though the meaning is right. This is why BLEU works well for translation, where reference wording is close to the only acceptable phrasing, but fails for open-ended generation. Anyone picking evaluation metrics for a generation pipeline can compare tradeoffs like this through daily.dev."}},{"@type":"Question","name":"What is delta attention in Kimi K3 and how does it avoid a growing KV cache?","acceptedAnswer":{"@type":"Answer","text":"Delta attention replaces the standard growing list of key-value pairs with one fixed-size matrix that behaves like a lookup table. For each token it reads the matrix's current guess for a key, compares it to the desired value, and writes only the difference (the delta), correcting old associations instead of appending new ones, which gives linear rather than quadratic cost, at the price of only approximate recall of any single token. Engineers tracking long-context attention tricks like this can follow the space on daily.dev."}},{"@type":"Question","name":"Why is LLM-as-judge evaluation biased toward longer answers and how can that be controlled?","acceptedAnswer":{"@type":"Answer","text":"Judge models reliably favor longer outputs regardless of actual quality, and they can also favor whichever answer appears first in the comparison. Controlling for this requires randomizing answer order across evaluation runs and explicitly controlling for length, otherwise pairwise comparisons between models, prompts, or retrieval configs will be skewed toward verbosity rather than correctness. Teams designing LLM evaluation pipelines can dig into pitfalls like judge bias on daily.dev."}}]} ```