--- title: Hugging Face Introduces Community Evals for Transparent... description: Hugging Face has launched Community Evals, a beta feature that lets benchmark datasets on the Hub host their own leaderboards and automatically aggregate... canonical: https://daily.dev/posts/hugging-face-introduces-community-evals-for-transparent-model-benchmarking-o6b6dztxt twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Hugging Face Introduces Community Evals for Transparent Model Benchmarking | daily.dev og:description: Hugging Face has launched Community Evals, a beta feature that lets benchmark datasets on the Hub host their own leaderboards and automatically aggregate... og:url: https://daily.dev/posts/hugging-face-introduces-community-evals-for-transparent-model-benchmarking-o6b6dztxt og:image: https://api.daily.dev/og/posts/o6B6dzTxT.png og:image:alt: Hugging Face Introduces Community Evals for Transparent Model Benchmarking og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Hugging Face Introduces Community Evals for Transparent Model Benchmarking **[InfoQ](https://daily.dev/sources/infoq)** · 3 min read · 0 upvotes · 0 comments ## Summary Hugging Face has launched Community Evals, a beta feature that lets benchmark datasets on the Hub host their own leaderboards and automatically aggregate evaluation results from model repositories. Benchmarks define evaluation specs via an `eval.yaml` file using the Inspect AI format, ensuring reproducibility. Model authors and community members can submit scores through pull requests, with all changes versioned via Git for full traceability. Initial supported benchmarks include MMLU-Pro, GPQA, and HLE. The system aims to address inconsistencies in reported benchmark scores across papers and platforms by making evaluation reporting decentralized, transparent, and standardized. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Featuring Every Eval Ever Results on Hugging Face Model Pages](https://daily.dev/posts/featuring-every-eval-ever-results-on-hugging-face-model-pages-ks9sdvnh2) · Hugging Face · 2 upvotes · 0 comments - [Kaggle introduces Community Benchmarks to allow for custom evaluations of AI models](https://daily.dev/posts/kaggle-introduces-community-benchmarks-to-allow-for-custom-evaluations-of-ai-models-xler4o2u6) · SD Times · 0 upvotes · 0 comments --- Tags: [#ai](https://daily.dev/tags/ai), [#machine-learning](https://daily.dev/tags/machine-learning), [#data-science](https://daily.dev/tags/data-science), [#llm](https://daily.dev/tags/llm) [View this post on daily.dev](https://daily.dev/posts/hugging-face-introduces-community-evals-for-transparent-model-benchmarking-o6b6dztxt) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Hugging Face Introduces Community Evals for Transparent Model Benchmarking","url":"https://daily.dev/posts/hugging-face-introduces-community-evals-for-transparent-model-benchmarking-o6b6dztxt","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/hugging-face-introduces-community-evals-for-transparent-model-benchmarking-o6b6dztxt"},"datePublished":"2026-02-19T11:01:26.147Z","dateModified":"2026-08-24T07:01:12.756Z","description":"Hugging Face has launched Community Evals, a beta feature that lets benchmark datasets on the Hub host their own leaderboards and automatically aggregate...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/c043dacb2e8fe195115837335e3cbb22?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/c043dacb2e8fe195115837335e3cbb22?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"InfoQ","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"InfoQ","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/afc3bced3e1e4b188dd9127017a60e0c","url":"https://daily.dev/sources/infoq"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/hugging-face-introduces-community-evals-for-transparent-model-benchmarking-o6b6dztxt","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai,machine-learning,data-science,llm","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"InfoQ","item":"https://daily.dev/sources/infoq"},{"@type":"ListItem","position":3,"name":"Hugging Face Introduces Community Evals for Transparent Model Benchmarking"}]} ```