--- title: Testing robustness against unforeseen adversaries description: OpenAI researchers introduce UAR (Unforeseen Attack Robustness), a new metric and methodology for evaluating how well neural network classifiers hold up... canonical: https://daily.dev/posts/testing-robustness-against-unforeseen-adversaries-cvwkzgymk twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Testing robustness against unforeseen adversaries | daily.dev og:description: OpenAI researchers introduce UAR (Unforeseen Attack Robustness), a new metric and methodology for evaluating how well neural network classifiers hold up... og:url: https://daily.dev/posts/testing-robustness-against-unforeseen-adversaries-cvwkzgymk og:image: https://api.daily.dev/og/posts/CvwkZGYMK.png og:image:alt: Testing robustness against unforeseen adversaries og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Testing robustness against unforeseen adversaries **[OpenAI](https://daily.dev/sources/openai)** · 5 min read · 0 upvotes · 0 comments ## Summary OpenAI researchers introduce UAR (Unforeseen Attack Robustness), a new metric and methodology for evaluating how well neural network classifiers hold up against adversarial attacks not encountered during training. The key finding is that robustness against one known distortion type rarely transfers to unforeseen attack types, meaning evaluating models on only familiar distortions gives a false sense of security. The three-step method compares model performance against diverse held-out distortions across a calibrated range of distortion sizes, benchmarked against adversarially trained models. A code package with attack implementations, pre-trained models, and calibrations is provided to help researchers adopt the methodology. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Your tests can’t catch a poisoned model](https://daily.dev/posts/your-tests-can-t-catch-a-poisoned-model-no5etuirt) · LeadDev · 0 upvotes · 0 comments - [AI Models Can Provably Hide Arbitrary Capabilities](https://daily.dev/posts/ai-models-can-provably-hide-arbitrary-capabilities-uy7uftyt0) · Hacker News · 0 upvotes · 0 comments - [OpenAI called the Hugging Face attack unprecedented. But we’ve been here before.](https://daily.dev/posts/openai-called-the-hugging-face-attack-unprecedented-but-we-ve-been-here-before--v76ctolmr) · MIT Technology Review · 0 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#neural-networks](https://daily.dev/tags/neural-networks), [#ai-safety](https://daily.dev/tags/ai-safety) [View this post on daily.dev](https://daily.dev/posts/testing-robustness-against-unforeseen-adversaries-cvwkzgymk) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Testing robustness against unforeseen adversaries","url":"https://daily.dev/posts/testing-robustness-against-unforeseen-adversaries-cvwkzgymk","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/testing-robustness-against-unforeseen-adversaries-cvwkzgymk"},"datePublished":"2026-07-09T18:39:12.027Z","dateModified":"2026-07-09T19:28:02.685Z","description":"OpenAI researchers introduce UAR (Unforeseen Attack Robustness), a new metric and methodology for evaluating how well neural network classifiers hold up...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/03396eb460131544d1824bfe216cf4f9?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/03396eb460131544d1824bfe216cf4f9?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"OpenAI","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"OpenAI","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/984b3632316f4afaa3d5213d5c64fe65","url":"https://daily.dev/sources/openai"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/testing-robustness-against-unforeseen-adversaries-cvwkzgymk","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,neural-networks,ai-safety","timeRequired":"PT5M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"OpenAI","item":"https://daily.dev/sources/openai"},{"@type":"ListItem","position":3,"name":"Testing robustness against unforeseen adversaries"}]} ```