<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/reversal-bench-a-reversibility-axis-and-reset-oracle-for-measuring-the-reset-free-rl-cliff-cikotp2lw" -->

---
title: REVERSAL-BENCH: A Reversibility Axis and Reset Oracle...
description: Apple researchers introduce REVERSAL-BENCH, a benchmark measuring how reset-free reinforcement learning agents fail as environmental irreversibility increases....
canonical: https://daily.dev/posts/reversal-bench-a-reversibility-axis-and-reset-oracle-for-measuring-the-reset-free-rl-cliff-cikotp2lw
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: REVERSAL-BENCH: A Reversibility Axis and Reset Oracle for Measuring the Reset-Free RL Cliff | daily.dev
og:description: Apple researchers introduce REVERSAL-BENCH, a benchmark measuring how reset-free reinforcement learning agents fail as environmental irreversibility increases....
og:url: https://daily.dev/posts/reversal-bench-a-reversibility-axis-and-reset-oracle-for-measuring-the-reset-free-rl-cliff-cikotp2lw
og:image: https://api.daily.dev/og/posts/ciKoTp2lw.png
og:image:alt: REVERSAL-BENCH: A Reversibility Axis and Reset Oracle for Measuring the Reset-Free RL Cliff
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# REVERSAL-BENCH: A Reversibility Axis and Reset Oracle for Measuring the Reset-Free RL Cliff

**[Apple Machine Learning Research](https://daily.dev/sources/mlapple)** · 2 min read · 0 upvotes · 0 comments

## Summary

Apple researchers introduce REVERSAL-BENCH, a benchmark measuring how reset-free reinforcement learning agents fail as environmental irreversibility increases. Using a tunable reversibility parameter and a ground-truth reset oracle across eight manipulation tasks in five physics engines, the benchmark reveals that reset-free agents get permanently trapped in unrecoverable states while episodic agents keep learning steadily. The study isolates irreversibility itself, not task complexity, as the cause, and tests a safety shield that predicts recoverability well but only prevents failure when the agent can physically avoid the trap. The benchmark suite, dataset, and reset oracle are being released publicly.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://machinelearning.apple.com/research/reversal-bench-rl-cliff>

## Questions this post answers

### What is the reversibility cliff in reset-free reinforcement learning?

It is a sharp failure mode where reset-free RL agents get permanently absorbed into irrecoverable states as environmental irreversibility increases, causing learning to halt entirely. Episodic agents with external resets do not show this failure and keep learning steadily. Testing across eight manipulation settings and five physics engines shows the breakdown is caused by irreversibility itself, not by obstacle or task complexity.

_Anyone building resilient RL agents can find related benchmarking research curated on daily.dev._

### How does REVERSAL-BENCH measure environment irreversibility for RL agents?

It controls reversibility through a continuous parameter denoted rho, ranging from 0 to 1, paired with a reset oracle that provides ground-truth verification of whether a state can still be recovered. The benchmark covers eight manipulation settings across five physics engines and includes a released multi-simulator dataset labeled with recoverability data alongside the oracle itself.

_Researchers comparing RL benchmarks can track new evaluation methods like this via daily.dev._

### Can a safety shield prevent RL agents from entering irreversible failure states?

A tested safety shield can accurately predict when a state is becoming irrecoverable, but active recovery only succeeds when the agent still has a physical path to steer away from the trap. Once the agent is geometrically committed to an irreversible transition, prediction alone does not prevent the failure, since it lacks a physical route to escape.

_Engineers evaluating safety mechanisms for autonomous agents follow findings like this on daily.dev._

## Similar posts on daily.dev

- [I Ran 392 Checks. Two AI Forecasts Still Flipped.](https://daily.dev/posts/i-ran-392-checks-two-ai-forecasts-still-flipped--wakblp2i8) · Medium · 0 upvotes · 0 comments
- [The Inversion Error: Why Safe AGI Requires an Enactive Floor and State-Space Reversibility](https://daily.dev/posts/the-inversion-error-why-safe-agi-requires-an-enactive-floor-and-state-space-reversibility-1tq03ycwr) · Towards Data Science · 0 upvotes · 0 comments
- [Your Agent Aced the Task. Will It Do It Again?](https://daily.dev/posts/your-agent-aced-the-task-will-it-do-it-again--yetkhtezb) · Hugging Face · 0 upvotes · 0 comments
- [I ran 252 questions through three local LLMs, and a messy reasoning trace says nothing about the answer](https://daily.dev/posts/i-ran-252-questions-through-three-local-llms-and-a-messy-reasoning-trace-says-nothing-about-the-ans-exje8b9yb) · XDA Developers · 1 upvotes · 0 comments

---

Tags: [#apple](https://daily.dev/tags/apple), [#robotics](https://daily.dev/tags/robotics), [#reinforcement-learning](https://daily.dev/tags/reinforcement-learning)

[View this post on daily.dev](https://daily.dev/posts/reversal-bench-a-reversibility-axis-and-reset-oracle-for-measuring-the-reset-free-rl-cliff-cikotp2lw)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"TechArticle","headline":"REVERSAL-BENCH: A Reversibility Axis and Reset Oracle for Measuring the Reset-Free RL Cliff","url":"https://daily.dev/posts/reversal-bench-a-reversibility-axis-and-reset-oracle-for-measuring-the-reset-free-rl-cliff-cikotp2lw","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/reversal-bench-a-reversibility-axis-and-reset-oracle-for-measuring-the-reset-free-rl-cliff-cikotp2lw"},"datePublished":"2026-09-17T15:09:20.399Z","dateModified":"2026-09-17T16:28:45.620Z","description":"Apple researchers introduce REVERSAL-BENCH, a benchmark measuring how reset-free reinforcement learning agents fail as environmental irreversibility increases....","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/8dda20dea5f9a8a0e25a1b7bb36a1a20?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/8dda20dea5f9a8a0e25a1b7bb36a1a20?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Apple Machine Learning Research","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Apple Machine Learning Research","logo":"https://media.daily.dev/image/upload/s--mD-1uSe0--/f_auto,q_auto/v1785654191/logos/mlapple","url":"https://daily.dev/sources/mlapple"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/reversal-bench-a-reversibility-axis-and-reset-oracle-for-measuring-the-reset-free-rl-cliff-cikotp2lw","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"apple,robotics,reinforcement-learning","timeRequired":"PT2M"}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Apple Machine Learning Research","item":"https://daily.dev/sources/mlapple"},{"@type":"ListItem","position":3,"name":"REVERSAL-BENCH: A Reversibility Axis and Reset Oracle for Measuring the Reset-Free RL Cliff"}]}
{"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/reversal-bench-a-reversibility-axis-and-reset-oracle-for-measuring-the-reset-free-rl-cliff-cikotp2lw#faq","mainEntity":[{"@type":"Question","name":"What is the reversibility cliff in reset-free reinforcement learning?","acceptedAnswer":{"@type":"Answer","text":"It is a sharp failure mode where reset-free RL agents get permanently absorbed into irrecoverable states as environmental irreversibility increases, causing learning to halt entirely. Episodic agents with external resets do not show this failure and keep learning steadily. Testing across eight manipulation settings and five physics engines shows the breakdown is caused by irreversibility itself, not by obstacle or task complexity. Anyone building resilient RL agents can find related benchmarking research curated on daily.dev."}},{"@type":"Question","name":"How does REVERSAL-BENCH measure environment irreversibility for RL agents?","acceptedAnswer":{"@type":"Answer","text":"It controls reversibility through a continuous parameter denoted rho, ranging from 0 to 1, paired with a reset oracle that provides ground-truth verification of whether a state can still be recovered. The benchmark covers eight manipulation settings across five physics engines and includes a released multi-simulator dataset labeled with recoverability data alongside the oracle itself. Researchers comparing RL benchmarks can track new evaluation methods like this via daily.dev."}},{"@type":"Question","name":"Can a safety shield prevent RL agents from entering irreversible failure states?","acceptedAnswer":{"@type":"Answer","text":"A tested safety shield can accurately predict when a state is becoming irrecoverable, but active recovery only succeeds when the agent still has a physical path to steer away from the trap. Once the agent is geometrically committed to an irreversible transition, prediction alone does not prevent the failure, since it lacks a physical route to escape. Engineers evaluating safety mechanisms for autonomous agents follow findings like this on daily.dev."}}]}
```

