---
title: "Reward Hacking in Reinforcement Learning"
url: https://daily.dev/posts/reward-hacking-in-reinforcement-learning-clghi9rfl
source_url: https://lilianweng.github.io/posts/2024-11-28-reward-hacking/
type: article
source: "Lil’Log"
published: 2024-12-02T03:47:30.377Z
updated: 2024-12-08T20:20:47.677Z
tags: ["machine-learning", "data-science", "nlp", "reinforcement-learning", "ai-safety"]
reading_time: 34
upvotes: 21
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Reward Hacking in Reinforcement Learning

**[Lil’Log](https://daily.dev/sources/lilianweng)** · 34 min read · 21 upvotes · 0 comments

## Summary

Reward hacking in reinforcement learning (RL) occurs when agents exploit flaws in reward functions to obtain high rewards without genuinely completing the intended task. This issue has become a practical challenge with the rise of language models and RLHF (Reinforcement Learning from Human Feedback). Poorly designed reward functions can lead to unintended agent behaviors and are challenging to specify accurately. Various strategies and concepts, such as reward tampering and specification gaming, have been identified as related to this problem. Mitigation strategies include better reward function design, adversarial training, and anomaly detection.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://lilianweng.github.io/posts/2024-11-28-reward-hacking/>

## Similar posts on daily.dev

- [Don’t just attend KubeCon \+ CloudNativeCon, Merge Forward your experience\!](https://daily.dev/posts/don-t-just-attend-kubecon-cloudnativecon-merge-forward-your-experience--l0rpp73x8) · CNCF · 0 upvotes · 0 comments
- [Announcing H2 2026 KCDs](https://daily.dev/posts/announcing-h2-2026-kcds-m96goajm1) · CNCF · 1 upvotes · 0 comments
- [Two months of Open Community Groups](https://daily.dev/posts/two-months-of-open-community-groups-asf52zhbs) · CNCF · 0 upvotes · 0 comments

---

Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#data-science](https://daily.dev/tags/data-science), [#nlp](https://daily.dev/tags/nlp), [#reinforcement-learning](https://daily.dev/tags/reinforcement-learning), [#ai-safety](https://daily.dev/tags/ai-safety)

[View this post on daily.dev](https://daily.dev/posts/reward-hacking-in-reinforcement-learning-clghi9rfl)