--- title: "How to Build an AI Agent That Runs its Own LLM Experiments with autoresearch" url: https://daily.dev/posts/how-to-build-an-ai-agent-that-runs-its-own-llm-experiments-with-autoresearch-brly8l88g source_url: https://www.freecodecamp.org/news/build-an-ai-agent-that-runs-its-own-llm-experiments-with-autoresearch type: article source: "freeCodeCamp" published: 2026-06-29T20:55:36.964Z updated: 2026-06-29T20:56:02.315Z tags: ["python", "ai-agents"] reading_time: 25 upvotes: 3 comments: 1 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How to Build an AI Agent That Runs its Own LLM Experiments with autoresearch **[freeCodeCamp](https://daily.dev/sources/freecodecamp)** · 25 min read · 3 upvotes · 1 comments ## Summary autoresearch is an open-source tool by Andrej Karpathy that lets an AI agent autonomously run LLM training experiments on a GPU. The agent edits a single training file (train.py), runs 5-minute training sessions, reads the validation bits-per-byte (val_bpb) metric, and decides whether to keep or revert each change — looping indefinitely. The 5-minute wall-clock budget prevents gaming via model size tricks. In Karpathy's depth-12 run, the agent found ~20 stacking improvements over ~700 experiments that transferred to a larger depth-24 model, cutting 'Time to GPT-2' by ~11%. The post covers the repo structure, how val_bpb works, the program.md playbook design, setup steps, and tuning tips for smaller GPUs. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community discussion Top comments from developers on daily.dev. **@kartiknvj** · 0 upvotes > An agent editing [train.py](http://train.py) and running its own experiments is exciting, and the part I'd watch closely is the stopping rule. Without a hard guard it can chase noise across runs and burn GPU budget optimizing for variance instead of a real gain. How does autoresearch decide a result is a genuine improvement versus run-to-run seed luck, does it repeat seeds before accepting a change? --- Tags: [#python](https://daily.dev/tags/python), [#ai-agents](https://daily.dev/tags/ai-agents) [View this post on daily.dev](https://daily.dev/posts/how-to-build-an-ai-agent-that-runs-its-own-llm-experiments-with-autoresearch-brly8l88g)