The Missing Piece of Agent Self-Improvement
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
Most AI agents don't learn from production failures — bugs get patched manually and the same class of mistake can recur. This post introduces Hermes, an open-source agent that saves successful strategies as reusable SKILL.md files and uses GEPA (Genetic-Pareto Prompt Evolution) to refine prompts offline without GPU fine-tuning. It then explains how Opik, an open-source observability platform, closes the production gap Hermes leaves open. Opik's workflow chains four layers: automatic tracing of every LLM call and tool invocation, Ollie (a built-in coding agent that diagnoses failures and proposes Git-style diffs for approval), plain-English test suites evaluated via LLM-as-a-judge, and an Agent Sandbox for end-to-end reruns. Together, Hermes learns from successes while Opik systematically converts production failures into diagnosed, verified, and locked-in regression tests. Step-by-step setup instructions are included for Linux, macOS, and WSL2 with Python 3.11+.
Table of contents
Postgres for time-series workloads at any scaleThe missing piece of agent self-improvementQuestions this post answers
What is GEPA in the Hermes agent and how does it improve prompts without retraining?
GEPA (Genetic-Pareto Prompt Evolution) is an offline optimization process in Hermes that analyzes execution traces from agent runs and keeps the prompt variants that perform best. It is an evolutionary search, not a training run, so it requires no GPU fine-tuning. It can improve prompts, skills, and tool descriptions purely by selecting better-performing variants from collected traces. Teams iterating on agent prompts without retraining track approaches like GEPA on daily.dev.
How does Opik's Ollie agent diagnose failures in an AI agent trace?
Ollie reads the span tree of an Opik trace, following every LLM call and tool invocation to identify the actual failure point. It can then inspect local source files via a single `opik connect` command, identify the responsible code, and propose a fix as a Git-style diff. Crucially, nothing is applied automatically — every proposed change waits for explicit developer approval. Developers debugging agent failures in production find the latest observability tooling on daily.dev.
What are the known limitations of Hermes agent self-improvement?
Hermes has three documented limitations: self-bias (it can rate its own weak solutions highly, risking low-quality skills being saved), skill overwrite (auto-generated skills can replace better hand-crafted ones), and offline GEPA (optimization only runs after someone manually collects traces and reruns the process, so real-time production failures are not automatically fed back into the learning loop). Keeping up with agent framework trade-offs like these is easier when daily.dev surfaces the relevant posts for you.