Six automatic optimization methods for LLM systems are compared, all sharing a common loop: an LLM proposes a change, an evaluator scores it, and the best changes are kept. The methods differ in what they optimize and what feedback they use. OPRO (Google DeepMind) treats the LLM as the optimizer using a leaderboard of past prompts. MIPROv2 (DSPy) jointly tunes instructions and few-shot examples via Bayesian search. TextGrad (Stanford) applies backpropagation-style natural-language criticism through a computation graph. GEPA (Berkeley) reads full execution traces to diagnose failures and maintain a Pareto frontier of candidates. AlphaEvolve (DeepMind) evolves code using two Gemini models and automated evaluators, producing results like a faster matrix multiplication algorithm. AutoResearch (Karpathy) runs a coding agent on ML training scripts, committing only improvements and using git as a log. Each method has distinct tradeoffs in compute cost, data requirements, and applicability.
