JetBrains Research introduces Step Rejection Fine-Tuning (SRFT), a technique to extract learning signal from failed LLM agent trajectories. Standard Rejection-sampling Fine-Tuning (RFT) discards entire runs that don't produce a correct final answer, wasting up to 61% of training data. A manual analysis of failed runs found that only ~24% of steps in failed trajectories are actually harmful — the rest are productive. SRFT uses a cheap LLM critic to label each step in unsuccessful trajectories as good, unnecessary, mistake, or recover, then masks the loss on mistake steps during training. This lets the model learn useful behaviors from failed runs without reinforcing bad habits. Experiments on SWE-bench Verified with Qwen2.5-Coder-32B show SRFT achieves 32.2% resolution vs. 30.9% for standard RFT, and even training solely on masked unsuccessful trajectories (29.7%) outperforms naively mixing resolved and unresolved data (28.5%). The paper was accepted to the DL4C workshop at ICML 2025.
