--- title: "Autoresearching BM25 on MSMarco" url: https://daily.dev/posts/autoresearching-bm25-on-msmarco-f7p9gsxkq source_url: http://softwaredoug.com/blog/2026/05/17/autoresearching-a-better-msmarco-bm25.html type: article source: "Software Doug" published: 2026-05-19T12:48:35.676Z updated: 2026-05-19T12:48:59.631Z tags: ["ai-agents"] reading_time: 8 upvotes: 0 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Autoresearching BM25 on MSMarco **[Software Doug](https://daily.dev/sources/softwaredoug)** · 8 min read · 0 upvotes · 0 comments ## Summary An experiment using an AI coding agent to iteratively improve BM25 search ranking on the MSMarco passage retrieval dataset. The agent starts with a baseline BM25 implementation and proposes code changes, accepting only those that improve NDCG on validation data. After 8 rounds, the agent discovered stopword removal for longer queries and a bigram phrase boost, achieving MRR near 0.2. However, gains plateaued due to overfitting to the minimarco sample — including odd stopwords like 'medicine' and 'vacation' that leaked from the validation set. The post reflects on lessons learned about data leakage in automated tuning and outlines future directions including better context management and using the full dataset. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Re-autoresearching MSMARCO BM25, on Vespa](https://daily.dev/posts/re-autoresearching-msmarco-bm25-on-vespa-4iv6vtfok) · Vespa Blog · 1 upvotes · 0 comments - [Can agents replace the search stack?](https://daily.dev/posts/can-agents-replace-the-search-stack--0ytkqcrrk) · Software Doug · 0 upvotes · 0 comments - [How I Reproduced BM25, Dense Retrieval, and SPLADE on a 16GB MacBook](https://daily.dev/posts/how-i-reproduced-bm25-dense-retrieval-and-splade-on-a-16gb-macbook-skiim8mno) · Towards Data Science · 0 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents) [View this post on daily.dev](https://daily.dev/posts/autoresearching-bm25-on-msmarco-f7p9gsxkq)