JetBrains has released the Kotlin Benchmark, an official open benchmark for evaluating AI coding agents on real-world Kotlin software engineering tasks. Based on the SWE-bench methodology, it features 105 tasks sourced from active open-source Kotlin repositories. Solutions are verified in containerized environments and must pass test validation. Initial results show Claude Code with Opus 4.7 xhigh leading at 85.71% resolution rate, followed by JetBrains Junie and Codex. All datasets and test harnesses are publicly available on GitHub, with a leaderboard at kotlinlang.org/benchmark. Future plans include broader Kotlin ecosystem coverage (Android, Kotlin Multiplatform), additional evaluation metrics beyond test passing, and more agent/model configurations.
