One kernel feature, 93% system throughput gone: A Red Hat Enterprise Linux 10.2 kernel regression and how to mitigate it

A kernel scheduler regression in RHEL 10.2 caused up to 93% throughput loss on CockroachDB workloads running on multi-NUMA AWS instances. The regression is bimodal — systems either run at full speed or enter a severe CPU work starvation state where CPUs sit 85-95% idle despite runnable threads waiting. The culprit is the NI_RANDOM scheduler feature, introduced to reduce overhead from failed newidle balance attempts, which inadvertently starves runnable threads on multi-NUMA hardware. The fix requires no reboot: disabling NI_RANDOM via a single sysfs write restores full throughput. A tmpfiles drop-in can persist the setting across reboots. The investigation involved vmstat analysis, thread stack traces, schedstat data, and targeted kernel reverts using Red Hat's kbisect-dbs tool to narrow 18,848 commits down to the responsible scheduler changes.