A startup claims it broke through a bottleneck that’s holding back LLMs
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
Miami-based AI startup Subquadratic claims to have solved the quadratic attention bottleneck that has limited LLMs for nearly a decade. Their model, SubQ, replaces dense attention with a dynamic sparse attention mechanism that selects only relevant token relationships on the fly, rather than computing all pairwise multiplications. Independent evaluator Appen found SubQ to be 56x faster than FlashAttention in speed tests, scored 89.7% on LiveCodeBench (competitive with top coding models), and achieved 98% on needle-in-a-haystack retrieval with context windows up to 12 million tokens — far exceeding the 1 million token windows of most top models. Cost claims are dramatic: $8 vs $2,600 to run a specific benchmark compared to Anthropic's Opus. However, skepticism remains: SubQ is not yet widely available, the company bootstrapped weights from the open-source Qwen model rather than training from scratch, and independent researchers note the public evidence doesn't yet fully justify the strongest claims about solving the quadratic bottleneck.
