<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/quoka-query-oriented-kv-selection-for-efficient-llm-prefill-tylqnjutl" -->

---
title: QUOKA: Query-Oriented KV Selection for Efficient LLM Prefill
description: An explainer walks through QUOKA, a query-oriented key-value selection method for making long-context LLM prefill more efficient. It builds up background on KV...
canonical: https://daily.dev/posts/quoka-query-oriented-kv-selection-for-efficient-llm-prefill-tylqnjutl
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: QUOKA: Query-Oriented KV Selection for Efficient LLM Prefill | daily.dev
og:description: An explainer walks through QUOKA, a query-oriented key-value selection method for making long-context LLM prefill more efficient. It builds up background on KV...
og:url: https://daily.dev/posts/quoka-query-oriented-kv-selection-for-efficient-llm-prefill-tylqnjutl
og:image: https://api.daily.dev/og/posts/TyLqnjuTL.png
og:image:alt: QUOKA: Query-Oriented KV Selection for Efficient LLM Prefill
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# QUOKA: Query-Oriented KV Selection for Efficient LLM Prefill

**[Medium](https://daily.dev/sources/medium_js)** · 12 min read · 0 upvotes · 0 comments

## Summary

An explainer walks through QUOKA, a query-oriented key-value selection method for making long-context LLM prefill more efficient. It builds up background on KV caching, the prefill vs. decode phases, and chunked prefill, then explains QUOKA's core mechanism: selecting diverse queries (those least similar to the mean query via cosine similarity), scoring historical keys against those queries, taking the maximum score across queries to avoid washing out rare-but-important signals, and keeping only top-scoring KV pairs for sparse attention. It also covers how QUOKA handles Grouped-Query Attention via a pre-aggregation trick that averages query heads sharing a KV head before scoring, reducing computation without replacing the original query heads used in actual attention.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://medium.com/@saba.anjumeng/understanding-quoka-making-long-context-llm-prefill-more-efficient-9ae71bd7cfeb>

## Questions this post answers

### How does QUOKA decide which historical KV cache entries to keep during long-context prefill?

QUOKA first computes the mean query of the current chunk, then selects the queries with the lowest cosine similarity to that mean, since those represent unusual information needs the majority would overlook. These selected queries score historical keys via cosine similarity, and each key's final importance score is the maximum across all selected queries, preserving strong single-query interactions before keeping only the top-scoring keys and their values.

_daily.dev surfaces research like this for engineers optimizing long-context inference pipelines._

### Why does QUOKA use maximum instead of average when combining multiple query scores for a historical key?

Averaging scores from multiple queries can hide that one query considers a key extremely important, since a moderate average score of 0.40 could actually be masking a single query's score of 0.90. QUOKA takes the maximum across selected queries instead, so a historical key is kept if even one representative query strongly needs it, rather than letting other queries dilute that signal.

_daily.dev helps developers tracking sparse attention and KV cache selection techniques stay current._

### How does QUOKA handle Grouped-Query Attention where multiple query heads share one KV head?

QUOKA uses a pre-aggregation trick: instead of scoring each query head against the shared key head separately and then averaging, it averages the normalized query head representations first and performs a single multiplication, exploiting the mathematical identity (Q1K^T + Q2K^T)/2 = ((Q1+Q2)/2)K^T. This averaged query is used only for KV selection, not for the actual attention computation, which still uses the original query heads.

_daily.dev keeps GQA and attention-efficiency researchers informed on techniques like this one._

## Similar posts on daily.dev

- [KV Cache Engineering for LLM Serving](https://daily.dev/posts/kv-cache-engineering-for-llm-serving-l5oenhwpt) · Daily Dose of Data Science \| Avi Chawla \| Substack · 0 upvotes · 0 comments
- [How LLMs Decide What to Forget: KV Cache Eviction Explained](https://daily.dev/posts/how-llms-decide-what-to-forget-kv-cache-eviction-explained-k6kfg62tc) · Medium · 2 upvotes · 0 comments
- [KV Caching in LLMs: A Guide for Developers](https://daily.dev/posts/kv-caching-in-llms-a-guide-for-developers-hfpim4eo0) · Machine Learning Mastery · 1 upvotes · 0 comments

---

Tags: [#ai-inference](https://daily.dev/tags/ai-inference)

[View this post on daily.dev](https://daily.dev/posts/quoka-query-oriented-kv-selection-for-efficient-llm-prefill-tylqnjutl)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"TechArticle","headline":"QUOKA: Query-Oriented KV Selection for Efficient LLM Prefill","url":"https://daily.dev/posts/quoka-query-oriented-kv-selection-for-efficient-llm-prefill-tylqnjutl","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/quoka-query-oriented-kv-selection-for-efficient-llm-prefill-tylqnjutl"},"datePublished":"2026-09-09T18:04:41.399Z","dateModified":"2026-09-09T18:43:12.335Z","description":"An explainer walks through QUOKA, a query-oriented key-value selection method for making long-context LLM prefill more efficient. It builds up background on KV...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e78e5ee688f5c84b07ff7bf10ecd39bb?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e78e5ee688f5c84b07ff7bf10ecd39bb?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Medium","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Medium","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/medium","url":"https://daily.dev/sources/medium_js"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/quoka-query-oriented-kv-selection-for-efficient-llm-prefill-tylqnjutl","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-inference","timeRequired":"PT12M"}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Medium","item":"https://daily.dev/sources/medium_js"},{"@type":"ListItem","position":3,"name":"QUOKA: Query-Oriented KV Selection for Efficient LLM Prefill"}]}
{"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/quoka-query-oriented-kv-selection-for-efficient-llm-prefill-tylqnjutl#faq","mainEntity":[{"@type":"Question","name":"How does QUOKA decide which historical KV cache entries to keep during long-context prefill?","acceptedAnswer":{"@type":"Answer","text":"QUOKA first computes the mean query of the current chunk, then selects the queries with the lowest cosine similarity to that mean, since those represent unusual information needs the majority would overlook. These selected queries score historical keys via cosine similarity, and each key's final importance score is the maximum across all selected queries, preserving strong single-query interactions before keeping only the top-scoring keys and their values. daily.dev surfaces research like this for engineers optimizing long-context inference pipelines."}},{"@type":"Question","name":"Why does QUOKA use maximum instead of average when combining multiple query scores for a historical key?","acceptedAnswer":{"@type":"Answer","text":"Averaging scores from multiple queries can hide that one query considers a key extremely important, since a moderate average score of 0.40 could actually be masking a single query's score of 0.90. QUOKA takes the maximum across selected queries instead, so a historical key is kept if even one representative query strongly needs it, rather than letting other queries dilute that signal. daily.dev helps developers tracking sparse attention and KV cache selection techniques stay current."}},{"@type":"Question","name":"How does QUOKA handle Grouped-Query Attention where multiple query heads share one KV head?","acceptedAnswer":{"@type":"Answer","text":"QUOKA uses a pre-aggregation trick: instead of scoring each query head against the shared key head separately and then averaging, it averages the normalized query head representations first and performs a single multiplication, exploiting the mathematical identity (Q1K^T + Q2K^T)/2 = ((Q1+Q2)/2)K^T. This averaged query is used only for KV selection, not for the actual attention computation, which still uses the original query heads. daily.dev keeps GQA and attention-efficiency researchers informed on techniques like this one."}}]}
```

