--- title: How Work is Organized Inside a GPU | daily.dev description: A newsletter covering three topics: a new open-weights JSON extraction model called Lift from datalab that converts documents into structured JSON in one step;... canonical: https://daily.dev/posts/how-work-is-organized-inside-a-gpu-jugfqh8ty twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How Work is Organized Inside a GPU | daily.dev og:description: A newsletter covering three topics: a new open-weights JSON extraction model called Lift from datalab that converts documents into structured JSON in one step;... og:url: https://daily.dev/posts/how-work-is-organized-inside-a-gpu-jugfqh8ty og:image: https://api.daily.dev/og/posts/jugFQH8tY.png og:image:alt: How Work is Organized Inside a GPU og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How Work is Organized Inside a GPU **[Daily Dose of Data Science \| Avi Chawla \| Substack](https://daily.dev/sources/dailydoseofds)** · 6 min read · 0 upvotes · 0 comments ## Summary A newsletter covering three topics: a new open-weights JSON extraction model called Lift from datalab that converts documents into structured JSON in one step; a visual explainer of how GPUs organize work, tracing the path from kernel to grid, thread blocks, warps, and SM scheduling, and why this hierarchy enables latency hiding and parallelism; and a comparison of three ways LLMs produce decisions - normal decoding, structured/schema-constrained decoding, and Jev-style scoring, which directly scores a fixed set of candidate answers instead of generating text. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How does a GPU organize work from a kernel launch down to actual execution on hardware? A kernel launch creates a grid representing the full workload, which is divided into thread blocks assigned to streaming multiprocessors (SMs). Each block is split into warps of 32 threads that execute the same instruction together. The SM switches between resident warps to hide memory latency, which is why having enough blocks and warps matters for keeping compute units busy. _Developers optimizing kernel launch configs can dig deeper into GPU scheduling behavior on daily.dev._ ### What is Jev-style scoring and how does it differ from structured output decoding in LLMs? Jev-style scoring evaluates a fixed set of predefined candidate answers and returns their probabilities directly, without generating any tokens, making it suited for routing, classification, or ranking tasks with known answers. Structured output decoding still generates a response token by token while a schema masks invalid tokens, which is better when values or nested fields aren't known in advance. _Teams choosing between decoding approaches for classification tasks can explore comparisons like this on daily.dev._ ### What is the Lift model from datalab and how accurate is it at JSON extraction from documents? Lift is an open-weights 9B parameter model that extracts structured JSON directly from a PDF or image given a schema, in a single step without separate OCR and repair stages. It achieves 90.2% field accuracy across 11,000 fields, nearly matches Gemini Flash 3.5 while running about 3x faster, returns null for missing fields instead of hallucinating, and is self-hostable via vLLM or Hugging Face. _Developers building document extraction pipelines can track new open-weights models like this via daily.dev._ ## Similar posts on daily.dev - [Your GPU Is a Kitchen With One Tiny Door](https://daily.dev/posts/your-gpu-is-a-kitchen-with-one-tiny-door-rbbxjvpyf) · Medium · 1 upvotes · 0 comments - [How a GPU Actually Works](https://daily.dev/posts/how-a-gpu-actually-works-t655zbnjv) · Daily Dose of Data Science \| Avi Chawla \| Substack · 0 upvotes · 0 comments - [AI Model Co-Design: Hardware-Friendly LLM Design](https://daily.dev/posts/ai-model-co-design-hardware-friendly-llm-design-8rbz8uqxh) · NVIDIA Developer · 0 upvotes · 0 comments - [Inside NVIDIA GPUs: Anatomy of high performance matmul kernels](https://daily.dev/posts/inside-nvidia-gpus-anatomy-of-high-performance-matmul-kernels-opvj5jeqt) · Programming Digest · 1 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#ai-inference](https://daily.dev/tags/ai-inference), [#cuda](https://daily.dev/tags/cuda) [View this post on daily.dev](https://daily.dev/posts/how-work-is-organized-inside-a-gpu-jugfqh8ty) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How Work is Organized Inside a GPU","url":"https://daily.dev/posts/how-work-is-organized-inside-a-gpu-jugfqh8ty","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-work-is-organized-inside-a-gpu-jugfqh8ty"},"datePublished":"2026-10-03T21:57:56.469Z","dateModified":"2026-10-03T21:58:18.466Z","description":"A newsletter covering three topics: a new open-weights JSON extraction model called Lift from datalab that converts documents into structured JSON in one step;...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7a72f2408794f1a4e4a3a70d2ef9d2d1?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7a72f2408794f1a4e4a3a70d2ef9d2d1?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Daily Dose of Data Science | Avi Chawla | Substack","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Daily Dose of Data Science | Avi Chawla | Substack","logo":"https://media.daily.dev/image/upload/s--4IHQgTOw--/f_auto/v1710503712/logos/dailydoseofds","url":"https://daily.dev/sources/dailydoseofds"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-work-is-organized-inside-a-gpu-jugfqh8ty","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,ai-inference,cuda","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Daily Dose of Data Science | Avi Chawla | Substack","item":"https://daily.dev/sources/dailydoseofds"},{"@type":"ListItem","position":3,"name":"How Work is Organized Inside a GPU"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-work-is-organized-inside-a-gpu-jugfqh8ty#faq","mainEntity":[{"@type":"Question","name":"How does a GPU organize work from a kernel launch down to actual execution on hardware?","acceptedAnswer":{"@type":"Answer","text":"A kernel launch creates a grid representing the full workload, which is divided into thread blocks assigned to streaming multiprocessors (SMs). Each block is split into warps of 32 threads that execute the same instruction together. The SM switches between resident warps to hide memory latency, which is why having enough blocks and warps matters for keeping compute units busy. Developers optimizing kernel launch configs can dig deeper into GPU scheduling behavior on daily.dev."}},{"@type":"Question","name":"What is Jev-style scoring and how does it differ from structured output decoding in LLMs?","acceptedAnswer":{"@type":"Answer","text":"Jev-style scoring evaluates a fixed set of predefined candidate answers and returns their probabilities directly, without generating any tokens, making it suited for routing, classification, or ranking tasks with known answers. Structured output decoding still generates a response token by token while a schema masks invalid tokens, which is better when values or nested fields aren't known in advance. Teams choosing between decoding approaches for classification tasks can explore comparisons like this on daily.dev."}},{"@type":"Question","name":"What is the Lift model from datalab and how accurate is it at JSON extraction from documents?","acceptedAnswer":{"@type":"Answer","text":"Lift is an open-weights 9B parameter model that extracts structured JSON directly from a PDF or image given a schema, in a single step without separate OCR and repair stages. It achieves 90.2% field accuracy across 11,000 fields, nearly matches Gemini Flash 3.5 while running about 3x faster, returns null for missing fields instead of hallucinating, and is self-hostable via vLLM or Hugging Face. Developers building document extraction pipelines can track new open-weights models like this via daily.dev."}}]} ```