--- title: Keeping Futhark off the GPU | daily.dev description: A compiler engineer describes a performance problem in the Futhark data-parallel language: all arrays get placed in GPU memory by default, which is disastrous... canonical: https://daily.dev/posts/keeping-futhark-off-the-gpu-d7p7mkzuj twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Keeping Futhark off the GPU | daily.dev og:description: A compiler engineer describes a performance problem in the Futhark data-parallel language: all arrays get placed in GPU memory by default, which is disastrous... og:url: https://daily.dev/posts/keeping-futhark-off-the-gpu-d7p7mkzuj og:image: https://api.daily.dev/og/posts/d7P7MkzUJ.png og:image:alt: Keeping Futhark off the GPU og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Keeping Futhark off the GPU **[Lobsters](https://daily.dev/sources/lobsters)** · 8 min read · 0 upvotes · 0 comments ## Summary A compiler engineer describes a performance problem in the Futhark data-parallel language: all arrays get placed in GPU memory by default, which is disastrous for inherently sequential algorithms like a distance-2 graph colouring step needed before a GPU-parallel Jacobian computation. The post walks through why Futhark's compilation model causes ruinous per-element GPU-to-CPU communication overhead, sketches a principled fix (deciding memory placement based on array usage) that was deemed too complex to implement right now, and describes the actual shipped workaround: a new #[cpu_function] attribute that forces a function's body and intermediate arrays onto the CPU while still returning results in GPU memory via a bulk copy. Several implementation bugs this surfaced (GPU kernel codegen assuming all arrays are in GPU memory, iota/replicate primitives) were patched. The hack is coarse but effective since colouring is now a tiny fraction of total runtime. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why does putting a sequential loop in Futhark on the GPU backend make it extremely slow? Because Futhark's GPU backends place all arrays in GPU memory, so any CPU-side indexing into that array inside a loop triggers a costly GPU memory transfer per access, with overhead of roughly a couple of microseconds per element. For a sequential algorithm like distance-2 graph colouring, this made the GPU backend three or more orders of magnitude slower than the sequential C backend. _Anyone tuning mixed CPU/GPU workloads can track these compiler-level tradeoffs on daily.dev._ ### How can I force part of a Futhark program to run on the CPU while the rest runs on the GPU? Use the #[cpu_function] attribute on a non-inlined Futhark function, which compiles its body to CPU code and stores its input and intermediate arrays in CPU memory, while copying the final result back to GPU memory so the rest of the program can continue treating everything as GPU-resident. It is a coarse workaround rather than a built-in separate-compilation feature. _Developers mixing sequential and parallel code paths can follow this kind of attribute-based workaround on daily.dev._ ## Similar posts on daily.dev - [Full flattening of nested data parallelism](https://daily.dev/posts/full-flattening-of-nested-data-parallelism-f2cwulaod) · Lobsters · 0 upvotes · 0 comments - [Futhark by Example](https://daily.dev/posts/futhark-by-example-vnchxphd7) · Hacker News · 0 upvotes · 0 comments - [I Put My Rust Code on a GPU. The Profiler Told Me to Stop.](https://daily.dev/posts/i-put-my-rust-code-on-a-gpu-the-profiler-told-me-to-stop--6e3tfkih6) · Medium · 0 upvotes · 0 comments - [Offloading Rust To GPUs Proves Capable Of High Performance With Memory Safety](https://daily.dev/posts/offloading-rust-to-gpus-proves-capable-of-high-performance-with-memory-safety-6ojskq9d9) · Phoronix · 0 upvotes · 0 comments - [Are arrays functions?](https://daily.dev/posts/are-arrays-functions--d6pjhqkpv) · Lobsters · 0 upvotes · 0 comments --- Tags: [#gpu](https://daily.dev/tags/gpu), [#compiler](https://daily.dev/tags/compiler) [View this post on daily.dev](https://daily.dev/posts/keeping-futhark-off-the-gpu-d7p7mkzuj) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Keeping Futhark off the GPU","url":"https://daily.dev/posts/keeping-futhark-off-the-gpu-d7p7mkzuj","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/keeping-futhark-off-the-gpu-d7p7mkzuj"},"datePublished":"2026-10-03T12:14:26.388Z","dateModified":"2026-10-03T12:14:47.395Z","description":"A compiler engineer describes a performance problem in the Futhark data-parallel language: all arrays get placed in GPU memory by default, which is disastrous...","image":"https://media.daily.dev/image/upload/s--HRgLpUt6--/f_auto/v1722860399/public/Placeholder%2003","thumbnailUrl":"https://media.daily.dev/image/upload/s--HRgLpUt6--/f_auto/v1722860399/public/Placeholder%2003","isAccessibleForFree":true,"articleSection":"Lobsters","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Lobsters","logo":"https://media.daily.dev/image/upload/s--tl8v_Fku--/f_auto,t_logo/v1698841318/logos/lobste.jpg","url":"https://daily.dev/sources/lobsters"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/keeping-futhark-off-the-gpu-d7p7mkzuj","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"gpu,compiler","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Lobsters","item":"https://daily.dev/sources/lobsters"},{"@type":"ListItem","position":3,"name":"Keeping Futhark off the GPU"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/keeping-futhark-off-the-gpu-d7p7mkzuj#faq","mainEntity":[{"@type":"Question","name":"Why does putting a sequential loop in Futhark on the GPU backend make it extremely slow?","acceptedAnswer":{"@type":"Answer","text":"Because Futhark's GPU backends place all arrays in GPU memory, so any CPU-side indexing into that array inside a loop triggers a costly GPU memory transfer per access, with overhead of roughly a couple of microseconds per element. For a sequential algorithm like distance-2 graph colouring, this made the GPU backend three or more orders of magnitude slower than the sequential C backend. Anyone tuning mixed CPU/GPU workloads can track these compiler-level tradeoffs on daily.dev."}},{"@type":"Question","name":"How can I force part of a Futhark program to run on the CPU while the rest runs on the GPU?","acceptedAnswer":{"@type":"Answer","text":"Use the #[cpu_function] attribute on a non-inlined Futhark function, which compiles its body to CPU code and stores its input and intermediate arrays in CPU memory, while copying the final result back to GPU memory so the rest of the program can continue treating everything as GPU-resident. It is a coarse workaround rather than a built-in separate-compilation feature. Developers mixing sequential and parallel code paths can follow this kind of attribute-based workaround on daily.dev."}}]} ```