--- title: Cactus Needle - The 26M Function Calling Model | daily.dev description: Cactus Needle is an open-source 26 million parameter model purpose-built for function calling, distilled from Gemini Flash. Unlike standard LLMs, it uses a... canonical: https://daily.dev/posts/cactus-needle---the-26m-function-calling-model-wel3ueg7j twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Cactus Needle - The 26M Function Calling Model | daily.dev og:description: Cactus Needle is an open-source 26 million parameter model purpose-built for function calling, distilled from Gemini Flash. Unlike standard LLMs, it uses a... og:url: https://daily.dev/posts/cactus-needle---the-26m-function-calling-model-wel3ueg7j og:image: https://api.daily.dev/og/posts/wel3UEg7j.png og:image:alt: Cactus Needle - The 26M Function Calling Model og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Cactus Needle - The 26M Function Calling Model **[Sam Witteveen](https://daily.dev/sources/samwitteveenai)** · 14 min read · 2 upvotes · 0 comments ## Summary Cactus Needle is an open-source 26 million parameter model purpose-built for function calling, distilled from Gemini Flash. Unlike standard LLMs, it uses a simple attention network with no FFN/MLP layers, removing the knowledge-storage components to achieve extreme compactness. Pre-trained on 200B tokens using 16 TPUs and post-trained on 2B synthetic function-calling tokens, it outperforms models 10-20x its size on single-shot function calling benchmarks. It can run at up to 6,000 tokens/second on consumer devices and can even be fine-tuned on a CPU. The model is available on HuggingFace with a GitHub repo including JAX/Flax pre-training code. Cactus, a YC startup, positions this for edge deployments — smart home devices, phones, Raspberry Pi — where inference cost is near zero and simple tool dispatch is all that's needed. Limitations include poor performance on parallel or reasoning-heavy function calls. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is Cactus Needle and how many parameters does it have? Cactus Needle is an open-source function calling model with only 26 million parameters, built by the YC startup Cactus. It uses a 'simple attention network' architecture that removes all feed-forward and MLP layers, relying purely on attention and gating, and claims up to 6,000 tokens/second decode and 1,200 tokens/second prefill on consumer CPU hardware. _Developers experimenting with tiny edge models for tool calling can track releases like this one on daily.dev._ ### How was Cactus Needle trained and what data was used? Cactus Needle was pre-trained on 200 billion tokens using only 16 TPUs for 27 hours, then post-trained on 2 billion tokens of synthetic function calling data spanning 15 tool categories. The team also applied quantization-aware training so the model retains accuracy after being quantized to an even smaller footprint. _Anyone comparing efficient training recipes for small models can follow developments like this via daily.dev._ ### Can Cactus Needle be fine-tuned without a GPU? Yes, Cactus Needle is small enough at 26 million parameters that it can be fine-tuned entirely on a CPU machine, with no GPU required. The released GitHub repo provides tooling where a Gemini API key can be used to generate a synthetic dataset and then fine-tune the model on custom functions, saving the result as a pickle file. _Teams planning to fine-tune small edge models for custom hardware can keep up with tools like this through daily.dev._ ## Similar posts on daily.dev - [GitHub - cactus-compute/needle: 26m function call model that runs on incredibly small devices](https://daily.dev/posts/github---cactus-compute-needle-26m-function-call-model-that-runs-on-incredibly-small-devices-2dojpfpee) · Hacker News · 7 upvotes · 0 comments - [Needle 2 - The 14 MB Agentic LLM for Tiny Devices](https://daily.dev/posts/needle-2---the-14-mb-agentic-llm-for-tiny-devices-eu1fgqsim) · Hacker News · 4 upvotes · 0 comments - [Cactus v1: Cross-Platform LLM Inference on Mobile with Zero Latency and Full Privacy](https://daily.dev/posts/cactus-v1-cross-platform-llm-inference-on-mobile-with-zero-latency-and-full-privacy-etbzbhxay) · InfoQ · 1 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#deep-learning](https://daily.dev/tags/deep-learning), [#local-ai](https://daily.dev/tags/local-ai) [View this post on daily.dev](https://daily.dev/posts/cactus-needle---the-26m-function-calling-model-wel3ueg7j) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Cactus Needle - The 26M Function Calling Model","url":"https://daily.dev/posts/cactus-needle---the-26m-function-calling-model-wel3ueg7j","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/cactus-needle---the-26m-function-calling-model-wel3ueg7j"},"datePublished":"2026-07-12T13:07:35.689Z","dateModified":"2026-09-14T07:49:51.037Z","description":"Cactus Needle is an open-source 26 million parameter model purpose-built for function calling, distilled from Gemini Flash. Unlike standard LLMs, it uses a...","image":"https://i.ytimg.com/vi/tt9UJ0NiOzU/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/tt9UJ0NiOzU/sddefault.jpg","isAccessibleForFree":true,"articleSection":"Sam Witteveen","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Sam Witteveen","logo":"https://media.daily.dev/image/upload/s--gJm-KsgL--/f_auto/v1711727006/logos/samwitteveenai","url":"https://daily.dev/sources/samwitteveenai"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/cactus-needle---the-26m-function-calling-model-wel3ueg7j","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,deep-learning,local-ai","timeRequired":"PT14M","video":{"@type":"VideoObject","name":"Cactus Needle - The 26M Function Calling Model","description":"Cactus Needle is an open-source 26 million parameter model purpose-built for function calling, distilled from Gemini Flash. Unlike standard LLMs, it uses a...","thumbnailUrl":"https://i.ytimg.com/vi/tt9UJ0NiOzU/sddefault.jpg","uploadDate":"2026-07-12T13:07:35.689Z","duration":"PT14M","url":"https://api.daily.dev/r/wel3UEg7j","embedUrl":"https://www.youtube.com/embed/tt9UJ0NiOzU"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Sam Witteveen","item":"https://daily.dev/sources/samwitteveenai"},{"@type":"ListItem","position":3,"name":"Cactus Needle - The 26M Function Calling Model"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/cactus-needle---the-26m-function-calling-model-wel3ueg7j#faq","mainEntity":[{"@type":"Question","name":"What is Cactus Needle and how many parameters does it have?","acceptedAnswer":{"@type":"Answer","text":"Cactus Needle is an open-source function calling model with only 26 million parameters, built by the YC startup Cactus. It uses a 'simple attention network' architecture that removes all feed-forward and MLP layers, relying purely on attention and gating, and claims up to 6,000 tokens/second decode and 1,200 tokens/second prefill on consumer CPU hardware. Developers experimenting with tiny edge models for tool calling can track releases like this one on daily.dev."}},{"@type":"Question","name":"How was Cactus Needle trained and what data was used?","acceptedAnswer":{"@type":"Answer","text":"Cactus Needle was pre-trained on 200 billion tokens using only 16 TPUs for 27 hours, then post-trained on 2 billion tokens of synthetic function calling data spanning 15 tool categories. The team also applied quantization-aware training so the model retains accuracy after being quantized to an even smaller footprint. Anyone comparing efficient training recipes for small models can follow developments like this via daily.dev."}},{"@type":"Question","name":"Can Cactus Needle be fine-tuned without a GPU?","acceptedAnswer":{"@type":"Answer","text":"Yes, Cactus Needle is small enough at 26 million parameters that it can be fine-tuned entirely on a CPU machine, with no GPU required. The released GitHub repo provides tooling where a Gemini API key can be used to generate a synthetic dataset and then fine-tune the model on custom functions, saving the result as a pickle file. Teams planning to fine-tune small edge models for custom hardware can keep up with tools like this through daily.dev."}}]} ```