--- title: AnduinOS Introduces Why AI: An On-Device,... description: AnduinOS has released Why AI (anduinos-why-ai), an open-source, fully offline AI command-line utility for Linux. Powered by llama.cpp and bundled with a Google... canonical: https://daily.dev/posts/anduinos-introduces-why-ai-an-on-device-privacy-focused-unix-ai-tool-sy0kal6b9 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: AnduinOS Introduces Why AI: An On-Device, Privacy-Focused UNIX AI Tool | daily.dev og:description: AnduinOS has released Why AI (anduinos-why-ai), an open-source, fully offline AI command-line utility for Linux. Powered by llama.cpp and bundled with a Google... og:url: https://daily.dev/posts/anduinos-introduces-why-ai-an-on-device-privacy-focused-unix-ai-tool-sy0kal6b9 og:image: https://api.daily.dev/og/posts/sy0KAL6b9.png og:image:alt: AnduinOS Introduces Why AI: An On-Device, Privacy-Focused UNIX AI Tool og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # AnduinOS Introduces Why AI: An On-Device, Privacy-Focused UNIX AI Tool **[AnduinOS](https://daily.dev/sources/anduinos)** · 3 min read · 3 upvotes · 0 comments ## Summary AnduinOS has released Why AI (anduinos-why-ai), an open-source, fully offline AI command-line utility for Linux. Powered by llama.cpp and bundled with a Google Gemma 4 E2B model, it runs entirely on-device with no background daemons, no telemetry, and no cloud connectivity. It integrates natively with UNIX pipes (stdin/stdout), enabling use cases like log analysis with journalctl, automated git commit message generation, and shell command generation via natural language. It also includes an OpenAI-compatible local HTTP server that can serve as a drop-in backend for tools like GitHub Copilot CLI. Installation is available via apt on AnduinOS 2.0+. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is Why AI in AnduinOS and how does it work without cloud connectivity? Why AI (anduinos-why-ai) is an open-source, on-device command-line AI utility for AnduinOS 2.0 and later that runs entirely offline with no telemetry or background daemons. It uses the llama.cpp framework with a bundled Google Gemma 4 E2B (Q4_K_M) model, supports Vulkan and CUDA acceleration, and reads from stdin/writes to stdout so it composes with standard UNIX pipes like grep, git, and journalctl. _Developers building local-first AI tooling can track releases like this one on daily.dev._ ### How do I connect GitHub Copilot CLI to a local AI model instead of a cloud API? Launch the Why AI local server with the -s flag, which starts an OpenAI-compatible HTTP server on port 8080 serving the bundled Gemma model. Then set COPILOT_PROVIDER_BASE_URL to http://127.0.0.1:8080/v1 and COPILOT_MODEL to gemma-4-e2b-it-q4_k_m.gguf before running copilot, allowing the CLI to use local inference instead of a cloud backend. _Anyone wiring local models into existing CLI assistants can follow similar setups shared on daily.dev._ ### How can I generate a Conventional Commits style commit message from staged git changes using a local AI tool? Pipe the output of git diff --cached into the why command with a prompt requesting a short commit message following the Conventional Commits specification and instructing it to output only the message. This runs entirely through local inference via llama.cpp without sending diff contents to an external service. _Developers automating commit workflows locally can find similar command-line AI patterns on daily.dev._ ## Similar posts on daily.dev - [Announcing AnduinOS v2.0.1: ARM64 Expansion, Edge AI, and a Globalized Ecosystem](https://daily.dev/posts/announcing-anduinos-v2-0-1-arm64-expansion-edge-ai-and-a-globalized-ecosystem-s44odjjob) · AnduinOS · 0 upvotes · 0 comments - [I Tried This Open Source ChatGPT Alternative on Linux, But Went Back to Ollama](https://daily.dev/posts/i-tried-this-open-source-chatgpt-alternative-on-linux-but-went-back-to-ollama-gbag5xspb) · It's Foss · 0 upvotes · 0 comments - [Free AI on Your Mac](https://daily.dev/posts/free-ai-on-your-mac-yklinvgqt) · Hacker News · 0 upvotes · 0 comments - [AI power tools: 6 ways to supercharge your terminal](https://daily.dev/posts/ai-power-tools-6-ways-to-supercharge-your-terminal-txighsnc8) · InfoWorld · 1 upvotes · 0 comments - [I turned my phone into an AI agent powered by my local LLM, and it installed software on its own](https://daily.dev/posts/i-turned-my-phone-into-an-ai-agent-powered-by-my-local-llm-and-it-installed-software-on-its-own-dt5c9kqpw) · XDA Developers · 2 upvotes · 0 comments --- Tags: [#privacy](https://daily.dev/tags/privacy), [#local-ai](https://daily.dev/tags/local-ai), [#llama-cpp](https://daily.dev/tags/llama-cpp) [View this post on daily.dev](https://daily.dev/posts/anduinos-introduces-why-ai-an-on-device-privacy-focused-unix-ai-tool-sy0kal6b9) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"AnduinOS Introduces Why AI: An On-Device, Privacy-Focused UNIX AI Tool","url":"https://daily.dev/posts/anduinos-introduces-why-ai-an-on-device-privacy-focused-unix-ai-tool-sy0kal6b9","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/anduinos-introduces-why-ai-an-on-device-privacy-focused-unix-ai-tool-sy0kal6b9"},"datePublished":"2026-07-25T10:53:41.403Z","dateModified":"2026-09-13T21:55:43.156Z","description":"AnduinOS has released Why AI (anduinos-why-ai), an open-source, fully offline AI command-line utility for Linux. Powered by llama.cpp and bundled with a Google...","image":"https://media.daily.dev/image/upload/s--ZrL_HSsR--/f_auto/v1722860399/public/Placeholder%2006","thumbnailUrl":"https://media.daily.dev/image/upload/s--ZrL_HSsR--/f_auto/v1722860399/public/Placeholder%2006","isAccessibleForFree":true,"articleSection":"AnduinOS","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"AnduinOS","logo":"https://media.daily.dev/image/upload/s--j_GDWumQ--/f_auto/v1756383694/logos/anduinos","url":"https://daily.dev/sources/anduinos"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/anduinos-introduces-why-ai-an-on-device-privacy-focused-unix-ai-tool-sy0kal6b9","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"privacy,local-ai,llama-cpp","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"AnduinOS","item":"https://daily.dev/sources/anduinos"},{"@type":"ListItem","position":3,"name":"AnduinOS Introduces Why AI: An On-Device, Privacy-Focused UNIX AI Tool"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/anduinos-introduces-why-ai-an-on-device-privacy-focused-unix-ai-tool-sy0kal6b9#faq","mainEntity":[{"@type":"Question","name":"What is Why AI in AnduinOS and how does it work without cloud connectivity?","acceptedAnswer":{"@type":"Answer","text":"Why AI (anduinos-why-ai) is an open-source, on-device command-line AI utility for AnduinOS 2.0 and later that runs entirely offline with no telemetry or background daemons. It uses the llama.cpp framework with a bundled Google Gemma 4 E2B (Q4_K_M) model, supports Vulkan and CUDA acceleration, and reads from stdin/writes to stdout so it composes with standard UNIX pipes like grep, git, and journalctl. Developers building local-first AI tooling can track releases like this one on daily.dev."}},{"@type":"Question","name":"How do I connect GitHub Copilot CLI to a local AI model instead of a cloud API?","acceptedAnswer":{"@type":"Answer","text":"Launch the Why AI local server with the -s flag, which starts an OpenAI-compatible HTTP server on port 8080 serving the bundled Gemma model. Then set COPILOT_PROVIDER_BASE_URL to http://127.0.0.1:8080/v1 and COPILOT_MODEL to gemma-4-e2b-it-q4_k_m.gguf before running copilot, allowing the CLI to use local inference instead of a cloud backend. Anyone wiring local models into existing CLI assistants can follow similar setups shared on daily.dev."}},{"@type":"Question","name":"How can I generate a Conventional Commits style commit message from staged git changes using a local AI tool?","acceptedAnswer":{"@type":"Answer","text":"Pipe the output of git diff --cached into the why command with a prompt requesting a short commit message following the Conventional Commits specification and instructing it to output only the message. This runs entirely through local inference via llama.cpp without sending diff contents to an external service. Developers automating commit workflows locally can find similar command-line AI patterns on daily.dev."}}]} ```