A walkthrough on running local LLMs on a Surface Pro 11's Neural Processing Unit (NPU) using Microsoft's Foundry Local and the R ellmer package. Since Ollama and LM Studio don't natively support NPU inference, the author adapted Microsoft's Python getting-started guide into R code. The solution starts the Foundry service, downloads and loads a model (Qwen2.5-0.5B), discovers the dynamic endpoint, resolves the model ID via the REST API, and connects ellmer's chat_openai_compatible to the local OpenAI-compatible endpoint to send prompts.
27 Impressions