--- title: Implementing the Agent Processor Unit: A Tutor That Runs... description: A detailed architecture writeup describes Akili, an offline-capable tutoring agent designed for refurbished, low-resource classroom hardware. The system... canonical: https://daily.dev/posts/implementing-the-agent-processor-unit-a-tutor-that-runs-offline-2bkchm4ae twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Implementing the Agent Processor Unit: A Tutor That Runs Offline | daily.dev og:description: A detailed architecture writeup describes Akili, an offline-capable tutoring agent designed for refurbished, low-resource classroom hardware. The system... og:url: https://daily.dev/posts/implementing-the-agent-processor-unit-a-tutor-that-runs-offline-2bkchm4ae og:image: https://api.daily.dev/og/posts/2bkchM4Ae.png og:image:alt: Implementing the Agent Processor Unit: A Tutor That Runs Offline og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Implementing the Agent Processor Unit: A Tutor That Runs Offline **[Medium](https://daily.dev/sources/medium_js)** · 11 min read · 0 upvotes · 0 comments ## Summary A detailed architecture writeup describes Akili, an offline-capable tutoring agent designed for refurbished, low-resource classroom hardware. The system borrows processor design concepts (memory hierarchy, MMU-like arbitration, paging) to manage a student's context across three local memory tiers, with only LLM inference optionally touching the network. It covers a doubly-linked-list based routing algorithm (BMJ) for surfacing relevant memory blocks, a bug where memory promotions were overwritten each turn due to duplicate in-memory copies, a vector-space dimension mismatch bug between the embedding pipeline and client, a safe AST-based calculator tool avoiding eval(), and measured performance numbers (turn latency, memory footprint, model sizing) for deploying on constrained devices like Raspberry Pi class machines or an M2 Air with 8GB RAM. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can I build a RAG system that works fully offline on low-resource hardware like a Raspberry Pi? Keep every component except the inference model local: embeddings, retrieval, and memory tiers should never touch the network. One implementation uses paraphrase-multilingual-MiniLM-L12-v2 in ONNX via fastembed for embeddings (252 MB of weights, shipped with the deployment) and LanceDB for local vector search, with only the LLM call optionally reaching a remote endpoint via a configurable LLM_PROVIDER. _daily.dev surfaces practical patterns for engineers designing offline-first or resource-constrained AI agents._ ### Why would a vector search return confidently wrong results instead of throwing an error? A mismatch between the embedding model used to build a vector index and the one used to query it does not raise an exception; it silently returns noise ranked as if it were relevant. This happens when the indexing pipeline and the client declare different embedding model configs, producing vectors of different dimensions that still technically compare against each other. _Debugging silent retrieval degradation is easier when developers track real-world RAG failure patterns via daily.dev._ ### Why shouldn't I use Python's eval() to let an AI agent execute calculator expressions from user input? Using eval() on model-generated or student-facing text creates an arbitrary code execution risk, since a string like __import__('os').system(...) would run as code. A safer approach parses the expression into an abstract syntax tree and only permits Constant and BinOp nodes, rejecting anything resembling a function call, since there is no way to write a call that looks like arithmetic. _Following secure coding patterns for AI tool execution is easier when developers track these tradeoffs on daily.dev._ ## Similar posts on daily.dev - [The architecture behind an AI tutor, four months in](https://daily.dev/posts/the-architecture-behind-an-ai-tutor-four-months-in-uqjo9alxn) · Medium · 4 upvotes · 0 comments - [Teaching a child in <1000 ms: the architecture behind a real-time tutor](https://daily.dev/posts/teaching-a-child-in-1000-ms-the-architecture-behind-a-real-time-tutor-8t3zegwa2) · Hacker News · 1 upvotes · 0 comments - [AI Agent Memory Explained in 3 Levels of Difficulty](https://daily.dev/posts/ai-agent-memory-explained-in-3-levels-of-difficulty-dgu2354gm) · Machine Learning Mastery · 2 upvotes · 0 comments - [Architecting efficient context-aware multi-agent framework for production](https://daily.dev/posts/architecting-efficient-context-aware-multi-agent-framework-for-production-9plaa76qf) · Google Developers · 2 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python), [#ai-agents](https://daily.dev/tags/ai-agents), [#rag](https://daily.dev/tags/rag), [#vector-search](https://daily.dev/tags/vector-search), [#edge-computing](https://daily.dev/tags/edge-computing) [View this post on daily.dev](https://daily.dev/posts/implementing-the-agent-processor-unit-a-tutor-that-runs-offline-2bkchm4ae) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Implementing the Agent Processor Unit: A Tutor That Runs Offline","url":"https://daily.dev/posts/implementing-the-agent-processor-unit-a-tutor-that-runs-offline-2bkchm4ae","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/implementing-the-agent-processor-unit-a-tutor-that-runs-offline-2bkchm4ae"},"datePublished":"2026-08-19T23:53:21.812Z","dateModified":"2026-09-13T20:09:23.763Z","description":"A detailed architecture writeup describes Akili, an offline-capable tutoring agent designed for refurbished, low-resource classroom hardware. The system...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5e56327f42dff0c80e97a9273cccae10?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5e56327f42dff0c80e97a9273cccae10?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Medium","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Medium","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/medium","url":"https://daily.dev/sources/medium_js"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/implementing-the-agent-processor-unit-a-tutor-that-runs-offline-2bkchm4ae","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python,ai-agents,rag,vector-search,edge-computing","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Medium","item":"https://daily.dev/sources/medium_js"},{"@type":"ListItem","position":3,"name":"Implementing the Agent Processor Unit: A Tutor That Runs Offline"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/implementing-the-agent-processor-unit-a-tutor-that-runs-offline-2bkchm4ae#faq","mainEntity":[{"@type":"Question","name":"How can I build a RAG system that works fully offline on low-resource hardware like a Raspberry Pi?","acceptedAnswer":{"@type":"Answer","text":"Keep every component except the inference model local: embeddings, retrieval, and memory tiers should never touch the network. One implementation uses paraphrase-multilingual-MiniLM-L12-v2 in ONNX via fastembed for embeddings (252 MB of weights, shipped with the deployment) and LanceDB for local vector search, with only the LLM call optionally reaching a remote endpoint via a configurable LLM_PROVIDER. daily.dev surfaces practical patterns for engineers designing offline-first or resource-constrained AI agents."}},{"@type":"Question","name":"Why would a vector search return confidently wrong results instead of throwing an error?","acceptedAnswer":{"@type":"Answer","text":"A mismatch between the embedding model used to build a vector index and the one used to query it does not raise an exception; it silently returns noise ranked as if it were relevant. This happens when the indexing pipeline and the client declare different embedding model configs, producing vectors of different dimensions that still technically compare against each other. Debugging silent retrieval degradation is easier when developers track real-world RAG failure patterns via daily.dev."}},{"@type":"Question","name":"Why shouldn't I use Python's eval() to let an AI agent execute calculator expressions from user input?","acceptedAnswer":{"@type":"Answer","text":"Using eval() on model-generated or student-facing text creates an arbitrary code execution risk, since a string like __import__('os').system(...) would run as code. A safer approach parses the expression into an abstract syntax tree and only permits Constant and BinOp nodes, rejecting anything resembling a function call, since there is no way to write a call that looks like arithmetic. Following secure coding patterns for AI tool execution is easier when developers track these tradeoffs on daily.dev."}}]} ```