A practical implementation of Google's always-on-memory agent pattern adapted for Obsidian notes, replacing vector databases with SQLite and direct LLM reasoning. Using Claude Haiku 4.5's 250K context window, the system stores structured memory records (~300 tokens each) allowing ~650 memories to fit directly in context. Three agents handle the workflow: IngestAgent extracts structured metadata from raw text, ConsolidateAgent finds cross-cutting patterns across memories, and QueryAgent synthesizes answers from both raw memories and consolidation insights. The system includes file watching for automatic Obsidian vault ingestion, supports text/images/PDFs, and integrates with Claude Code and Kiro-CLI. The core argument is that vector databases are retrieval optimizations for large-scale problems — at personal note scale, large context windows make them unnecessary overhead.

13m read timeFrom towardsdatascience.com
Post cover image
Table of contents
The SetupAn Insight That Changes the MathThe ArchitectureWhat Actually Gets StoredWhy No Vector DBSeeing It In ActionWhat’s NextTry ItAbout
129 Impressions