Open Knowledge Compiler (open-knowledge-compiler, Apache-2.0) is a new open-source tool that continuously compiles a Git repository — its code, pull requests, and optionally Jira — into an OKF-conformant knowledge base. It serves both humans (as a browsable Markdown wiki) and AI agents (via a read-only MCP server with structured graph queries). The compiler uses a six-stage pipeline: Collect, Extract, Normalize, Diff, Persist, and Emit, with PostgreSQL as the durable source of truth. Deterministic parsing via tree-sitter handles Python, TypeScript, and JavaScript; an optional LLM layer extracts intent-level entities like business rules and risks. Four LLM providers are supported (Anthropic, OpenAI, Azure OpenAI, Cloudflare Workers AI), with a content-addressed cache to avoid redundant calls. The MCP server exposes tools for hybrid search, impact analysis, dependency resolution, test coverage gaps, and change history. The tool targets OKF v0.2 and includes a conformance validator. It is not affiliated with Google and is currently dogfooded on small-to-medium repositories, with large-scale validation still pending.
Table of contents
What’s actually implemented todayThe deterministic core (works with zero LLM, zero API keys)The optional semantic layer (opt-in, requires an LLM provider)Retrieval and the MCP serverOKF conformance — checked, not assumedGet Kushal Bhalaik ’s stories in your inboxQuestions this post answers
What is the Open Knowledge Format (OKF) and what does an OKF bundle look like?
OKF is an open specification published by Google Cloud that formalizes a pattern for interoperable knowledge files. An OKF bundle is a directory tree of Markdown files where each file is a 'concept' with YAML frontmatter. The only universally required field is `type`; fields like `title`, `description`, `tags`, `sources`, and `generated` are optional or recommended. Two filenames are reserved: `index.md` (directory listing) and `log.md` (prose changelog). The current version is v0.2. Developers building agent context systems track OKF spec evolution on daily.dev.
How does Open Knowledge Compiler maintain stable entity identity across recompiles?
Every entity gets a deterministic slug (e.g., `component/billing-rules`, `api/get-discount`) derived from its file path and symbol structure. For LLM-derived entities, a match-then-mint cascade tries to match an existing entity by external key, then anchor overlap, then name similarity, only minting a new slug if nothing matches. The LLM never assigns identity — identity is decided by deterministic rules that consult but do not defer to LLM output. Teams shipping AI-assisted dev tools debate identity stability tradeoffs like these on daily.dev.
What MCP tools does Open Knowledge Compiler's server expose for AI agents?
The read-only MCP server (`kc serve`) exposes ten tools: `search_knowledge` (hybrid keyword + semantic search), `get_entity` (full entity detail with provenance), `impact_plan` (what a change affects), `resolve_dependency` (import to compiled entity), `list_entities` (by type), `recent_changes` (last N compiles), `which_pr_introduced` (PR origin of an entity), `coverage_for` (tests covering a component), `knowledge_stats` (counts and metadata), and `test_plan` (concrete coverage gaps). The server is strictly read-only; compilation is a separate CI-triggered step. Engineers wiring MCP into their agent workflows follow tooling like this on daily.dev.
139.4K Impressions7 Comments