A folder of PDFs isn’t knowledge. So I built one that compiles.
I implemented Karpathy’s LLM Wiki pattern inside my agent vault: drop a source into a read-only folder, a script registers it with a SHA-256 hash, and an AI agent compiles it into interlinked, Wikipedia-style pages that get linted like code. It doesn’t retrieve — it compiles. Raw truth stays immutable; understanding accumulates on top.
A storage bucket rots.
My vault was becoming a great storage bucket — and raw retrieval finds passages but never builds a connected picture.
Worse, it happily mixes original source truth with model interpretation, and nothing stops a source being silently edited.
I wanted something better: feed real documents into the brain so agents could interrogate a structured, cited knowledge graph instead of brute-force re-reading everything — and so no source could ever be quietly altered or contradicted without a trace.
When I saw Andrej Karpathy’s LLM Wiki pattern, it matched exactly what I was missing: the general pipeline from raw source to queryable understanding.
Compile, don't retrieve.
Raw truth stays immutable in one layer; the AI compiles interlinked understanding into a second; a governance layer keeps the graph healthy. Ingest one source at a time — register before you compile, link everything both ways, lint early.
Drop a source into the immutable raw/ layer; wiki-register-source.sh hashes it (SHA-256), captures size and MIME, generates a kebab-case source note, and logs a dated entry. An AI agent compiles it into the wiki/ layer’s five page types, linking everything both ways; wiki-lint.sh then checks the graph and exits non-zero to gate the commit. The raw layer is never edited — if a source is wrong, you add a correcting note, you don’t rewrite history.
Scripts
Schema & Rules
Intake
Compile Engine
Substrate
Deferred (Phase 2+)
A working scaffold with a real first batch.
Verified from the live wiki/ directory and wiki/log.md.
The first batch registered ~25 sources: one public SEO topic (“How Search Engines Work”) plus a larger private personal-document batch — each file hashed with SHA-256, extracted where text was available, and logged with its date. The private batch is intentionally not shown; every public visual on this page uses only the neutral SEO example.
What exists and works — and what's honestly Phase 1.
A functioning Phase-1 LLM Wiki: schema, ingest workflow, lint rules, a working registration script, a working lint script — with real ingests completed and logged.
A working scaffold with a proven ingest.
- Schema, ingest workflow, and lint rules authored in 00-system/.
- A working wiki-register-source.sh and wiki-lint.sh.
- The SEO source compiled into source / entity / concept / synthesis / question pages — including a “How Search Engines Work for SEO” synthesis linking back to its registered source.
- A larger real-world batch registered, hashed, extracted, and compiled into document maps and explicit open version/OCR questions.
- Every source logged with its hash and date.
Manual, supervised, one source at a time.
- No daemon, watcher, vector index, or GBrain is wired in yet — those are the documented next phases.
- It’s a working scaffold with a small proven batch, not a large automated knowledge engine.
- Compiled material stays draft — it can’t be bulk-promoted into canonical business or client claims without review.
Framed exactly right: "implemented the pattern and ran the first ingests," not "large automated knowledge engine." Phase 1 is manual and supervised on purpose.
A pile of files vs a linked graph.
I brought the pattern in and sequenced the build.
I judged that Karpathy’s LLM Wiki pattern fit our Obsidian + git + agent-memory foundation, and made the sequencing call: simple file-native version first, raw immutable, GBrain deferred.
I directed the build of the Phase-1 scaffold — the schema, the ingest workflow, the lint rules, and the register/lint scripts — then approved it and greenlit the first ingests. I set the rule that compiled wiki material stays draft and can’t be bulk-promoted into canonical claims without my review. I’m the architect and owner; the agent compiled under my direction. The pattern is Karpathy’s and GBrain is Garry Tan’s — the implementation and integration are mine.