Most people still use AI like a search box.
They upload files, ask a question, get an answer, and move on. The next time they ask, the AI starts over. It retrieves chunks, rebuilds context, and tries to synthesize the answer again.
That works for simple questions. It breaks down when the work needs memory.
Andrej Karpathy’s “LLM Wiki” idea points to a better pattern: instead of treating documents as a pile of searchable files, let the AI build and maintain a structured knowledge base over time.
The difference is important: search retrieves information, but a wiki compounds it.
RAG Retrieves. A Wiki Compounds.
Most document-based AI systems use retrieval-augmented generation, or RAG. You give the AI a set of documents. When you ask something, the system searches for relevant chunks and generates an answer.
That is useful, but it has a weakness: the AI is rediscovering the knowledge every time.
A persistent wiki changes the model.
When a new article, meeting note, transcript, research paper, customer call, or internal document is added, the AI does not just index it. It reads it, summarizes it, connects it to existing topics, updates related pages, flags contradictions, and improves the overall structure.
The knowledge base gets smarter each time.
That is the key idea: the AI is not just answering questions. It is maintaining the system of knowledge underneath the answers.
The Three-Layer Model
Karpathy describes a simple architecture:
- Raw sources: the original files, notes, articles, screenshots, transcripts, or documents
- The wiki: AI-written markdown pages with summaries, entities, topics, comparisons, and links
- The schema: instructions that tell the AI how the wiki should be structured and maintained
This is powerful because it keeps the source material separate from the AI’s interpretation.
The raw documents remain the source of truth. The wiki becomes the working knowledge layer. The schema becomes the operating manual.
That is a clean division.
Why This Matters for Businesses
Every business already has scattered knowledge.
It lives in Slack threads, emails, proposals, call notes, Google Docs, CRM records, meeting transcripts, client feedback, and half-finished strategy documents.
The problem is not that the business lacks information. The problem is that the information does not compound.
People forget. Files get buried. New employees repeat old questions. Projects lose context. Strategy resets every time someone opens a new chat window.
An LLM-maintained wiki solves a different problem than search. It creates continuity.
Imagine a business wiki where AI keeps current pages for:
- Clients
- Campaigns
- Products
- Services
- Competitors
- Sales objections
- SEO strategy
- Internal processes
- Meeting decisions
- Open questions
- Lessons learned
After every meeting, the AI updates the relevant pages. After every customer call, it updates the account history. After every new article or report, it revises the topic pages. After every project, it records what worked and what failed.
That is not just documentation. That is operational memory.
The Human Still Matters
The strongest version of this system does not remove the human.
It changes the human’s job.
The person curates sources, asks good questions, reviews the synthesis, and decides what matters. The AI handles the tedious maintenance: summaries, links, updates, contradictions, logs, and cross-references.
That division makes sense.
Humans are good at judgment. AI is good at structured repetition.
The failure point in most knowledge bases is maintenance. People do not keep wikis updated because the work is boring and never-ending. AI does not care. It can touch 15 related files in one pass, update the index, add a log entry, and preserve the trail.
That is where the leverage is.
Why Markdown and Git Make Sense
One of the best parts of the LLM Wiki pattern is that it does not require a complicated platform.
Markdown files are simple. They are portable, readable, editable, and version-controlled. Tools like Obsidian make the wiki easy for humans to browse, while Git provides history and collaboration.
That matters because business knowledge should not be trapped inside another SaaS silo.
A plain-text knowledge base can move with the company. It can be searched, backed up, reviewed, branched, audited, and connected to other tools.
Simple wins.
The Bigger Takeaway
AI memory should not just mean “remember what I said last time.”
Real AI memory should mean: build something durable from what we learn.
Karpathy’s LLM Wiki idea is compelling because it turns AI from a temporary assistant into a long-term knowledge worker. The AI does not just answer. It organizes. It updates. It reconciles. It maintains.
For businesses, this is where AI gets serious.
The companies that win with AI will not be the ones with the most prompts. They will be the ones that build systems where knowledge compounds.
Source: Andrej Karpathy – LLM Wiki gist