TL;DR: You've read thousands of articles. You can use almost none of them right now. The bottleneck in knowledge work isn't thinking. It's forgetting. Andrej Karpathy just showed a system where an LLM organizes your research into a living wiki, and the questions you ask feed back into it. No elaborate RAG pipelines. Just markdown, folders, and a loop that compounds.
Think about how many articles you've read this year. Papers you've skimmed. Threads you've bookmarked. Podcasts you half-listened to while cooking.
Now ask yourself: how many of those insights are available to you right now, in this moment, for the thing you're working on today?
The number is embarrassingly close to zero. Not because you're lazy. Not because you're not smart. Because your brain is a leaky bucket, and it always has been. You pour knowledge in, and most of it drains out before you need it. Every new project, every new question, you start from scratch. Even though the insight you need is somewhere in your past. You just can't reach it.
That's the real problem with knowledge work. Not thinking. Forgetting.
What did Karpathy actually build?
Andrej Karpathy, former Senior Director of AI at Tesla and founding member of OpenAI, shared a system this week that sounds almost too simple to be interesting. 1
Raw documents go into a folder. Articles, papers, repos, datasets, anything. An LLM reads them, then compiles everything into a structured markdown wiki. Summaries, backlinks, conceptual categories. Obsidian serves as the frontend. You browse the wiki like a personal Wikipedia.
That part alone isn't new. People have been building "second brains" in Notion and Obsidian for years. The difference is what happens next.
When you ask the system a question, the LLM doesn't just answer it. It researches its own wiki and synthesizes a response. Karpathy then often files that response back into the knowledge base. The wiki grows. The next question is easier to answer because the system now knows more than it did an hour ago.
Karpathy says he's running this at around 100 articles and 400,000 words. No elaborate RAG pipeline. Just organized markdown and an LLM that maintains its own indexes. "I rarely touch it directly," he wrote. 1
Think of it like a research assistant who doesn't just answer your questions. They reorganize your entire filing cabinet after every conversation, so the next question takes half the time.
Why does the loop matter more than the tool?
The tool is markdown files and Obsidian. You could rebuild this in a weekend. The loop is what makes it work.
Most "second brain" systems die. You start a Notion workspace, organize it beautifully for two weeks, then life happens and it decays. The organization was the hard part, and it depended entirely on you showing up to maintain it. You were the bottleneck.
Karpathy's system flips that. The LLM maintains the organization. The LLM runs "health checks" to find inconsistencies and suggest new articles. The system maintains itself. Every time you use it, it gets better. Not because you put in extra effort, but because using it is the maintenance.
That's compound interest applied to knowledge. Each question doesn't just give you an answer. It makes every future question cheaper. The blank page dies, not because AI writes for you, but because AI remembers for you.
I wrote about Karpathy's AutoResearch two days ago. A loop that runs ML experiments while you sleep. Same pattern showing up again: the loop is the invention, not the tool. A simple cycle that compounds is worth more than a sophisticated tool that doesn't.
Do we even need bigger context windows?
Here's the contrarian part. The AI industry is racing toward bigger context windows. 1 million tokens. 10 million. Bigger windows and structured memory aren't mutually exclusive, but the default assumption is clear: if we can fit everything into one prompt, the model will figure it out.
Karpathy's system uses markdown files and folders.
Developer JUMPERZ put it well: "Agents that own their own knowledge layer do not need infinite context windows. They need good file organisation and the ability to read their own indexes. Way cheaper, way more scalable, and way more inspectable than stuffing everything into one giant prompt." 2
There's something familiar here. I keep noticing that constraints beat complexity. In product design, in engineering, and now in AI architecture. The pneumatic tyre hasn't changed in a century. The iPhone has been the same rectangle since 2017. And maybe the answer to AI's memory problem isn't a bigger brain. It's a better filing cabinet.
A 10-million-token context window is brute force. An organized knowledge base with good indexes is architecture. One scales with money. The other scales with use.
Where does this go?
Karpathy sees the endpoint. "Every question to a frontier-grade LLM spawns a team of LLMs to automate the whole thing," he wrote. "Iteratively construct an entire ephemeral wiki, lint it, loop a few times, then write a full report. Way beyond a .decode()." 1
Today, it's one person and one loop building a knowledge base over weeks. Tomorrow, a swarm of agents builds an entire wiki per question. Assembling, cross-referencing, linting for errors, then handing you the distilled result. Not a chat response. A researched report backed by a temporary knowledge base that was purpose-built for your specific question, then discarded.
The compound interest endpoint isn't just "you never start from zero." It's "you never even have to ask twice."
Key takeaways
- The bottleneck in knowledge work isn't thinking. It's forgetting. You've already had most of the insights you need. You just can't connect them to what you're working on now.
- Karpathy's system is a loop, not a tool. Raw documents → LLM-compiled wiki → Q&A that feeds back into the wiki → compound growth. No elaborate RAG. Just markdown and folders.
- Self-maintaining beats self-organizing. Traditional second brains decay because you're the maintenance bottleneck. This system maintains itself. Using it is the upkeep.
- Bigger context windows might be the wrong bet. Good file organization and LLM-maintained indexes can be cheaper, more scalable, and more inspectable than stuffing everything into one massive prompt.
- The blank page is a symptom. The disease is forgetting. The cure is a system where every question makes the next one easier.
I break down things like this on LinkedIn, X, and Instagram. Usually shorter, sometimes as carousels. If this resonated, you'd probably like those too
