After seeing Karpathy's tweet about using LLMs to build personal wikis from research documents, I realized I'd already been using something similar like this internally for our R&D.
So I cleaned it up and open-sourced it.
What it does: You drop a folder of raw documents (PDFs, papers, notes, code, 60+ formats) and the LLM compiles them into a structured markdown wiki with backlinked articles, concept pages, and a master index. It then compresses everything into a .aura archive optimized for RAG retrieval (~97% smaller than raw source data).
How it works:
pip install aura-research research init my-project # copy docs into raw/ research ingest raw/ research compile research query "your question" Key design decisions:
- No embeddings, no vector databases. Uses SimHash + Bloom Filters instead. Zero RAM overhead.
- Built-in 3-tier Memory OS (facts / episodic / scratch pad) so the LLM doesn't forget important context across sessions
- The wiki is just
.mdfiles, browse in Obsidian, VS Code, or whatever you like - Works with any LLM provider (OpenAI, Anthropic, Gemini) or as an agent-native tool inside Claude Code/Gemini CLI where no API key is needed
- Everything runs locally. No data leaves your machine.
The "no embeddings" choice: I deliberately avoided the standard RAG pipeline (chunk → embed → vector search). Instead, the LLM compiles the knowledge into a well-structured wiki with an index. When you query, it reads the index, finds the 2-3 relevant articles, and only loads those. The LLM is smart enough to navigate a good file structure, you don't need a separate embedding model if the knowledge is properly organized.
GitHub: https://github.com/Rtalabs-ai/aura-research PyPI: pip install aura-research
Would love feedback from this community, especially on the "structured wiki vs vector embeddings" tradeoff. Looking forward to your thoughts!
Also thinking about packaging this into a product, any insights would be appreciated!
[link] [comments]
