https://reddit.com/link/1rv38qs/video/z3f8s0g50dpg1/player
Quick update to a demo I posted earlier.
Previously the system handled ~12k documents.
Now it scales to ~32k documents locally.
Hardware:
- ASUS TUF Gaming F16
- RTX 5060 laptop GPU
- 32GB RAM
- ~$1299 retail price
Dataset in this demo:
- ~30k PDFs under ACL-style folder hierarchy
- 1k research PDFs (RAGBench)
- ~1k multilingual docs
Everything runs fully on-device.
Compared to the previous post: RAG retrieval tokens reduced from ~2000 → ~1200 tokens. Lower cost and more suitable for AI PCs / edge devices.
The system also preserves folder structure during indexing, so enterprise-style knowledge organization and access control can be maintained.
Small local models (tested with Qwen 3.5 4B) work reasonably well, although larger models still produce better formatted outputs in some cases.
At the end of the video it also shows incremental indexing of additional documents.
[link] [comments]
