Don't Retrieve, Navigate: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG

arXiv cs.CL / 4/17/2026

💬 OpinionIdeas & Deep AnalysisTools & Practical UsageModels & Research

共有:

Key Points

The paper argues that conventional RAG limits LLM agents because they only passively consume retrieved results without understanding corpus structure or what has not been retrieved.
It introduces Corpus2Skill, which converts an enterprise document corpus into a hierarchical, navigable “skill directory” offline using iterative clustering and LLM-generated summaries at multiple levels.
At serving time, an LLM agent uses the explicit hierarchy to decide where to look, drill down topic branches via progressively detailed summaries, and retrieve full documents by ID.
The explicit tree enables better backtracking and the ability to combine evidence across different branches, improving reasoning over scattered information.
Experiments on WixQA show Corpus2Skill outperforms dense retrieval, RAPTOR, and agentic RAG baselines across all evaluated quality metrics.

Abstract

Retrieval-Augmented Generation (RAG) grounds LLM responses in external evidence but treats the model as a passive consumer of search results: it never sees how the corpus is organized or what it has not yet retrieved, limiting its ability to backtrack or combine scattered evidence. We present Corpus2Skill, which distills a document corpus into a hierarchical skill directory offline and lets an LLM agent navigate it at serve time. The compilation pipeline iteratively clusters documents, generates LLM-written summaries at each level, and materializes the result as a tree of navigable skill files. At serve time, the agent receives a bird's-eye view of the corpus, drills into topic branches via progressively finer summaries, and retrieves full documents by ID. Because the hierarchy is explicitly visible, the agent can reason about where to look, backtrack from unproductive paths, and combine evidence across branches. On WixQA, an enterprise customer-support benchmark for RAG, Corpus2Skill outperforms dense retrieval, RAPTOR, and agentic RAG baselines across all quality metrics.