When I built an AI research assistant for a law firm, the feature I thought would be a nice-to-have turned out to be the one they use most.
The system has an annotation feature. Any user can select text in a document and leave a comment. Something like "this interpretation was overruled by ruling X in 2024" or "this applies only to NRW, not nationally" or "our firm's position differs, see internal memo Y."
Technically here's what happens. Comments are stored in PostgreSQL linked to the document ID, page number, and selected text. When a query comes in, the system does two things. First it fetches comments attached to the specific documents that were retrieved by vector search. Second it fetches ALL comments across ALL documents regardless of what was retrieved. Both get injected into the LLM's context.
The second part is important. If a senior lawyer annotated document A saying "this is outdated" but the query only retrieved documents B and C, the system still sees that annotation through the global comments injection. The cache refreshes every 60 seconds so new comments are picked up almost immediately.
The prompt tells the model to treat these annotations as authoritative expert notes and to prioritize them when they contradict the document text.
Why this matters more than I initially thought:
Legal knowledge goes stale. A court ruling from 2022 might be superseded by a 2024 decision. Without the annotation system you'd need to re-ingest documents, update metadata, maybe re-chunk everything. With annotations a senior lawyer just writes "superseded by X" and the system incorporates that knowledge on the next query. No engineering work needed.
It also captures institutional knowledge that doesn't exist in any document. Things like "our firm interprets this more conservatively than the standard reading" or "client X has specific requirements around this clause." That kind of knowledge lives in senior lawyers' heads and normally gets lost when they retire or leave.
The legal team started using it within the first week without any training. They were already used to annotating PDFs with comments. This just made those comments searchable and part of the AI's knowledge base.
If you're building RAG for any domain where expert interpretation matters (legal, medical, financial, academic), consider building an annotation layer. Better embeddings and fancier retrieval will improve your baseline. But letting domain experts directly correct and enrich the AI's knowledge is a multiplier that no amount of model improvement can replicate.
[link] [comments]
