Technology is the easy part. Confidentiality boundaries and retrieval granularity are what actually determine whether this should be built at all.
Written in response to a recurring question in legal technology communities: AI document search for a small firm — what actually works?
Different clients' files and due diligence material have to be strictly isolated. That is a professional obligation, not a nice-to-have. Technically: tag every document with client ownership and privilege level at ingest, then filter the candidate set by user identity before retrieval.
The ordering matters. A lot of systems retrieve first, then hide results based on role. Under RAG that is unsafe — once the text is in the model's context, hiding it from the UI does not undo anything.
If a firm's confidentiality requirements rule out any structural processing, the honest answer is don't do it, rather than take the project and figure it out later. That boundary belongs in the first conversation.
Someone asking "how do we usually draft the liability clause" wants a clause. If you chunk by fixed length, you will split the condition from the consequence, retrieve the second half, and get a confidently misleading answer.
Split on clause and section boundaries, and carry the section context into each chunk. The retrieved fragment is then complete on its own — no missing precondition to mislead with.
Ask for a specific section number, case citation, or term of art and pure vector search returns something semantically adjacent but wrong. In legal work "close enough" is not acceptable — and it is dangerous precisely because it looks right.
Keyword plus semantic, then rerank. That is the baseline here, not an optimization. Any vector-only setup will fail on citations and section numbers.
No OCR and layout reconstruction means multi-column pages turn into scrambled text and retrieval is pointless. If the material is entirely paper and undigitized, step one is digitization, not AI. Getting this backwards wastes the entire budget.
If a standard only exists in a partner's head and was never written down, the system will not report a gap — it will construct a plausible answer from whatever it found. This is the single thing most worth guarding against.
Get a knowledge-gap list at handoff.
A system that admits its gaps is far more trustworthy than one that answers everything and invents the parts it cannot reach.
Confidentiality requirements vary enormously between firms. We tell you first whether your material permits structural processing at all — if it does not, we say so.