An agent that knows your business, not the open web
A retrieval system that answers from your documents, cites the source, and admits when it does not know. The admission part is what makes it usable.
The most common failure of a knowledge assistant is not a wrong answer. It is a confident answer from the public internet about a process that only exists inside your company. Retrieval over your own documents fixes that — but only if the evaluation is part of the build, not an afterthought.
Citing the source is not a nice-to-have
If the answer does not carry the document and paragraph it came from, nobody can check it, so nobody will trust it, and you are back to searching by hand. Citation is what turns a plausible answer into a usable one.
We tune the retrieval so answers cite their source and admit when they do not know. "I could not find that in your documentation" is a correct and useful answer. A confident guess is not, and it is the one that gets quoted in a meeting.
Chunking is where the quality is decided
Most of a retrieval system is decided before the model ever sees your data: what counts as a chunk, how documents are structured on the way in, what gets embedded and when. Teams that spend a week tuning the prompt and a day fixing the ingestion get the inverse of the result they wanted.
Clean source material beats clever retrieval almost every time. If your runbooks live in eleven PDFs with six competing versions, fixing that is worth more than any model change.
Test it against questions you already know the answer to
Before it goes live, take twenty questions your team already answers confidently and check the system against them. Include the awkward ones — the process that only two people know, the exception to the documented rule. That set is the evaluation, and it takes an afternoon to build.
Where the answers have to come from instead
Some of the best answers are not in documents at all. They are in the CRM record, the ticket history, the order. When the question is "what is going on with this account", the answer lives in your systems, not your wiki.
That is an integration problem rather than a retrieval problem — CRM and ERP integration from $999 connects the live record so the agent can answer with current data instead of a document written last quarter.