Knowledge Bases for AI
An AI assistant is only as good as what it can read. How to structure what your business knows so a machine can use it.
Most AI assistants underperform for a boring reason: there's nothing good for them to read.
The business knowledge exists — in people's heads, in email threads, in a shared drive with four versions of the same document. An assistant pointed at that produces confident answers assembled from whichever version it found.
Fixing the knowledge is most of the work, and it has the useful property of being valuable even if you never deploy an assistant at all.
Read the documentation standards · Request Your Free Systems Audit
What an AI needs that a person doesn't
A new employee reading your documentation can tell when something is out of date, ask a colleague, and notice that two pages contradict each other.
An AI system does none of that reliably. It will read both contradictory pages and produce an answer that blends them.
So the requirements are stricter:
One answer per question. Contradictions get resolved into confident nonsense.
Explicit over implied. "Standard terms apply" means nothing without the terms nearby or linked.
Dated and current. Without dates, old content competes equally with new.
Scoped. A document that mixes a procedure, a policy and some meeting notes retrieves badly, because only part of it is relevant to any given question.
Structure that retrieves well
One topic per document
A page covering refunds, returns and exchanges will be retrieved for all three and will include irrelevant material each time. Three pages retrieve better than one.
Headings that state the content
"Refund eligibility: time limits" retrieves better than "Section 3." The heading is a strong signal about what the section contains.
Self-contained sections
Retrieval usually returns a chunk, not a whole document. A section that begins "As mentioned above..." arrives without the above.
Each section should make sense read alone. This means some repetition of context, which is the opposite of good writing for humans — and it's the right trade-off here.
Front-load the answer
State the conclusion first, then the detail. Both retrieval and readers benefit.
Prose over tables for the important parts
Tables are efficient for people and often lose their structure when chunked. If a rule matters, state it in a sentence as well as showing it in a table.
What to include
Procedures. How things get done.
Policies and rules. What's allowed, thresholds, approvals.
Reference data. Prices, specifications, terms, service descriptions.
Decisions and their reasons. Why things are the way they are. This is the knowledge that leaves with people.
Frequent questions, with answers. From real customer and internal questions, not invented ones.
What to leave out
Anything with credentials or secrets. Ever. Not even in a document the assistant "shouldn't" surface.
Personal data beyond what's needed. Scope matters — an assistant with access to everything will eventually surface something it shouldn't.
Drafts and half-finished thinking. An AI can't tell a draft from a decision.
Meeting notes, generally. They're a record of discussion, not conclusions, and they're full of statements that were considered and rejected.
Content whose real home is elsewhere. If the source of truth is a CRM or a live system, the assistant should read that, not a stale copy.
Resolving contradictions first
Before connecting anything, find the contradictions. They're the single biggest cause of bad output.
Common sources: two versions of a price list, a policy updated in one place and not another, a procedure that changed without the document changing.
A practical method: list the twenty questions your business gets asked most. For each, find every document that answers it. Where you find more than one answer, resolve it.
This takes a day or two and improves answer quality more than any amount of configuration.
Maintenance is the whole game
A knowledge base is accurate on the day it's assembled.
Every document has an owner and a review date. Shown in the document, not in metadata the assistant can't see.
Changes propagate from one place. One home per fact. Everything else links.
Retired content is removed, not left in place. An archived document that's still readable will still be read.
New knowledge has a route in. When someone works out the answer to a question nobody had documented, there needs to be an obvious place to put it. Without that route, the knowledge base ages from the day it's built.
Measuring whether it works
Log the questions. Specifically:
- Questions the assistant answered confidently
- Questions it couldn't answer
- Questions where the answer was wrong
The second list is your documentation backlog, in priority order, generated by real demand. It's the most useful output of the whole exercise.
The third list needs investigating individually — a wrong answer is usually a contradiction or a stale document, and both are findable.
Scope deliberately
An assistant with access to everything is harder to trust and harder to debug than one with access to a defined set.
Start narrow: one area, one audience, one clearly bounded set of documents. Expand when it's demonstrably working.
The common pattern is the opposite — connect everything, get mixed results, and conclude that the technology isn't ready. The technology is usually fine. The scope was unbounded.
A realistic sequence
- Pick one area. Customer-facing FAQs, or internal procedures for one team.
- List the twenty most common questions.
- Find the answers. Note which exist in writing and which only exist in someone's head.
- Resolve the contradictions.
- Write what's missing. The list from step 3.
- Structure it: one topic per document, self-contained sections, dates and owners.
- Connect the assistant to that set only.
- Log everything and review weekly for a month.
- Expand when the logs show it's working.
Steps 1 to 5 take most of the time and deliver most of the value — including to the people who were answering those twenty questions manually.
An assistant can only be as reliable as what it reads. Fix the knowledge first; it's worth doing regardless.
Want your knowledge organised properly?
A free systems audit covers what your business knows, where it lives, and what it would take to make it usable.