How to Stop Your AI From Making Things Up

TL;DR: AI hallucinations in customer-facing agents aren't a model problem — they're a configuration problem. A well-scoped knowledge base plus explicit retrieval instructions reduces hallucination rates to near-zero for the questions your agent is designed to answer.

There's a specific nightmare every business owner has when they first deploy a customer-facing AI agent: a visitor asks a reasonable question, and the agent confidently answers with something completely wrong.

The fix isn't a better AI model — it's configuration. An agent restricted to its uploaded knowledge base with an explicit instruction to acknowledge gaps rather than guess produces accurate, verifiable answers. Hallucination happens when agents are allowed to draw from general training data instead of your specific documents.

Why AI Agents Hallucinate in the First Place

Language models are trained to generate fluent, helpful responses. When a question falls outside their knowledge, they don't say "I don't know" by default — they generate the most plausible-sounding answer they can construct from their training data.

For a general AI like ChatGPT, this is a known limitation that's hard to eliminate entirely, because the knowledge boundary is huge and fuzzy. For a custom agent with a bounded knowledge base, it's a solvable problem — because you define exactly what the agent knows.

The root cause of most hallucinations in custom agents isn't the model. It's one of these three things:

1. An incomplete knowledge base — the question is reasonable but the answer isn't in the documents. The agent tries to help anyway.

2. Missing retrieval instructions — no explicit instruction tells the agent what to do when it can't find the answer.

3. Ambiguous scope — the agent's instructions don't clearly define what it's qualified to answer, so it drifts into topics it shouldn't touch.

Fix all three, and hallucinations become rare enough to stop worrying about.

One concrete illustration of the knowledge gap problem: a restaurant that installs an AI agent without uploading their menu will have an agent that attempts to describe their offerings from general training data — producing plausible-sounding but potentially entirely wrong information about dishes, prices, and dietary accommodations. The customer who follows that advice and arrives expecting something different has a bad experience the business created. The fix is complete knowledge base coverage, not a more capable AI model.

Step 1: Build a Knowledge Base That Actually Covers the Questions

The most common hallucination trigger is a gap between what users ask and what's in the knowledge base. The agent wants to help, the answer isn't there, and it improvises.

Before you go public with an agent, run a simple exercise: list the 20 questions your customers or visitors are most likely to ask. Then check whether each one has a clear, accurate answer somewhere in your uploaded documents.

If 5 of those 20 questions don't have answers in the knowledge base, you'll see hallucinations on those 5 questions. The fix is the same every time: write the answer and add it to the knowledge base.

Alysium supports 11 file types — PDFs, Word docs, plain text, and more — so you can add answers in whatever format is easiest. A simple Q&A text document works perfectly. You don't need to restructure everything; just patch the gaps.

Step 2: Write an Explicit Knowledge Boundary Instruction

This is the single most impactful thing you can do to prevent hallucinations — and it takes about 30 seconds.

In your agent's behavioral instructions, add a line like this:

"If you cannot find a clear answer to a question in your knowledge base, say so honestly. Tell the visitor: 'I don't have specific information on that — please reach out to [contact] directly for an accurate answer.' Do not guess or generate an answer you're not confident about."

That instruction changes how the agent handles knowledge gaps. Instead of improvising, it acknowledges the limit and redirects. Visitors respect that — an honest "I don't know" builds more trust than a confident wrong answer.

The instruction format that works most reliably: a direct statement that the agent should answer only from the knowledge base, followed by specific language for what to say when it doesn't know something. Vague instructions like "stay on topic" leave the model significant interpretive latitude. Explicit instructions like "if the knowledge base doesn't contain a clear answer, say: I don't have that information in my current knowledge base — please reach out directly" give it a script to follow, which produces consistent behavior across thousands of conversations.

The instruction format that works most reliably: a direct statement that the agent should answer only from the knowledge base, followed by specific language for what to say when it doesn't know something. Vague instructions like "stay on topic" leave the model significant interpretive latitude. Explicit instructions like "if the knowledge base doesn't contain a clear answer, say: I don't have that information in my current knowledge base — please reach out directly" give it a script to follow, which produces consistent behavior across thousands of conversations.

Step 3: Add Retrieval Instructions That Constrain the Scope

Beyond the general knowledge gap instruction, you can write retrieval-specific rules for particular topic areas.

Useful patterns:

• Pricing: "For any question about pricing, only answer from the current pricing document. If pricing isn't in your documents, say the visitor should confirm directly as prices may have changed."
• Policies: "For cancellation, refund, or policy questions, answer only from the policy document. Don't interpret or extrapolate — quote the policy as written."
• Medical/Legal/Financial: "Do not provide any medical, legal, or financial advice. Acknowledge the question and direct the visitor to a qualified professional."

These targeted retrieval instructions create topic-level guardrails. They're especially valuable for high-stakes questions where a wrong answer has real consequences.

Retrieval instructions work differently from behavioral instructions — they govern what the agent does when searching the knowledge base rather than how it presents itself. A retrieval instruction like "only use information explicitly present in uploaded documents — do not infer, extrapolate, or reason beyond what is stated" creates a hard boundary that the agent respects even when the user's question is close to — but not exactly in — the knowledge base. This is the instruction that eliminates the plausible-but-fabricated answer problem.

Step 4: Test Specifically for Hallucination Scenarios

Most people test their agent with the questions it should answer well. That's necessary but not sufficient. Hallucination testing requires a different approach.

Run three types of test conversations before going public:

Gap tests: Ask questions you know aren't in the knowledge base. Does the agent say it doesn't know, or does it improvise? Any improvised answer on a gap question is a hallucination risk in the wild. Write down every gap you find — they all need knowledge base entries before you go public.

Edge case tests: Ask questions that are adjacent to what the agent knows but slightly outside scope. "What's your return policy on custom orders?" when your policy document only covers standard orders — what does it do? If it interpolates a policy that doesn't exist, that's a scope instruction issue.

Adversarial tests: Try phrasing questions in ways that might push the agent outside its lane. "What do you personally think about competitor X?" or "Just between us, what's the actual policy?" These reveal whether scope boundaries are holding under pressure.

A good test battery is 20–30 questions minimum — about half designed to succeed (questions it should answer well) and half designed to expose gaps or push boundaries. Anything that produces an improvised wrong answer needs a fix before real users see it.

Document every case where the agent generates an answer it shouldn't. Each one needs either a knowledge base update, an instruction update, or both. Budget 30–45 minutes for thorough pre-launch testing — it's the highest-ROI time you'll spend on the whole build.

Step 5: Review Real Conversations Regularly

Even a well-configured agent needs monitoring. User behavior in the real world is more creative and unpredictable than any test battery.

Alysium's analytics dashboard gives you full conversation history with search — you can filter by date range and search across all conversations for specific topics. Build a monthly habit: search for your highest-risk topics, read a sample of conversations, and look for any responses that don't match what you'd want to say.

When you find a problematic response pattern, trace it back to the root cause: gap in the knowledge base, missing instruction, or scope ambiguity. Fix the root cause, not just the symptom. A one-sentence addition to the knowledge base eliminates that hallucination pattern permanently — much faster than trying to predict it in advance.

One useful monitoring habit: after your first week of real traffic, review the helpfulness ratings. Low ratings on specific question types are usually a signal of knowledge gaps or wrong answers — start your investigation there rather than reading every conversation.

Most well-maintained agents see hallucination rates drop close to zero within the first month of real usage — because every gap you patch via monitoring is a gap that won't recur. The agent gets smarter, in a sense, with every round of real-world feedback.

Ready to build an agent that actually stays on-topic? Start free on Alysium — the configuration tools for controlling agent behavior are built into every account.

For more on writing instructions that control agent behavior, see What to Put in Your AI Agent's Instructions. For understanding how your knowledge base shapes answers, see What Happens When You Upload a Document to an AI Agent.