Building an AI concierge that doesn't hallucinate

An AI concierge is only useful if people can trust what it says. The failure mode that kills these projects isn't a wrong tone or a slow response, it's the bot confidently telling a customer something false about your prices, your hours, or your policy. Once that happens, nobody trusts it again.

The technique that fixes this is retrieval-augmented generation, or RAG. Instead of asking the model what it knows, you retrieve the relevant facts from your own data and hand them to the model as context, then ask it to answer using only that. Here's how I build them.

The shape of RAG

Three steps. Turn the user's question into a search, pull the matching documents from your knowledge base, and give those documents to the model with an instruction to answer from them. The model becomes a summarizer of your facts rather than a source of its own.

def answer(question: str) -> str:
    chunks = retrieve(question, k=5)
    context = "\n\n".join(c.text for c in chunks)
    return llm.complete(
        system=GROUNDED_PROMPT,
        user=f"Context:\n{context}\n\nQuestion: {question}",
    )

The whole trick lives in retrieve and GROUNDED_PROMPT. Get those right and hallucination drops to near zero.

Chunking and embedding your data

You can't feed the model your entire knowledge base, so you split it into chunks and store each one as a vector embedding. When a question comes in, you embed it too and find the chunks whose vectors are closest. That's semantic search, matching on meaning rather than keywords.

def ingest(documents: list[str]):
    for doc in documents:
        for chunk in split(doc, size=500, overlap=50):
            vector = embed(chunk)
            db.store(text=chunk, embedding=vector)

The overlap matters. If you cut cleanly at 500 characters, a fact that straddles the boundary gets split across two chunks and neither reads as a complete answer. A 50-character overlap keeps sentences whole. Chunk size is a tradeoff: too small and you lose context, too big and you dilute the match with irrelevant text. I start at 500 and adjust based on how the answers read.

The prompt is a guardrail

This is where you tell the model, in plain language, that it is not allowed to make things up.

GROUNDED_PROMPT = """
You are a support assistant. Answer using only the context provided.
If the context does not contain the answer, say "I don't have that
information, but I can connect you with the team." Never guess. Never
use knowledge outside the context.
"""

That one instruction, "if the context does not contain the answer, say you don't know," is the difference between a bot that admits its limits and one that invents a refund policy. Models follow this well when the escape hatch is explicit. Give it a graceful way to say "I don't know" and it takes it.

Show your sources

Because every answer is grounded in retrieved chunks, you can show which ones. I return the source documents alongside the answer so the user can verify, and so I can debug bad answers by checking whether the retrieval pulled the right context or the wrong one.

return {
    "answer": response,
    "sources": [{"title": c.title, "url": c.url} for c in chunks],
}

When an answer is wrong, this tells you immediately whether the model reasoned badly or the retrieval fed it the wrong facts. Nine times out of ten it's retrieval, which means the fix is better chunking or a better embedding, not a bigger model.

Handling the "I don't know" case well

A concierge that says "I don't have that information" too often feels broken, even when it's being honest. So the escape hatch should route somewhere useful: a human handoff, a contact form, a link to the right page. The goal isn't to answer everything, it's to never answer wrongly and always offer a next step. Customers forgive "let me connect you with someone." They don't forgive being told the wrong price.

Keep the knowledge base fresh

RAG is only as current as your data. If prices change and the embeddings don't, the bot confidently quotes yesterday's numbers, which is its own kind of hallucination. I run ingestion on a schedule, or trigger it when the underlying content changes, using the same background-job setup from FastAPI with Celery. Re-embedding a knowledge base is exactly the kind of slow work that belongs on a queue, not in a request.

What this buys you

Done this way, the concierge answers from your facts, admits when it can't, cites its sources, and stays current. It stops being a liability and starts deflecting real support volume. The model matters far less than the plumbing around it. A mid-tier model with good retrieval beats a frontier model that's guessing, every time.

RAG isn't the hard part. Trustworthy retrieval and an honest prompt are, and those are engineering problems, not AI ones.