Consider a two-person SaaS whose support inbox is mostly six subjects: reset my password, why did my card fail, how do I add a teammate, where is the export, does this integrate with what I already pay for, and how do I cancel. This is a hypothetical, but you probably recognise the shape. Every one of those is answered in the docs, and every one still arrives as a ticket, because people search documentation badly, give up, and type at a human.
An AI documentation chatbot is the obvious response, and the obvious trap. Retrieval over text you already wrote has a clean pass-fail test: the answer is in your docs or it isn't. The trap is that almost everything deciding whether it works happens before a model is called, while most tutorials spend their words on the part that matters least.
Your first action today: pull last month's ticket subjects and pick ten questions whose answers you already know. That list is your eval set, and it should exist before any code does.
What does an AI documentation chatbot actually do?
An AI chatbot for documentation retrieves the passages of your docs that match a customer's question, then asks a model to answer using only those passages and to cite them. That constraint is the product.
Remove it and you have a general-purpose model guessing about your pricing tiers in front of a paying customer. The pipeline underneath is short: ingest the docs, split them into chunks, embed each chunk as a vector, embed the question at query time, pull the nearest chunks, rerank them, and hand the survivors to a model with an instruction that amounts to answer from this or say you don't know. Then you attach source links and decide what happens when retrieval comes back thin.
That is a lot of surface. NVIDIA's 2024 paper on enterprise RAG chatbots, Akkiraju et al., "FACTS About Building Retrieval Augmented Generation-based Chatbots", draws on three internal chatbots and presents fifteen RAG pipeline control points, covering things like embedding choice, query rephrasing and reranking. Fifteen places to tune, and only one of them is the model.
Why does retrieval matter more than the model?
Because the failure you meet in production is usually a fluent answer built on the wrong chunk, and no model swap fixes a retrieval miss. Chunking, data quality and reranking decide what the model gets to see.
kapa.ai, a vendor that builds docs bots, published a field guide to seven RAG mistakes in January 2026. Bad chunking and poor data quality are numbers one and two, a missing reranker is number four, and none of the seven is "you picked the wrong LLM". It is one vendor's list, so treat it as a checklist rather than a measurement.
Chunk on structure, not on token count
A fixed 512-token window can slice a sentence in half and sever a heading from the procedure under it, so the retrieved chunk reads like a fragment of a manual for a different product. Split on the boundaries your technical writer already put there: headings, then paragraphs. Your docs are a tree, and the tree is free signal you can either throw away or use. This is my recommendation from building retrieval pipelines, not a benchmark result, so test it on your own ten questions.
Contradictions produce confident blends
Picture a docs page that says the free tier allows three seats, a changelog entry from four months ago that says two, and a migration guide that says three but links the old billing page. Retrieval returns all three with near-identical similarity scores, and the model writes a smooth blended answer that is wrong. The model did what you asked. You supplied contradictory context and no rule about precedence. Fix it upstream: date your docs, put a canonical-source flag in chunk metadata, and instruct the model to surface conflicts rather than resolve them.
Should you build it, buy it, or rent the retrieval layer?
Rent the retrieval layer for a first version: a managed RAG API keeps chunking control and your own chat surface, while skipping vector-database operations. Buy a support-platform agent if you have no engineering time; build the full pipeline only when retrieval quality is your proven bottleneck.
| Approach | What you operate | Where it breaks |
|---|---|---|
| Support-platform agent (Fin, Zendesk AI, Intercom) | A doc sync and a fallback rule; the vendor owns retrieval | Content the crawler mangles, no control over chunking, per-resolution pricing that scales with your growth |
| Managed RAG API (Gemini File Search, OpenAI file search) | An upload job, a chunking config and your own chat surface | A ceiling on retrieval tuning; you still build escalation, analytics and the widget |
| Self-assembled pipeline (vector DB, embeddings, reranker) | Everything, including the re-index job and the eval suite | Maintenance: it becomes a service with a runbook and an on-call burden you did not budget for |
The middle row is cheap to try. Google's File Search documentation says you are charged for embeddings at indexing time, storage costs nothing, and query-time embeddings are free, although document tokens returned to the model are billed as ordinary context. Google's pricing page lists Gemini Embedding 2 at $0.20 per million text tokens (checked 30 September 2026). As arithmetic, not a quote: a hypothetical help centre of 500,000 tokens would cost about ten cents to index. Prices change, so read both pages before you budget.
Why the bot has to cite, and what it costs when it doesn't
Every answer should link to the doc section it came from. That is your liability boundary and your only cheap evaluation signal.
In February 2024 the British Columbia Civil Resolution Tribunal held Air Canada responsible for wrong information its website chatbot gave a customer about bereavement fares. McCarthy Tétrault's write-up of Moffatt v. Air Canada records the tribunal's reaction to the airline's argument that the chatbot was a separate legal entity: "This is a remarkable submission." The chatbot sat on Air Canada's own site, so the airline answered for it.
My reading, labelled as inference: anything your bot says about pricing, refunds, SLAs or data handling should be treated as a statement your company made. A citation lets the user check the claim against the canonical page. It also gives you a log of which section produced which answer, and without that log you cannot tell "the docs were wrong" from "retrieval missed" from "the model invented it". Those need three different fixes. If retrieval falls below your confidence threshold, the bot should say it doesn't know and hand over.
How often should you re-index when the docs change?
Re-embed when roughly 10 to 15 percent of your data has changed, and hang the trigger on your docs deploy rather than a calendar. That threshold comes from kapa.ai's guide, which calls the alternative embedding rot and says an unmaintained RAG system becomes outdated in under six months.
Nothing errors when this goes wrong. Your bot keeps answering, just from the docs you had last quarter. If your docs live in git, add a pipeline step that re-indexes changed files and stores a content hash per chunk so unchanged sections are skipped. If they live in a CMS, use the publish webhook.
The escalation path is a product decision
Give the human agent the transcript, the retrieved chunks and the confidence score, so no ticket starts with "the bot said something, no idea what". Designing that handoff is what decides whether your support team defends the bot or routes around it.
What does it cost per resolved ticket?
Your real resolution rate decides the economics, not the vendor's headline. Intercom's Fin site says it averages 76% resolution across 12,000+ customers, a claim I read on 30 September 2026 and cannot verify independently.
A vendor number is measured by the vendor's definition of resolved. A conversation that ended without a human can also describe a customer who got a confidently wrong answer and left, so I would not plan headcount around it. Measure your own: tag every conversation the bot handled alone, sample fifty a week, and read them yourself. Have you read fifty transcripts from your own bot? That is a better question for a board deck than any deflection percentage.
Ship it in this order
- Write the ten-question eval set from real tickets before any code.
- Index the docs with a managed RAG API. Chunk on headings and keep the config in version control, so a chunking change is a reviewable diff.
- Turn citations on by default and instruct the model to answer only from retrieved context.
- Set the confidence floor and build the handoff before a customer sees it: transcript, chunks, score.
- Rate-limit and scope it per tenant. The Calia AI assistant in Callidus OS, which I built on React, Firebase and Stripe Connect, runs on Vertex AI with thirteen prompt-injection patterns, 30 and 100 requests per hour limits and a 90-day GDPR cleanup, because an AI surface inside a multi-tenant product is an attack surface before it is a feature.
- Re-run the ten questions after every docs deploy. A regression then arrives as a bug with a cause.
If you are still choosing what sits underneath, the best stack for a SaaS MVP will settle more about your delivery speed than the bot does, and my AI-augmented development workflow describes how I build these. For AI surfaces beyond support, see AI SaaS solutions for business, and if the vocabulary is new, start with the definition of SaaS.
Tonight, open your help centre and search it with a frustrated customer's words rather than your headings. How many of your top six ticket subjects can you answer in under thirty seconds? If the number is low, your first sprint is documentation, not machine learning.