A chatbot that answers customer questions is now easy to add to a website. A chatbot that answers them correctly, stays inside what your business can promise and hands over to a person at the right moment takes more thought. This note covers what chatbots are good at, where they fail, and the design choices that decide whether yours helps or hurts. We build AI solutions, so consider our view with that in mind. We also say plainly that for some businesses the right answer is not to add one yet.
What a chatbot does well
- Repeated, factual questions. Opening hours, delivery areas, how to reset a password, where to find an invoice. The answers exist, they rarely change and people ask them all day.
- Out-of-hours coverage. A visitor at night gets an immediate answer or a clear promise about when a person will reply.
- Triage. Collecting the order number, the product and the problem before a person picks it up saves everyone a round of questions.
- Finding things. Pointing to the right page, form or document in plain language is often more useful than a search box.
Where chatbots fail
- Inventing answers. Language models can produce confident text that is wrong, including prices, policies and promises that do not exist. This is the central risk.
- Edge cases. An unusual order, an angry customer or a complaint with several parts is exactly where a script or a model gets lost.
- Sensitive situations. Bereavement, health, money trouble or safety. These need a person.
- Anything needing judgement or authority, such as approving a refund outside the policy.
- Dead ends. A bot that cannot understand and cannot hand over leaves the customer worse off than having no bot.
You are responsible for what it says
In 2024, a Canadian tribunal ruled in Moffatt v. Air Canada that an airline was responsible for wrong information its website chatbot gave a customer about a fare. The airline had argued that the chatbot was a separate entity. The tribunal rejected that. A summary of the case is worth reading. It was decided under Canadian law and is not binding in the UK, but the point is a sound one: to the customer, the chatbot is the business. Treat every answer it gives as something you published.
Design choices that matter
Answer from your own material
The safest pattern is to have the bot answer only from documents you control, such as the help centre, the policies and the product pages, and to show the source of each answer. When the material does not contain the answer, the bot should say so and offer a person, rather than guess. This is often called retrieval-based answering, and it greatly reduces, but does not remove, wrong answers.
Limit what it may do
Write down what it may answer, what it must never answer (prices not published, legal or medical advice, refunds outside the policy) and when it must hand over. Give it no power to change orders or money unless you have tested that thoroughly, with confirmation steps for the customer.
Make the handover easy
Offer a clear route to a person at any point, pass on the conversation so the customer does not repeat themselves, and tell the customer what to expect: who will reply and when. If nobody is available, say so honestly.
Be open that it is a bot
Tell visitors they are talking to an automated assistant. This is also good practice for data protection; the Information Commissioner's Office has guidance on AI and data protection covering transparency, fairness and how personal data is used.
Mind the personal data
Customers will paste names, addresses and sometimes much more into a chat box. Decide what is stored, for how long and who can read it, tell people in your privacy notice and check where your AI supplier processes data and whether it uses conversations to train its models. Avoid collecting what you do not need.
Test before customers see it
- Collect a few hundred real questions from your email, phone notes and chats.
- Run them through the bot and mark each answer correct, partly correct, wrong or refused.
- Look hardest at the wrong answers. Fix the material, the instructions or the limits, then rerun the same set.
- Try to break it: ask for discounts, ask it to ignore its rules, ask about things you do not sell.
- Let your own staff use it for a while before the public does.
Measure, review, adjust
After launch, read a sample of conversations every week at first. Track how many conversations are resolved without a person, how many are handed over, how many customers ask for a person straight away, and what the bot could not answer. The last list is useful in itself: it shows the gaps in your documentation. Update the source material whenever a policy changes, because a bot that quotes last year's policy is worse than none.
Start small
Pick one task with clear answers, such as order status or delivery questions. Make it work well, show that the numbers improve and that customers are not frustrated, then add the next. If you cannot yet write down your own answers clearly, fix that first. A chatbot repeats whatever the documents say, including their gaps.