Chatbot Analytics: 15 KPIs You Should Track

Chatbot analytics answers a simple question: does the bot help your customers and your business, or does it just produce conversation logs? This article presents 15 chatbot KPIs, explains how to calculate each one and shows which five metrics a small business should sensibly start with. There is also a monthly routine for evaluation.

Before you measure: define the chatbot's goal

An online shop for bike accessories uses its chatbot mainly for questions about delivery times and sizes. A beauty salon wants to book appointments through it. A tax firm only takes callback requests. All three can collect the same metrics, but not every one matters equally to each of them.

So write down in one sentence beforehand what the bot is for. For example: it should answer standard questions immediately and enter appointment requests into the booking system without further queries. This sentence tells you which metrics you really need to look at. All the others are background noise.

The 15 most important chatbot KPIs at a glance

KPI	What it shows	How to calculate it
1. Completion rate	Conversations that reach their goal	Conversations where the goal was reached divided by all conversations
2. Conversation length	How long a request takes	Average from start to last message
3. Satisfaction	How customers rate the conversation	Short rating at the end of the conversation
4. Recognition rate	How often the bot understands the request	Check a sample: correctly recognised divided by checked
5. Handover rate	How often a person has to take over	Handovers divided by all conversations
6. Response time	How quickly the bot responds	Time between customer message and reply
7. Conversion	Purchases, bookings or enquiries after the chat	Conversations with a conversion divided by all conversations
8. Returning users	Whether customers use the bot again	Users with more than one conversation per month
9. Error rate	Wrong or unsuitable answers	Check a sample: errors divided by checked
10. Lead quality	Whether contacts from the chat turn into orders	Orders from chat leads divided by chat leads
11. Cost per conversation	What one conversation costs	Monthly costs including maintenance time divided by conversations
12. Drop-off rate	Where customers leave the conversation	Drop-offs per conversation step
13. Resolution time	Until a request is fully dealt with	Time from first message to resolution, including after handover
14. Add-on sales	Additional purchases after recommendations	Orders with a recommended item divided by recommendations
15. Recommendation	Willingness to recommend the service	Net Promoter Score from a short survey

Not every one of these figures is supplied automatically by your system. Recognition and error rates, for example, can only be determined honestly if a person regularly reads a sample. In return, they are often the most revealing.

Five chatbot KPIs to start with

Small businesses should not start with 15 metrics. These five give a usable picture in the first few months:

1. Completion rate: does the bot do what it is there for?
2. Handover rate: how much work still ends up with the team?
3. Drop-off rate per step: at which point do customers leave? Often it is a single, clumsy question.
4. Error rate from the sample: does the bot give wrong information, for example on prices or opening hours?
5. Conversion or lead quality: do the conversations lead to bookings, orders or jobs?

Add the remaining KPIs when a specific question comes up. Add-on sales, for example, only become interesting once your shop bot recommends products, as described in the article on product recommendations in e-commerce.

Chatbot analytics as a monthly routine

One hour a month is enough for most businesses. Here is a practical routine:

1. Take the five core figures from the dashboard and put them next to last month's values.
2. Read twenty random conversations, including all those with a poor rating.
3. Note the three most common questions the bot could not answer.
4. For each of these questions, add to the knowledge base or decide that the bot hands over.
5. Check whether handed-over conversations were dealt with promptly by the team.

A made-up example shows what this can look like. Suppose a beauty salon notices that many clients abandon the booking conversation at the same point: the question about the desired treatment, which the bot expects as free text. The sample shows that clients are unsure what the treatments are called. The salon changes the question to a short selection of the five most common treatments plus an option for other requests. The following month, it checks whether drop-offs at this point decrease. The routine is designed for exactly these small, targeted corrections.

Record the results in a simple table. After six months, you will see whether the figures are moving and whether your changes have had an effect. Our article Measuring chatbot ROI shows how to turn this data into a cost-benefit calculation.

Typical misinterpretations

A low handover rate looks good, but it can mean the bot fobs customers off with unsuitable answers instead of handing over to a person in time. So always read the sample before you celebrate the figure.

Short conversations are not automatically good conversations. Someone who gives up after two messages also shows up as a short conversation. And high satisfaction based on few ratings says little, because usually only very satisfied or very annoyed customers leave a rating.

Pay attention to data protection when analysing the logs: set deletion periods, restrict access to a few people and mention the analysis in your privacy policy. For a comparison with chats handled by people, see our article Chatbot vs. live chat.

Frequently asked questions

Which metric is the most important?

The one that fits your bot's purpose directly. For an appointment bot, it is the number of appointments booked; for a service bot, the completion rate without handover. But one metric on its own is never enough; it always needs the sample as a cross-check.

How many conversations do I need for meaningful figures?

With only a few dozen conversations a month, rates fluctuate a lot. In that case, look at longer periods and read more conversations in detail rather than comparing percentages.

Does the AI notice change my metrics?

The number of customers who ask straight away for a person may go up. That is no reason to leave the notice out, because the EU AI Act requires transparency. Better to track these requests as a separate metric and offer an easy way to request a callback.

Does every chatbot system provide these reports?

Most provide conversation volumes, duration and handovers. Conversion and lead quality usually require a connection to the shop, booking system or CRM. Before buying, ask exactly which data you can export.

Conclusion

Good chatbot analytics is not a graveyard of numbers but a short monthly routine with five metrics and twenty conversations read. If you work this way, you spot weaknesses early and can improve the bot in a targeted way. The guide to implementing a chatbot shows how to set up the bot to be measurable from the start. For online retailers, the page AI sales assistant for online shops gives an overview.