Chatbot analytics: the metrics that show real performance

Which chatbot metrics reveal whether your assistant genuinely helps customers and the business, how to review conversations and how to turn insights into improvements.

By the UK Technology Hub teamPublished 4 minute read

The short answer

The most useful chatbot metrics measure outcomes: issues genuinely resolved, customer satisfaction, accuracy of answers, successful handoffs, conversions or bookings and cost per resolution. Raw conversation counts and "containment" can mislead if customers simply give up. Combine metrics with regular conversation reviews, track unanswered questions and use findings to improve content and flows.

A chatbot dashboard showing thousands of conversations looks impressive. But were customers helped? Did they give up? Did they buy? Meaningful analytics answer those questions.

Core metrics

Resolution rate

The share of conversations where the customer's need was met. Confirm resolution through customer feedback, actions completed or the absence of follow-up contact.

Containment rate, with caution

The share of conversations handled without human involvement. High containment can mean success, or that customers gave up. Always pair it with satisfaction and resolution.

Handoff rate and reasons

How often conversations go to people, and why. Our chatbot handoff guide covers designing good handoffs.

Customer satisfaction

Short ratings at the end of conversations, such as thumbs up or down, plus optional comments.

Answer accuracy

Review samples of answers for correctness against your content. Automated checks can flag answers without supporting sources.

Unanswered questions

Questions the bot couldn't answer reveal content gaps.

Business outcomes

  • Leads captured.
  • Bookings made.
  • Purchases influenced.
  • Support tickets avoided.

Cost per resolution

Platform and AI usage costs divided by resolved conversations. Our AI chatbot ROI guide explains how to compare with alternatives.

Diagnostic metrics

  • Conversation length and drop-off points.
  • Most common topics.
  • Fallback responses ("I don't understand").
  • Response times.
  • Usage by channel, page and time of day.

Conversation reviews

Numbers don't tell the whole story. Review a sample of conversations weekly:

  • Were answers accurate and helpful?
  • Did the tone fit your brand?
  • Where did customers struggle?
  • Were sensitive topics handled appropriately?

Protect privacy by limiting access and masking personal data where possible.

Turning insights into improvements

  1. Add or improve content for unanswered questions.
  2. Adjust flows where customers drop off.
  3. Fix inaccurate answers at the source.
  4. Refine handoff triggers.
  5. Add actions for frequent requests.

Dashboards

Build a simple dashboard showing resolution, satisfaction, handoff, top topics, unanswered questions and business outcomes, with trends over time.

Segment the data

Compare performance by topic, channel, language and customer type to find specific problems.

Common mistakes

  • Celebrating conversation volume.
  • Using containment as the main success measure.
  • Never reading conversations.
  • Not connecting chatbot data to business outcomes.

A worked example

An online travel company's chatbot had high usage, but customer satisfaction was mixed. Analysis of conversations showed that many customers asked about changing bookings, which the assistant could only explain, not complete. Others dropped off when asked to log in. The company added a secure booking change action and simplified login within the chat. Resolution rates rose, handoffs fell for booking changes and satisfaction scores improved. The analysis also revealed outdated answers about baggage rules, which were corrected.

Reviewing conversations

Read a sample of conversations each week, including resolved, unresolved and handed-off chats. Look for:

  • Wrong or incomplete answers.
  • Questions the assistant couldn't handle.
  • Confusing replies.
  • Points where customers gave up.
  • Moments of frustration.

Tagging and categorising

Tag conversations by topic and outcome. Categories show which topics drive volume and where improvements matter most.

Satisfaction surveys

Ask a short question at the end of conversations, such as "Did this solve your problem?" Combine answers with behavioural data for a fuller picture.

Sharing insights

Share findings with product, marketing and operations teams. Chatbot conversations often reveal confusing policies, website problems and product issues beyond support.

Privacy in analytics

Mask personal data in analytics, limit access to conversation logs and follow your retention policy.

Setting targets

Set realistic targets for resolution, satisfaction and handoff rates based on your baseline, and review them as the assistant improves.

Cost per resolved conversation

Track what each resolved conversation costs, including platform and model fees. This metric helps you judge efficiency over time and compare the assistant fairly with other support channels. A rising cost per resolution can signal longer conversations, poor answers or unnecessary use of expensive models.

Comparing channels

If your assistant runs on several channels, such as your website, messaging apps and voice, compare metrics by channel. Customers often behave differently on each, and problems may appear only in one place.

How we help

Our AI chatbot development team builds analytics into every assistant, our AI solutions team analyses conversations at scale to find patterns and our web app developers build dashboards combining chatbot and business data.

Frequently asked questions

What is a good chatbot resolution rate?

It depends on use case and complexity. Track your trend and pair it with satisfaction.

Is containment rate a good metric?

Only alongside satisfaction and resolution. High containment can hide frustrated customers giving up.

How often should I review chatbot conversations?

Weekly samples are a good habit, more often after launches or changes.

This guide is general information, not legal, tax or financial advice, and details such as prices, features and rules can change. Product and company names mentioned are trademarks of their respective owners and are used only to identify those products; UK Technology Hub is not affiliated with or endorsed by them.

Related guides

All AI chatbots & assistants guides