In this part of our blog, lets see how chatbot metrics, which you choose for your chatbot, influence conversational analytics, and set you up for chatbot success. If you are feeling lost about the relevance of metrics and conversational analytics for chatbots, you can find part 1 of this blog here. Reading our blog on crafting conversational UI will weed out any other doubts that might have about the relevance of this topic.
Previously, we have discussed about the importance of measuring the performance/efficiency of your AI chatbot. We went further in detail about how choosing the right metrics is important to measure your chatbot’s effectiveness. We also discussed the broad classifications of metrics that we thought were important for measuring your chatbot’s performance.
Having discussed how user metrics can inform some of your analytical decisions, let us now discuss the importance and relevance of message and other key chatbot metrics.
A walkthrough on message metrics
While user metrics inform you about the overall trend in your user base, they are not enough to craft individual’s interactions with your chatbot. Message metrics will help you gather details that will help you in crafting individual experiences, speed up customer service, and increase engagement. Here are the ones we believe are most relevant.
Conversation-starter messages
This is the number of messages where the conversation is initiated by the chatbot. This is a crucial metric for measuring how organic your platform is. Compelling your chatbot users to engage with the bot by sending out messages is something one can do only in the initial stages. As time passes, organisations should focus on lowering this metric’s numbers. If your chatbot’s main focus is on digital marketing or customer relation management, it will serve you well to have the engaged users that interact with your chatbot directly.
Bot messages
Bot messages are the number of messages sent by the bot during a conversation with active users. The more the number of messages sent by the chatbot in a conversation, the longer it is. Longer conversations are better in terms of metrics, however, care should be taken that the chatbot’s replies are all standard. We don’t want it stuck in a loop repeating the same message over and over, this generally happens when bots cannot comprehend the users’ input.
In messages
This metric records the number of incoming messages from users in a conversation. This will inform you how engaged your users feel in the conversation, if it’s significantly low, you probably should rethink using a chatbot. In such cases you are better off using a social media page or account to reach out to your user base.
Miss messages
This metric captures the number of messages that your chatbot could not process. This metric may be hard to calculate, you need to record all the times a message is misinterpreted by the chatbot. This becomes a key metric when you start engaging users from a different region than you currently serve due to the idiomatic use of language.
Total conversations
This metric records all the successful conversations during a day. That means the conversation was started, relevant information was exchanged and the conversation reached a proper end. This will also point to the number of times your chatbot engaged users thoroughly.
New conversations
This metric captures the total number of conversations started by both new and returning users. While new users are inexperienced with the platform, returning users might be interested in a different topic, a new problem or just about another order. User metrics and message metrics can act as key metrics for metric construction. This will in turn will facilitate the creation of other traditional digital marketing metrics.
Understaning bot metrics
Monitoring AI chatbot performance metrics is key when understanding conversational analytics. Only when you can measure how your bot responds in different situations, can you draw informed insights about its performance. Here are some metrics to consider.
Retention Rate
This metric records the number of users that return to interact with your chatbot within a specific time frame. Since you need to engage your users for a longer time to generate insights about their preferences, this metric plays a very important role. You can achieve higher retention rates by making conversations more engaging or using promotional campaigns. If this can be achieved organically, it is the best. A high-quality chatbot that caters to the needs of customers can ensure higher retention rates.
Goal Completion Rate (GRC)
This metric is helpful when tracking the percentage of successful engagement through a chatbot. Every user is unique and will want to access different services and information on your website. In any given scenario, when the bot completes a task assigned to it the GRC increases. For example, if the bot can be responsible for delivering updates about an order, every time it completes this task, the GRC will improve. This metric basically allows you to record every time your bot receives an input and responds with relevant information.
Goal completion time and mode
This metric will record each time you accomplish a goal and the modes that you choose. This helps in informing your decisions about which are the best modes of interaction with your bot and fastest. This metric helps when you are trying to figure out new layouts. Minimising the effort in goal completion can improve your customer experience.
Fall Back Rate (FBR)
The FBR metric records the number of times your bot has failed or reached a near failure situation. Generally, we prefer lower FBR, in case of higher values for FBR consider changing the source of data.
Here's another guide on chatbot metrics to track!
Do check out this 55+ tech podcasts to know more on how digital devices have change our lives.
Haven’t built a bot yet? We can fix that right away, click here to begin your bot building journey!
