More than ten years ago, the marketing world grew accustomed to Google Analytics, the measurement solution soon becoming as ubiquitous as websites. With chatbots growing in use among brands today, Google is looking for a second lightning strike with Chatbase, a cloud-based analytics platform that was in beta earlier this year. Chatbase helps marketers measure bot performance answering customer care and support questions, showing subject-matter expertise, and curating relevant product or service information.
In my post Chatting About Your First Chatbot, I explained that chatbots are popular because of their ability to engage customers quickly with answers tailored to their requests.
Chatbase is aimed at evaluating chatbots engagement in order to make the customer’s digital experience better. It also reflects a significant nuance in analytics planning due to elements of basic bot design.
How bot analytics help marketers
Let’s look at the shift in digital media to understand how bot analytics can benefit marketers. When you launch a website you establish the website structure. You develop it on a wire frame, select content, and most likely optimize content through A/B testing. You have an overall starting point to review content.
Chatbots introduce more dynamic content via (often AI-driven) responses to user requests. User requests are more varied than static content that appears on a web or app page. This means a fluidity exists for metrics that does not exist when interpreting the reception of static content from a page, link, or media element.
Chatbase is designed to match those dynamic metrics by providing conditional analysis surrounding user requests. Analytics platforms have normally reported metrics based on observation — in this instance, observed activity on a webpage. In the case of bots, machine learning helps analysts sift through user request conditions to derive causal observations — based on what a user does.
Chatbase in practice
- As a first step, analysts view Chatbase reports in a browser dashboard similar to that of Google Analytics. Straightforward volume metrics are displayed. A timeline for user growth is shown at the top of the dashboard, breaking down active users by months, weeks, and days. The metrics can be compared across chat platforms, allowing marketers to optimize under-performing ones.
- There is also a Session panel that indicates the level of engagement. The metrics include daily sessions, sessions per user, and user messages per session. There is no benchmark indicator in this panel. Marketers can note session time per user, and user messages per session, as indicators of successful engagement, but would have to keep separate records to determine if this metrics are falling into a range for standard performance.
- There is a Retention panel, which arranges data in a similar to the cohort reports in Google Analytics. It arranges retention activity by week after a given period.
- The Messages report is designed to identify and cluster poorly-received requests that the bot encounters. It displays two columns. One column shows users queries were “handled” – meaning that it took an action as a result of the request. The other column “not handled” shows which phrases or responses received no action. A machine learning algorithm sifts through these phrases and highlight problems. The end result is providing opportunities to make request responses better.
- The Transcripts report, with a layout similar to the Messages Report, helps analysts conduct a conversation drill down quickly, saving development time in pinpointing where that interaction can be conducted more efficiently.
- The Session Flow report visualizes activity across sessions that lead to a conversion rate. The graphic looks similar to visitor flow reports in Google Analytics, but it displays the request phrase with indicators of successful tasks or unanswered requests. There is also a funnel report graphic as well. These graphics can complement the Transcripts report for additional insights into drop-outs from chatbot engagement.
With digital commerce increasingly involving voice and immersive content on messenger platforms, analytics must account for chatbot engagement. Chatbase is able to analyze how customers are being served on messenger platforms, and has the capacity to operate on the popular platforms such as Facebook Messenger, Slack, and Skype. With a robust analytics installed, the user can refine chatbots for better speech recognition, better responses, and better customer experience.
