Pause, pivot, and find the valuable use case: learnings from Allstate’s conversational AI journey

We invited Hiremath to share his insights into the benefits and challenges that arise when AI meets the art of human-to-human conversation.

Conversational AI is a set of systems that enable humans to have a conversation with a computer, as Hiremath explains. Amazon Alexa, Google Assistant, and Siri are the commonly used consumer products, where Natural language processing (NLP) and machine learning (ML) concepts are used to simulate conversation, and where the machine is designed to understand user requests and provide relevant responses. Used in the enterprise setting, conversational AI not only ensures that companies are available at their customers’ preferred time and channel, but also helps respect customers' time by eliminating long queues.

Sharpen the use case and scale

Allstate is one of the companies that have successfully implemented this technology. The Allstate team had ambitious plans when they first set out to create a standalone virtual assistant in 2017. The intent was that the assistant could answer any question their agency users had. Even though the solution was built on the latest technologies at the time, they quickly realized that the virtual assistant ended up being just one more tool in front of the users.

"We thought we created something valuable, and people would come and use it. But the users didn't see the value. The feedback was that there are too many tools already. We had to take a pause and pivot from there."

Listen to Data Insiders podcast episode with Ajay Hiremath on Spotify.

The team set out to repurpose the solution and introduced it in the existing chat channel, as a “first line of defence”, as Ajay Hiremath calls it. If the chatbot was trained on the topic, it would provide the answer. If not, it would connect the conversation to a human representative. This approach started to gain trust among the users, enough for Allstate to eventually shut down the phone channel in the contact center.

As a result, the volume in the chat channel exploded and started to cause significant queue times for insurance agents waiting for service. The solution performed well with new users whose questions were simple, but it couldn’t help with complex conversations with experienced agents.

The next step for Ajay Hiremath and his team was to change the scope of the chatbot once again, now to route the conversations to the appropriate specialized team in the contact center. This approach took the complexity off from insurance agents’ shoulders and gained good feedback from the users, as it allowed them to ask their question and trust that the chatbot would figure out who needed to address it. As Allstate had now found the narrower use case that provided real value to the users, they were able to successfully scale the solution.

“As we saw benefits in agency channel, we ventured into customer service and sales and started introducing conversational AI to all these channels. Now we have a conversational layer in our claims, roadside assistance, and as an employee-facing solution in the contact center.”, Hiremath lists. But what are the key learnings from this eventful journey that took about 5 years and included some twists and turns?

“The lesson learned was that we should not think about conversational AI as one size fits all solution. We need to be very specific with the use cases we want to go after, as well as the user community that we are targeting.”

Plug-and-play architecture

To make sure they can keep up with the evolving technology, Allstate invested in building their own core technology platform to orchestrate different AI solutions and connect to their existing applications. This gives them flexibility and freedom to onboard Amazon Lex, Google Dialogflow, Facebook Messenger or any solution from emerging niche players, depending on what best suits their needs.

Ajay Hiremath sees the integrations to the legacy systems as one of the key success factors for their conversational AI journey, as they enable end-to-end automation of processes. He gives a simple example of a password reset. “One solution would be that a chatbot provides step-by-step guidance to reset the password. Another option, that we prefer, is to provide end-to-end automation through integrating chatbot with identity management systems and enable it to reset the password automatically after authenticating the end-user.”

For companies with large consumer and user base, there’s great potential in providing self-service by integrating the latest AI technologies with legacy solutions.

“For us to be successful, we need to be 100% digital, and available to our customers in the channel that they want to interact with us at the given time. Conversational AI fits very well into this aim. It is a key component in our ecosystem, and I see the importance only growing in the coming years.”

Measuring the performance

Allstate evaluates the performance of their conversational AI solutions by containment metric, meaning how many of the conversations were fully handled by the chatbot. Hiremath is proud of 38-40% containment achieved in the consumer channel.

"Out of the million conversations, around 400,000 are completely managed by the bot. That’s a good success for us!"

Another key metric is intent accuracy, which measures how well the bot can understand users’ intent. Allstate’s intent accuracy is high: 80% of the time, the chatbot understands the reason why a user is interacting with Allstate. The remaining 20% of the conversations go to the pool of utterances that developers and designers use to optimize the solution.

Looking forward to the future, Hiremath shares insight into Allstate´s roadmap:

“We want to make our chatbots even more effective. We are also looking at newer technologies and right now our focus is on integrating knowledge graphs into the chatbots and the conversational AI platform itself.”