05 August 2024
Large language models performed very well in classifying the freely written feedback text and proposed draft measures, too.
The wellbeing services county of Central Finland is responsible for the social, health and rescue services of around 273 000 people in central Finland. Feedback on services is collected through a variety of channels: a form on the website allows feedback on all services in the wellbeing services county, while SMS feedback and tablet devices are used for limited services. Open feedback is also available through each channel.
”The analysis and classification of open feedback came up in our discussions with Tietoevry, and we decided to investigate whether generative AI could be used to classify open customer feedback data," says Service Manager Jaana Peltokoski from the Information Management service area of the wellbeing services country of Central Finland.
The pilot utilised the Microsoft Azure OpenAI service, where the classification was entirely done by generative AI using large language models. In the pilot, more than 4 900 open, anonymised customer feedback given in 2023 were classified into different categories, such as appointment, queuing, telephone or customer service, or encounter. Most of the feedback was related to reception, treatment, encountering and treating the patient, or customer service. Feedback was also divided according to tone into positive, neutral and negative.
The results of the pilot surprised all parties
The results surprised both the research team and Peltokoski. The large language models performed very well in classifying open feedback data, and they even recognised the tone of voice of the feedback. From the customer feedback data, it was also possible to determine with relative reliability what type of measures should be taken based on the feedback given.
”It’s great that we had the opportunity to participate in this pilot with Tietoevry. It gave us insights and perspective on how AI can be used in open feedback and text analysis. The automated analysis and processing of open feedback clearly showed that information can be generated faster than currently for the use of services and as a basis for service development,” Peltokoski says.
“We do have feedback channels, but the most important thing is what we do with the information and how we process it. Automating the analysis of open feedback would free up resources to be allocated to things like coaching and training on customer experience processes."
Peltokoski considers the most important development area of multi-channel customer feedback management to be more systematic utilization of feedback information and its integration into service development work.
The goal is to respond quickly to customer feedback
”Central Finland is a customer, who boldly wanted to pilot new technology to classify customer feedback data and take the initial step in using AI to improve resident services," praises Head of AI and Data Solutions Niina Siipola from Tietoevry Care.
”For years, Tietoevry Care has been building the foundation for using data and AI in social and healthcare services. Our team has a long experience in the use of AI and the development of language models, and therefore we know where AI is suitable and where it is not. Based on this pilot, it seems perfectly suited for automating manual analysis work. While AI cannot solve all challenges, it can act as an effective assistant in this case, saving resources and helping to prioritise feedback”.
The ongoing changes in welfare service counties may give people the feeling that they are not being heard and that they cannot influence things.
”We strive to make it easy for customers to give feedback and that it leads to swift action. The aim of the wellbeing services county is to ensure that our residents’ experience of wellbeing, health and safety is among the best in the country. Therefore, customer feedback data must be commensurate and transparent for the customers and staff of the wellbeing services county. We want to communicate to our customers that the feedback we receive is valuable to us and that the information will be used to improve our operations and services,” concludes Jaana Peltokoski.
All photos: Mika Nuorva
Would you like to learn more about knowledge-based management?
Read more:
Lifecare Lakehouse Analytics (tietoevry.com)
A new era of knowledge-based management in the wellbeing services counties