Sentiment analysis of phone calls has been around in the telephony world for a number of years. Traditionally used in highly regulated and customer service companies to track customer satisfaction and required specialist applications.
OpenAI with ChatGPT has taken the tech industry by storm in 2023 and left us in awe of what is now possible with Generative AI learning models, or artificial intelligence as it’s referred to.
Here at Callroute, we wanted to see how AI could add value to our customers by integrating ChatGPT into our product.
The product team discussed multiple opportunities and decided on adding AI to our existing call recording solution.
Using ChatGPT we are able to transcribe call recordings into text and then process them through OpenAI to determine the sentiment of the conversation and feed the result to customers using a simple classification of positive, neutral, and negative sentiment.
Going further, we asked ChatGPT to tell us what it thought was the overall emotion of the call using three adjectives to provide more meaningful data to customers.
The results from ChatGPT are incredibly accurate.
With our new call analysis feature released today (19th April 2023), customers who use our call recording product now have the option to leverage AI generated call sentiments with every recording.
Positive sentiment is displayed as a green smiling face, neutral a grey straight face, and a negative call as red sad face.
Hovering over the sentiment icon, you can view the emotion of the call as determined by ChatGPT. To download the transcript, simply click the face. You can also listen to the recording play by clicking on the sound/play icon.
Comparing ChatGPT Sentiment Analysis to a Traditional Analyser
Using ChatGPT for sentiment analysis instead of a traditional call analyser such as CallMiner should be considered carefully.
The number one point to remember when using ChatGPT for call sentiment analysis is that it relies on a text transcription to determine the sentiment. It is not yet capable of analysing raw audio to understand the tone of voice or the human emotion and context hidden within the conversation. This means that the call sentiment is based purely on a textual representation of the conversation.
In conjunction with this limitation, the sentiment decision is only as good as the generated text. ChatGPT is excellent at transcribing audio into text with an accuracy rate of 99%+ based on English as the language source.
Whereas traditional analysers are not only able to transcribe, but they are able to analyse the audio of a conversation and measure voice tones and pitch to determine the sentiment of the call.
As a result, traditional analysers may provide a more certain determination over the call sentiment versus ChatGPT at this moment. However, it will not be long before AI improves to close this gap.
Limitations of Using ChatGPT for Call Analysis
As mentioned, the biggest limitation of ChatGPT for call sentiment analysis is its current inability to analyse audio. Everything must be converted to text, so a lot of human emotion and context is removed before AI can analyse sentiment.
The second limitation is that because ChatGPT relies on the full conversation to be provided as text as its model, there is no way currently for it to determine who said what in the transcription.
In-call Sentiment Analysis
Traditional Call Analysers are able to provide deeper in-call sentiments because they intercept in-progress calls and analyse in real-time. Could ChatGPT do something similar to provide in-call sentiment change?
Yes, it can. Although it will not be real-time, ChatGPT’s transcription provides additional metadata that identifies a data point that can be converted back to time. In addition, you can split the conversation into different models. For example, splitting the conversation into individual sentences and asking ChatGPT to provide a sentiment decision for each.
Using the sentiment per sentence and the transcription metadata, you can overlay the sentiment into each section of the call showing a timeline of how the sentiment changed over the call duration.
You can then ask ChatGPT to provide a sentiment of the overall call for a summary determination.
Cost of ChatGPT Call Sentiment Analysis
Using ChatGPT for call sentiment analysis requires the use of two AI products, Whisper and Completion. Whisper is used to transcribe audio to text and completion is used to determine the sentiment of the text.
Whisper costs $0.006 per minute of audio to transcribe. While the text AI processing (sentiment) is priced from $0.03 per 1,000 tokens for the context (the question you ask it) and $0.06 per 1,000 tokens for the model (text input) based on the 8K context (its intelligence level, 32K is the more intelligent version).
What is a token? A token is generally either a word or a punctuation. “Mary had a little lamb.” would cost 6 tokens for example.
Typically, our findings show that 1,000 model tokens is about 4.5 minutes of normal conversation. Based on this, the cost of sentiment analysis per 4.5-minute call would cost about $0.12c.
For customers looking to provide added value to their call recordings to help them identify staff training or customer improvement opportunities, sentiment analysis provided by Callroute using ChatGPT could be an affordable option versus traditional analysers.
The ability to use sentiment analysis across all phone calls not just contact center can provide increased intelligence to better improve overall customer satisfaction and company performance without the need for integration or hardware is another huge benefit.
If you would like to learn more about how Callroute can add value to your voice communications, why not schedule a demo