Customer focus with Sentiment Analysis
A satisfied customer is the basis for business success. To do this, we have to understand what the customer wants to achieve & what prevents him from doing so.
Author: Amancio Bouza
The customer problem must be identified and a solution must be offered.
But that is easier said than done. An example from the tourism industry: Anita books a river cruise on the Seine to Paris for two people with a travel agency. She goes on this trip and then returns to the agency to give her feedback.
With which yes/no question can the agency find out what Anita's goals were, whether she achieved them and if not, why not? This would require closed questions of this kind:
- Were you planning on going up the Eiffel Tower?
- Did you get up the Eiffel Tower?
- If not, was the queue too long?
- Are you satisfied with the Eiffel Tower?
Closed questions or yes/no answers can be quickly quantitatively evaluated. But were the right questions asked? Was the customer able to express his opinion and emotional attitude with yes/no?
It is easier if only the following two (open) questions are asked:
- What did you want to achieve with this trip?
- If you did not achieve it, why not?
The agency receives a natural language response from Anita. Such answers contain statements about what the goal of her trip was, whether she reached it and if not, what prevented her from doing so. Such answers are time-consuming to evaluate, because you cannot simply add them up. But unlike before, they provide high-quality feedback.
This is where Sentiment Analysis comes in. It is a procedure that automatically analyzes the evaluation of natural language statements, identifies core statements or opinions and classifies them in terms of emotional attitude (sentiment).
In the following, three business benefits that can be achieved through sentiment analysis are highlighted: higher customer satisfaction through improved products, the recognition of problems in real-time and a differentiation in the market. Afterwards it will be shown what Sentiment Analysis is and how it basically works. Finally, a summary follows.
Business benefits of Sentiment Analysis
New technologies like Artificial Intelligence (AI) and Sentiment Analysis are technically interesting. If we can generate business benefits from them, then it will be exciting! Three such business benefits are outlined here.
Higher customer satisfaction through improved products
With natural language comments the customer describes his experience with a product and expresses his opinion and attitude towards it. By means of Sentiment Analysis we can examine these comments in detail. We identify problem areas, determine key statements and opinions and classify them in terms of emotional attitude. This gives us valuable insights into whether the customer is satisfied and how we should improve the product if necessary.
For example: Anita says: "We didn't want to spend our romantic weekend waiting in line at the Eiffel Tower." A negative attitude towards the queue at the Eiffel Tower is identified. The agency could now expand and thus improve the travel offer with a skip-the-line ticket.
Identify and act in real-time problems
Today, a customer can quickly communicate his dissatisfaction to the whole world via social media. This results in two problem areas:
- The customer is angry and chooses a new provider.
- His public comment harms the brand and reputation and makes it more difficult for the provider to attract new customers.
But this form of feedback is also an opportunity! Sentiment Analysis enables us to automatically interpret comments in real time and initiate measures to increase customer satisfaction.
For example: Anita is in Paris right now and she complains on Facebook: «I imagined Paris to be more romantic.» The agency could pass this information on to the hotel receptionist with the reference to give Anita a secret tip for a romantic trip the following day: «Enjoy a picnic in the Jardin du Luxembourg». Anita would no longer be disappointed in Paris or annoyed by the trip (1) and would ideally also publish a romantic, happy, satisfied post/comment about her picnic (2).
Differentiation in the market: better understanding of competition
Public comments on review portals often also refer to products/offers of the competition. With Sentiment Analysis you can analyse the customer comments of the competition and use them for your own profiling and market positioning.
If we know what customers value most from the competition and what they don't, we can use these insights to differentiate ourselves better or expand our range of products. Combined with the analysis of customer segments, we can deliver more targeted marketing messages, for example.
«Our best competitors reveal our weaknesses. The goal is not to „beat“ our competition but rather to improve ourselves.» Simon Sinek
For example: Anita is back from her Paris trip and writes as a review: «Paris was beautiful. The travel agency gave me a lot of tips for must-see sights. Unfortunately, I missed some of them because I didn't realize that some of them were right next to each other.»
At the same time a customer who booked through another travel agency also wrote a review: «Paris was beautiful. The agency suggested me a perfect route to visit as many must-see sights as possible in the shortest possible time. Wonderful!»
The other client saw many more sights in the same time than Anita. The crucial difference was that he was given a suggested route to visit all the sights that were in close proximity. Anita only received a simple list of sights. Some of them are very far away from each other or the list did not include which sights are next to each other. Anita's agency can learn from the other agency and also suggest such sightseeing routes to make it easier for their clients to see more of Paris in the future.
What is Sentiment Analysis?
A person has had an experience with a product and expresses it in written form, for example as a comment. This text contains: Who says what about whom/what. With Sentiment Analysis, an opinion is extracted from the "what" and the respective emotional attitude (sentiment) is classified.
To explain it with the example of Anitas Reise:
Anita has returned to Paris with her boyfriend from her Seine river trip. The trip was a package deal offered by the travel company Travelito. Anita had positive and negative experiences, which she now shares on TripAdvisor.
She writes as a comment: «The bed at the City Hotel was very uncomfortable. The breakfast was excellent for it, the food was very good and the service left nothing to be desired. We had a wonderful time and will definitely book again via Travelito.»
Anita's comment contains interesting information. This includes topics, opinions, emotions and attitudes.
Sentiment Analysis makes it possible to identify topics such as accommodation, food, service, etc., to extract Anita's opinions about them, to determine her emotion about them and to recognise her attitude (sentiment) accordingly, or to determine whether she is satisfied or dissatisfied with them. In this way we receive qualitative feedback from a customer and can improve our offer if necessary.
If Anita could only have given an asterisk rating for a Net Promoter Score (NPS), she would probably have given a 7 out of 10. Then we would have to guess why it didn't turn out to be a 10 or we would simply not know. NPS is therefore a good KPI (arithmetically), but a bad advisor on customer needs and offer improvements.
How does Sentiment Analysis work?
In the following we will show how sentiment analysis extracts an opinion from a text and, if necessary, classifies an attitude from it. First, certain topics have to be organized. Then the opinion can be extracted and the attitude of a person can be classified. This collection and analysis can reveal valuable insights. These enable the company to improve a product, react to customer needs and position itself more effectively in the market. For the sake of simplicity, essential steps such as data cleansing and preparation are omitted.
Certain thematic fields
In order to analyze and understand large amounts of text, especially comments, we must first organize them accordingly. To do this, we first determine the subject areas. This process is known as Topic Mining.
Topic fields can be named explicitly (e.g. City Hotel = accommodation) or described implicitly (e.g. breakfast = food). Depending on the application, it may make sense to combine topic fields like City Hotel and Food into «Room and Board». The comment by Anita mentioned in the previous example («The bed in the City Hotel was very uncomfortable. The breakfast was excellent, the food was very good and the service left nothing to be desired. We had a wonderful time and will definitely book again via Travelit») but also contains a latent topic like «customer loyalty».
The way in which topic areas are defined is called Topic Modelling. The easiest way to define a topic is by using so-called keywords (e.g. hotel, food). However, there are also more complicated methods such as Latent Semantic Analysis (LSA) or Latent Dirichlet Allocation (LDA) to identify hidden topic areas (e.g. customer loyalty). These are often described implicitly by a set of words, so-called bag of words. LSA or LDA are statistical methods for identifying relevant bag of words.
Now we classify comments and assign them to one or more topics. For this purpose we use traditional document retrieval methods.
The following picture shows subject areas that we extracted from real comments on behalf of an international travel company. The font size of the word shows how often the word was mentioned as a topic in the comments. In this case we have only used negative comments. Here we quickly see, for example, that there are many negative comments on Paris, Normandy and Lyon.
To extract opinions from a text, we need to be able to automatically analyze sentences like «The bed in the City Hotel was very uncomfortable». That means we must be able to decompose sentences as follows.
We use techniques from Natural Language Processing and Understanding, such as Part-of-Speech Tagging (e.g. object, verb, adjective), or Entity Recognition (person, place, organization).
Classify opinions regarding attitude
There are two approaches to determining the attitude of a person:
- Machine Learning: From examples an algorithm learns to understand word combinations and associate them with an attitude.
- Dictionary: Each word is associated with a positive, negative or neutral attitude, which is weighted.
Most cognitive services in the cloud combine both approaches and work very well for generic or domain-unspecific applications.
Analyse and collect insights and react to them!
From comments we have identified subject areas, extracted opinions on them and classified them in terms of attitude. Based on this we can now gain insights.
Product and service improvement
Based on the insights gained, we can define where the greatest dissatisfaction comes from. Thus we know what we should improve in our product or offer.
For example: Customers are constantly dissatisfied with the City Hotel, which has a negative impact on their attitude towards the package tour as a whole. We can now replace the City Hotel with another hotel in the same price range.
Increase customer satisfaction
We can analyze comments in real time, for example on social media. Once we have identified a negative comment, we can assign it to a topic and automatically determine the cause. From this we can derive measures to increase customer satisfaction again.
For example: Anita writes on Facebook that the bed in the hotel is uncomfortable. We could now offer her an additional pillow.
Competition and market analysis
We can now apply the same procedure to customer comments for the competition. From this we can gain valuable insights into what customers value in the competition and what they don't. From this, we can derive measures that differentiate us more clearly or copy the competitors' recipe for success.
For example: Beatrice raves about the croissants that are available for breakfast in another hotel. We could now offer a special latte macchiato instead of the normal breakfast coffee.
A satisfied customer is the basis for business success. To do this, we have to understand what the customer wants to achieve and what prevents him from doing so. His feedback also provides us with competitive advantages and enables us to strengthen our position in the market.
We have outlined three business benefits as examples:
- Better tailoring the product to customer needs in order to increase customer satisfaction.
- Reacting to customer needs in real time. This keeps the customer satisfied, increases the chance of retention and makes it easier to acquire new customers.
- Gain insight into the alternatives of the competition in order to copy or counteract their advantage.
Other examples of business benefits are:
- Customer Churn Prevention, to minimize customer churn
- Smart Assistants, to improve customer touch points
- Social Listening, to gain insights into the current and general perception of products or companies A customer can best express his experiences (feedback) in natural language. However, analyzing such statements is time-consuming.
A customer can best express his experiences (feedback) in natural language. However, analyzing such statements is time-consuming.
Sentiment Analysis is a method to automatically examine natural language statements. Topics are organized, opinions are extracted and classified in terms of emotional attitude (sentiment). This results in insights that enable considerable competitive advantages and increases in efficiency.
Today, the technologies for sentiment analysis are quickly available, especially via the cloud as Cognitive Computing Services or Machine Learning-as-a-Service. With the right expertise, sentiment analysis can be successfully applied and effective business benefits can be derived.