Data driven organizations analyze customer behavior for better profitability. Most organizations are already data-centric and have existing systems collecting clickstream data, guest check-ins, orders, payments, surveys, ratings and reviews. One can tap into external sources as well, prominently social media posts and comments to know what the public talks about the organization.

Research tells us that organizations that leverage customer behavioral insights outperform peers by 85 percent in sales growth and more than 25 percent in gross margin [1] – McKinsey

Though there are a variety of customer behavior analyses, we will focus on customer sentiments for this article. Deciphering the customer’s   true feelings towards products and service will help organizations cater to them with better options and improve experience and relationships. The focus is particularly on the analysis of text responses in the form of reviews, survey comments and public posts.

Sentiment Analysis – Setting the context

Customers are heavily influenced by peer reviews and hence this data becomes a potential factor for increase or decrease in footfall. In social media sites, a public review/post by a person about a brand or product attracts attention of their friends / colleagues and their future decisions on the brand and purchase are influenced by the sentiment behind the post. A single statement thus impacts a chain of customers and can either make the link stronger (loyalty) or break it (churn).

Sentiment analysis (sometimes known as opinion mining) refers to the use of natural language processing, text analysis and computational linguistics to systematically identify, extract, quantify, and study affective states and subjective information.

A restaurant review “Food is decent but service is so bad.” contains positive sentiment towards food but strong negative sentiment towards service. Classifying the overall sentiment as negative would neglect the fact that food was actually good.

Aspect based sentiment analysis (ABSA) is a variation of the traditional version where an aspect refers to an attribute or component of a product, food and service in the previous case or things like screen of a cell phone or the picture quality of a camera. ABSA involves multiple sub-tasks such as identifying products from the text, determining the corresponding sentiment/polarity and classifying the identified products in broader aspects.

 

 

Google trends chart

 

Figure 1 The Google Trends chart shows the global interest in Sentiment Analysis is steadily on the rise [2]

Aspire’s Sentiment Analysis Engine is an accelerator built using industry grade NLP and Machine Learning components and helps process volumes of customer feedback and classify them by sentiments and aspects. It can be customized to suit your organization by predefining the aspects you need and training the models using the given data. It also supports updates on feedback and improves with continued experience.

The Sentiment Analysis task can be broadly split into the following categories as explained in the reference picture[Figure-2]:
overview of the sentiment Analysis

Figure 2 Overview of the Sentiment Analysis phases

1.  Preprocessing step

Most often raw text responses from customers are prone to spelling errors, short codes, incomplete sentences and even demographic linguistic idiosyncrasies. The quality of results depends heavily on the quality of data provided. The pre-processing phase tries to normalize the raw text into chunks of processed words so as to minimize the error.

Some preprocessing text mining steps include removal of common stop words, sentence detection and split, lemmatization etc.

2.  Feature extraction

The processed word chunks are fed to the next phase – features [3] are extracted / engineered to build a feature vector [4] that is used in the process of building the model. Vector space models represent words in a continuous vector space where semantically similar words are mapped to nearby points. Additional features can be included, like Parts-Of-Speech tagging – identifying adjectives, adverbs and nouns from the sentence, N-gram generation – combination of words adjacent to one another in a given window etc. to capture the relationship between words in a sentence.

3.   Sentiment Analysis Engine (Watch the Webinar)

The Sentiment Analysis Engine comprises of a set of sub-tasks each characterized at solving a particular portion of the complete task using supervised machine learning models.

  • Product/Entity Recognition

Similar to Named Entity Recognition (NER [5]) the terms undergo a process of locating and classifying named entities into pre-defined product master list, already defined by business. This phase helps identify the different unique products mentioned in the review. In Figure-2, ‘sandwich’ and ’Vanilla ice cream’ are the products identified. Additionally, this phase can also classify the product into broader categories such as bread and dessert provided the metadata information is already available.

  • Polarity Identification

This phase identifies the sentiment associated with the product identified in the preceding step. The customer’s interest level on the particular product is assigned to one of the following polarity labels: positive, negative, or neutral. In the example, ‘awesome’, ‘tasted great’, ‘clean’ attribute to positive while ’rude’ translates to negative polarity.

  • Aspect Category Detection

The identified entities are classified on a broader level and grouped into a set of pre-defined aspects. Examples – sandwich and vanilla ice cream are classified as Food, server as Service, place as Ambience.

Benefits

  • Improved product and service management based on customer feedback. Align your products and features in accordance with their tastes.
  • Minimize customer churn by identifying major pain points and correcting them. One way to get back a lost customer can be to provide a promotion on a product/complement that was previously given positive sentiment.
  • Adopt a better market strategy and launch targeted campaigns. Also measure the success of your campaign by analysis of public sentiments.
  • Generate additional customer-centric features through identification of taste and likes/dislikes, which forms an added input for personalized recommendations.
  • Improve customer service and reduce complaints.

Knowing your customers is important; listening to their concerns and acknowledging their views and opinions really closes the loop and improves your organization’s bottom line.

Sai Gopalakrishnan

Big Data Scientist at Aspire Systems
An insightful person who likes to learn, implement and focus on the process and thereby achieve the target goals. Prefers to work in a challenging environment where Innovation is focused and considered routine. Interested in working on technologies to provide solutions that have a positive impact on the global society, preferably causing a revolution.

Latest posts by Sai Gopalakrishnan (see all)