Sentiment analysis is the process of extracting information from a body of text. The polarity of the text (positive, negative or neutral), lets one know how people think and feel about a particular topic, issue or individual.
A further level of insight is available through opinion mining, which lets one know what factors are driving the sentiment.
How does sentiment analysis work?
Sentiment analysis relies on the use of artificial intelligence and analytical techniques (sometimes overlaid with human insight in the form of a crowd in order to extract data around emotion and opinion from large volumes of text.
While social media dialogue often forms the basis of sentiment analysis campaigns – people share their thoughts about everything from politics to products, economics and more online, a fact that allows for real-time analysis of shifts in public sentiment around key issues – all text is inherently minable.
Why analyse sentiment? Organisations are interested in sentiment analysis for a variety of reasons. A multinational might want to collect sentiment data for the purposes of market research or to better understand customers’ experiences. Governments and political parties might want to get a read on the public’s feelings towards a proposed policy or towards a candidate standing for election.
Essentially, however, there is a great deal of commonality – the ultimate aim of collecting sentiment or opinion data is to power better decision making.
Sentiment analysis – key concepts
Establishing sentiment polarity – whether the public’s feelings are, broadly, positive or negative toward a given topic – is the most foundational aspect of any sentiment analysis campaign.
To assess polarity, the use of emotive keywords such as like, love, dislike or hate is analysed in order to develop a sense of the climate of feeling. An auto manufacturer, for example, might be interested to know that 60% of online dialogue relating to its new model of SUV is positive, whereas 40% is negative.
While understanding overall polarity can be helpful, further refinement is still required. To this end, measuring sentiment strength is an important aspect of any sentiment analysis programme.
Commonly used for this purpose are rating scales: a score of -5, for example, might be assigned to a comment with a high degree of negativity (Hate it!); +5 to one full of praise (Adore it!). 0, by contrast, would be the score notched up by a neutral statement.
The result is a far greater amount of granularity when it comes to understanding the sentiment landscape.
Consider a hypothetical situation faced by a smartphone manufacturer’s senior decision makers. Simply knowing that the company’s new model of phone is failing to meet users’ expectations provides no guidance. Discovering that the user interface is specifically to blame, however, makes strategic change actionable – more detail enables better decision making.
As such, topic-level breakdowns are among the most crucial aspects of a sentiment analysis campaign. The process involves breaking down chatter around a given subject into further sub-topics and assessing the feelings associated with each.
A financial services company might discover, for example, that while it is succeeding in terms of face-to-face customer service, its online banking portal is the source of frustration, and is driving away potential customers. A powerful sentiment analysis campaign would likely be able to go further still, revealing that banking portal’s document upload process is, for example, the critical issue.
Is artificial intelligence infallible?
Unfortunately, as sophisticated as AI has become, the complexity of human communication often still confounds machines. The result is that sentiment analysis campaigns driven purely by machine processing can often deliver inaccurate results.
Everything from irony to sarcasm to figurative language, slang and local idiosyncrasies of expression can cause interpretive problems. So too can complex or meandering sentences as well as typing and grammatical errors.
One solution is to layer an element of human insight over the analytical work performed by machines. In other words, get real people – a crowd – to refine the work done by AI. Human beings are not only better at dealing with sarcasm and local references and rating sentiment, but they’re better at performing fine-grained, topic-level analysis too – meaning that a greater degree of both detail and accuracy can be achieved.
The growing importance of sentiment analysis
Given the ubiquity of online social platforms, sentiment analysis is coming to replace everything from traditional market research to polling. The technique allows organisations an up-to-the-minute read on how the public feels and thinks – and, crucially through opinion mining why they feel and think as they do.
In today’s world of online engagement major strategic decisions are being made using opinion data. Accurate online data is no longer a nice to have, but rather a critical component of any organisation, but how does one achieve this.