Deriving Business Value from Big Data using Sentiment analysis A ComTechAdvisory Whitepaper
BIG DATA AND ITS POTENTIAL APPLICATIONS
A couple of years ago , ComTech undertook some research in the commodity trading and risk management arena around the value of social media as data . The results were not particularly startling in terms of the interest levels expressed in something so chaotic and voluminous as social media 1 . Yet , when it was explained in the context of a meaningful example , interest levels heightened considerably . Since then , many big data opportunities and application areas within commodity trading and risk management have been identified and they encapsulate a whole raft of new possibilities including , for example
/ Trading Analysis
- Predictive analytics
- Pre-trade decision support analytics including sentiment analysis
/ Market Risk
- On demand risk management
- Predictive indicators
- Exposure simulations
/ Regulation & Compliance
- Trade surveillance
- Fraud management
- Regulation / Compliance audits
Many of these areas are already being actively explored and productized . For example , Bloomberg announced that it is enhancing its enterprise compliance platform to provide next-generation communications surveillance functionality and analytics to meet increasingly stringent regulatory guidelines , prevent market abuse , and deepen visibility into the commercial use of social media . The financial trading dashboard managed by Thomson Reuters is another example , and it uses sentiment analysis data to track news on 20,000 stocks and thousands of commodities . It parses text from multiple sources , looks for keywords , tone , relevance and freshness to provide sentiment analysis for traders to act upon .
Indeed , sentiment analysis has developed rapidly as a technology that applies machine learning and makes a rapid assessment of the sentiments expressed in the various types of unstructured data available today in the form of social media , news and blogs . These sources of information can move the market and are measured quantitatively . Analysts and investors digest financial news and their perceptions can rapidly impact the market and move stock and / or commodity prices .
However , making use of masses of unstructured data of variable quality and reliability isn ’ t an easy undertaking . It requires a high degree of specialism , usually provided by data science , and those with the expertise to deploy the right combination of analytics , machine learning , data mining and statistical skills as well as experience with algorithms and coding in order to explain the significance of data in a way that can easily be understood by others . Part of the problem is in understanding exactly what is meant by the words used by social commentators and others . This means dealing with synonyms , spelling errors , use of different languages such as Latin , polysemy ( where one word can actually have many meanings ) and the sheer volume of data , amongst other issues . If these issues can be resolved , then one is left with the ability to track brand perception and business opinion trends that might have real business value .
The types of data that can be analysed using this approach include news feeds from almost any source , as well as social media content from popular tools such as Twitter . Taking the data in near real-time , stripping out noise and irrelevant content and using Natural Language Processing ( NLP ) and machine learning in an attempt to extract useful meaning can result in information that has immediate and actionable value in the form of , for example , sentiment analysis .
1 ) Making More of Data Using AI , Commodity Technology Advisory White Paper , 2014
© Commodity Technology Advisory LLC , 2015 , All Rights Reserved .