6
Dirty Data, Dirtier Results
How CRMs and Big Data Analytics Get Corrupted
and What to Do about It
One mantra of the nutrition industry is “you are what you eat.” That statement is just as true of
statistical analytics and market research as it is of people. By that standard, many companies
and their databases aren’t on healthy diets.
Those in the research community who work on customer satisfaction or Voice-of-the-Customer
(VoC) studies see this on a weekly basis. We get a file from the client with their customers and
customer contacts and discover that some proportion of the records is simply wrong.
Most studies these days are conducted online, and result in massive numbers of unanswered email
invitations. Since many mail servers will not return error messages, the lack of any response is
uninformative.
By-Victor Crain
What are the opportunities & threats for the
survey research industry?
10
There are two urgent topics that need to be addressed. The first is data security with regard to
the EU. The death of the Safe Harbor framework, which allowed data to be imported from the EU
onto USA based servers, is a very significant event. The EU is becoming very strict about data
privacy and this is something many USA based survey companies have to address immediately.
There are no easy solutions, but unless the issue of security and the EU is tackled head on many
current multinational survey companies may find they are restricted to USA data collection only.
The second one is, as ever, mobile. The industry has talked for decades about making surveys
easier to complete, making them shorter and so forth. The truth is that very little, with a few
notable exceptions, has been done about it. In 1998 I wrote a paper for an ESOMAR conference
which pointed out that web respondents are more likely to drop out of web surveys on long
grid questions. It is now 2015 and I STILL SEE discussions about grids and how they need to be
By-Andrew Jeavons
From Big Data to the Big Picture…
17
How can we deliver actionable advice from data?
Very good question and a great place to start. Our clients at Skopos are mostly large
brands and organisations, who are daily bombarded with more data and data sources than
ever
nowadays, what with the emergence of internal customer (behaviour) analytics,
website analytics, social media streams, plus the more traditional campaign and market
research trackers, projects, etc. Careful selection, control and the triangulation is largely our
philosophy for handling this, although being adaptable in this changeable world also helps we
find. With the new large (big) datasets from internal sources where this is disparate and too
large to data-mine in one go, we recommend carefully and ideally scientifically selecting (or
sampling) the data so it is relevant and representative, and in smaller volumes – so it is more
manageable (whilst still being representative and trustworthy).
By-Darren Mark Noyce
Do we really understand the customers?
The cognitive processes that consumers use are fundamentally unchanged – most purchase
decisions are still based on fast thinking that is more intuitive and less rational than has
been believed in the past. The average time it takes for a consumer to place an item in their
basket is 4.6 seconds. However, what has changed are the tools that consumers have at their
disposal to make purchase decisions and the diversity of shopping experiences.
This goes way beyond online shopping through websites. It includes new behaviors ranging
from showrooming, how apps are used to plan shopping trips, or the use of apps to access
information at the point of purchase. Further, the shopping experiences that consumers can
choose from are much more diverse than in the past. For example, instead of shopping at one
big store for all grocery needs, consumers are shopping at different stores and channels for
different items.
By-Dave Lundahl
23