Before we succeed at Big Data, let’s get the small data right!
Almost four years to the date, Donald J Trump succeeded in an unexpected election victory to become the 45th president of the United States. In time allegations surfaced that the Trump campaign had used the services of a data analysis firm, Cambridge Analytica, who claimed to have access to some 5 000 data points on some 50 million Facebook users. This supposedly allowed the Trump campaign to create psychographic profiles and targeted advertising to swing the vote. A scandal of such proportion created many ripples around privacy but also created a realisation as to the realms of possibility of data analytics.
Big Data, 4IR, machine learning and AI are all terms which have become significant talking points of late. Many organisations around the world are investing heavily in data analytics. It has simultaneously become a career of choice for many and a source of fear of irrelevance for those who don’t have some requisite skill in the area. Certainly, what we are seeing in the infancy of big data is truly impressive.
There is a ‘but’ though. I have been fortunate to have led and been part of teams which make use of data analysis and analytics. Although I have in no way seen the length and breadth of big data capabilities, some things have become evident.
First and foremost is the fact that just because a conclusion is mathematically supported does not mean it is the right answer. The root cause here is often the lack of understanding of the actual problem that we are trying to solve. The eagerness to dive into data, the desire to unwrap shiny new tools, and the rush to conclude all too often undermine the time one should spend actually understanding the heart of the issue we are trying to solve. My honest belief is that there is no shortage of answers, what we are short of is the right questions.
Secondly, many poor conclusions are reached from not questioning ‘the obvious’ and sense-checking ‘the given’. More often than not, because the machines have started to do the heavy lifting for us, and perhaps because we just assume they are smarter than us, we take the basics for granted – ‘does the starting point make sense’, ‘does the input reconcile to everything else I have seen’ and ‘does the outcome have broader consequences’ are questions that we should never feel embarrassed to ask.
Lastly, and perhaps the most important of all, is that the most valuable data analysis is that which is supported by the broader perspectives of those who have experience of the problem, be it a call centre agent who listens daily to customer frustrations or an experienced CEO who sees the organisation in its entirety. One must practise humility in asking for opinions and exercise patience in understanding the perspectives of those around the table. Respect the relationships between the diverse teams, their contributions and the invisible thread of human interactions that turn answers into solutions.
In summary, it is not the Big Data that gets us the right conclusion, but all the small data in between.