Big Data is hard to do right. There, I said it. I have seen countless implementations of Big Data that are either forever in their infancy or are not producing the results they were supposed to throw off. It’s become so common that there are actually three key questions that are missed in these instances that just end up meaning the whole project was a big waste of time and money.
Of course, it is always a matter of asking the right questions, getting the answers that seem connected to those questions, and making the most of those responses. But there are the right questions to ask of your data, and then there are the right questions to ask of your overall investment in your data platform—in truth, perhaps the most important questions of all are asked of your platform because they set the stage for everything else that follows out of your system.
The most important question is: why?
If you have been around a consultant before or you are a consultant, you will inevitably reach the point in the initial briefing you have with a consultant or a customer where the consultant asks some professional sounding variation of the same question: so what exactly is it that you are trying to do here? Are you trying to arrest a decline in sales? Are you trying to get on top of growing customer complaints and dissatisfaction? Do you want to grow a new market or establish a new channel and need some expectation of interest and how it may perform for you in the future? If you were two years ahead of the current moment, what would this investment, this system have told you that you didn’t already know, and how would you have used that information to get to the desired destination?
Unfortunately far too many Big Data projects exist because the Big Data buzzword is shiny, and new, and all the rage. C level executives, even outside the CMO, get on board the Big Data bandwagon and start authorizing huge investments of time and money to get up on a data platform and in a lot of cases, this entire strategy is based on the flimsy justification of “because everyone else is.” The thing is, data is unique to every organization as are the people behind it. A comprehensive evaluation of exactly what you want to accomplish from having data, as well as a survey and inventory of the expertise and investments your specific organization needs to make, is necessary but far too often overlooked when embarking on a Big Data investment. This is problematic, to say the least.
Of course all of the great questions asked in the world do not matter if there is not a solid, confirmed timeline for answering them. Asking “when” is a key moment, too. Big Data projects are absolutely not immune to the hang ups associated with big corporate projects, including scope creep, approvals, misstated or uncommunicated objectives, lack of leadership and stakeholder buy in, and everything else you have learned to avoid. And even once you have passed that hurdle, establishing concrete timeframes for getting answers to the questions you have set out in advance is the only way to succeed—otherwise you will have a bunch of business analysts and data scientists performing queries ad infinitum, searching for some kernel of wisdom in a sea of data much like SETI searches for alien communications among the dark skies of the universe.
Finally, ask: “Who?” In essence, you have to make sure you have the right people on the bus. As unpopular as it is to say, you need people with business experience as well as mathematical and statistics expertise in your Big Data program or else you are doomed to get a lot of interesting statistical analyses that hold little to no bearing on your real world business and the marketing activities that could go along with it. You need data scientists and business analysts working together to tease the answers to your main questions out of the data as well as being able to understand what might be an actionable insight and what is merely phenomenon with no clear action point.
Fair warning – if you do not master those three key questions, your Big Data project is almost certainly doomed to mediocrity of failure. Learn them and know when to ask them.