I caught up with a former colleague the other day. He’s also in the analytics space so we were sharing notes on the state of the industry. He made a very astute comment about analytics and I like the succinctness of what he said. We were talking about how there are a number of tech startups focusing on the analysis of the data. Hadoop and other NoSQL tools that give companies the ability to look at data, transform data, run machine learning processes on data, etc. That’s not the problem, though. My colleague said, “It’s all about getting the data in and moving it out. It’s the ingress and the egress”.
Keeping the historical trickery of the word ‘egress‘ aside, this is a great statement. I would argue that if you talk to the folks who get their hands real dirty when it comes to data analysis, they would agree that the hardest parts are getting the data into their analysis platform and then moving data out to where it needs to go.
The analysis is the cool part…that’s true. I dare any academic data nerd to go to a LAK conference and not be impressed. When it comes to ingress, it’s kind of the opposite of cool and sexy. Data loads, data cleansing, chron jobs, field mappings, data dictionaries. It’s not necessarily difficult work…it’s just slow and methodical. Focusing on the egress, this step is so important because if you don’t put the data in front of someone who changes what they do because of the information, then all of the analysis goes for naught. The system needs to generate a visualization or connect to a reporting tool or send a data packet to a business application so that someone can make a decision.
This is the ‘workflow’ of data analytics. All phases are important, but it’s hard to focus on just one of them. Without the whole flow, the value to the business is incomplete. I like cool-looking visualizations just as much as the next guy, but I’ve developed a questioning eye when it comes to flashy charts. Can you operationalize the creation of that chart? Can you embed the chart in a user-facing app so someone can take action? Good questions to ask in order to ensure any investment in analytics.