Data Junkyard

Abdication of Responsibility... This is a term that has been circling in my head over the last few months. What it means is that despite a person's ability to perform their duty... they don’t.
Unfortunately, we’ve been seeing an abdication of responsibility trend inside many organizations over the last few years. The story goes something like this. IT and Business have struggled when it comes to turning data into information. The Business doesn’t involve IT in its inner circle, because they believe IT is too slow to get anything done. And IT thinks the business doesn’t know what it's doing with the data. Amongst this strife, the Big Data pushers introduce the Data Lake as the Shangri-La for all things analytics related. The business salivates, money is spent, the data lake emerges, and in the end, business users are left scratching their head. “Wasn’t this supposed to get easier?” “How come we’re still spending the same amount of time to conform data and produce analytics?” And this is where the abdication of responsibility surfaces... whether IT is aware of it or not, it is their responsibility to make the data lake strategically meaningful. I’m not excusing Business stakeholders. But all too often we see IT teams that know in the back of their heads that the Data Lake doesn’t conform the data… and they wash their hands of it.
A data lake without any conforming logic provides nothing more than a data junkyard; forcing business users to take on the task of logically integrating the data from the pieces and parts of disparate application data. Having a data lake is not an excuse for IT departments to check out of the process of delivering strategically conformed data that is meaningful to the business.
Companies living in these data junkyards have no problem feeding those that willing to rummage through raw data. However, the further up in management you get, the greater the information starvation becomes. This is because no amount of casual data rummaging can get a consistent strategic picture of the organization's operation. There’s just too much data to conform and too many applications to deal with.
So yes, IT and Business will STILL need to engage with each other, despite the presence of a Data Lake. IT still needs to model data so the building blocks for strategic analytics can easily be assembled.
So if someone is pitching a solution to you, remember that there are two fundamental problems you must solve.
The locality of the data
The conformity of the data
Where we often see an abdication of responsibility is on the conformity side. This is because bringing the data physically together is easy. But bringing data logically together is what takes data analysts and architects months of hard work. Particularly when it’s being done across many processes. And guess what, it can’t be done without the business. So rather than try to bridge that gap, often IT and Business teams just skip it. But here’s the thing. You can’t just skip it. Somebody down the line has to ultimately piece together how the data logically conforms. But pushing that down to individuals just means you have a horde of people inconsistently conforming data in their own way. Which leaves your organization with multiple versions of the truth.
This isn’t a problem that is going to be solved by buying the latest shiny BI tool. BI will just put decoration on that bad data. This is a data problem, and solving it requires a joint partnership between IT and Business.
If you would like to see how Intricity approaches this effort, take a look at one of the videos I produced a while back called “Peeling the Onion: Developing a Data Warehouse”.
But I recommend you reach out to Intricity and talk with a specialist. We can help you obtain this conformity of data, starting by bridging the Business and IT stakeholders through facilitated sessions which can drive progress towards scalably turning raw data into information.