Videos, Data Warehousing, Business Intelligence, Snowflake, Blog

Data Maturity: $100M-$200M in Revenue

Jared Hillam

Jared Hillam

February 6, 2019

 

In our previous video we talked about the Discovery Stage which pointed out the data challenges of smaller organizations. In this video, we’re going to cover the Inclusive stage which usually represents companies that are between 100-200 million dollars in size. These organizations are seeking to prioritize inclusiveness in both audience and data sources, and they usually take one of two directions to start.

 

In the first, and most likely direction, the organization seeks to standardize onto one Business Intelligence platform. This enables the institution to agree on format, which can be a critical juncture on getting teams to work together. This also produces cohesiveness between layers of management because analytics can finally be shared.

The second likely direction is the development of a data lake. A data lake helps the organization to point all their data assets to a single location. So instead of maintaining disparate connections to each data silo, the organization can work from one source of data.

Now I want to take a pause to point out that I’ve outlined the value which the organization actually receives from these solutions, and NOT the value the stakeholders predicted to get. And this is where a critical juncture occurs. If the organization doesn’t rapidly transition to the Truthful Stage they will be under the impression that their Business Intelligence tool was bad, or that their Data Lake deployment was a failure. Let me explain...

In the early Inclusive Stage the organization is still a bit naive about fixing the complexities of their own data. The impression that Business Intelligence magically produces information from data is pervasive with business stakeholders. This asymmetry of information is something analytics vendors are happy to prop up in order to close a deal. However, it leads to frustration later on when the collective output doesn’t behave like the pristine demo. I have a whole video dedicated on this topic titled “Beware the BI Demo”.

The same naiveness is true with a data lake. The perception that bringing all the data together into one source which everybody can draw from, sounds like it would solve a data integration problem. However, organizations that invest in data lakes find themselves still endlessly rummaging through data. Additionally decisions made using data sourced from data lakes often creates multiple versions of the truth. This is because nothing has been done to conform the data into information which the entire organization can draw from. I also have a video dedicated to this topic titled “Data Junkyard”.

Not everybody in the inclusive stage experiences such frustrations. The loosely organized individual analytics teams often prefer the lack of data structure and BI standards. This is particularly true with data scientists and analysts that seek to do data experiments. However, even these parties spend much of their time conforming data over and over without consistency between users, and can benefit greatly from some conformity.

While the Inclusive Stage is a frustrating inflection point, nearly every organization has to experience it. Because budgets to actually solve these problems need to be much more comprehensive, and thus tied to a valid justification. And this is much more realistic to do in an organization that has lived through such pains.

I’ve included a white paper in the video description titled “BI...Still No Pixie Dust” which delves into some of the challenges pointed out in this video in more detail.

In the next video in this series we’re going to talk about companies that are between 200 Million -1 Billion dollars in size. If you have questions in the meantime you can reach out to us and talk with an Intricity Specialist.

Related Post

Snowflake Data Breach... Now What?

Snowflake's data breach affected 165 customers through stolen credentials. Discover the security measures that could have prevented it and how to protect your data in the future.

Read More

Using AI for Code & Metadata Conversions on Data Systems

LLM-based code conversions can have challenges and successes. Explore real-world insights and best practices for navigating these projects.

Read More

Medallion Architecture, From the Late 90's?

Much like AI is all-the-buzz today, the early 2000’s buzz tech was the Data Warehouse. Like AI today, everybody said they were doing it, but few were actually aware of what they were doing...

Learn More