Videos, Data Warehousing, Blog

Real Data Warehouse?

Jared Hillam

Jared Hillam

April 11, 2014

 

https://www.intricity.com/data-warehousing/peeling-the-onion-developing-a-data-warehouse/

The above link is to the video referenced about how to build a Data Warehouse

Text from the Video:

Sometimes organizations think they have a Data Warehouse, when what they really have is just a Data Warehouse by name without its qualities.  So in this video we’re going to answer the question, “How do you know if you have a Data Warehouse?”  And we’re going to answer this through a couple of short stories.

The first story is about a report developer that is feed up with having to draw data from multiple locations to get information for the business.  To fix this, the organization creates a database where all the tables needed for reporting can be found in one place, and this new database gets updated regularly with the latest data through scheduled refreshes.  And somewhere down the line this dumping ground of report tables gets officially dubbed as the Data Warehouse.

The second story is about an astute DBA that is very good at creating views.  You can think of a view as a data set that is processed by the database on the fly from multiple tables.  These views are what supply the data to the reports. And all the logic for those views is sitting in code which really only the DBA understands. Somewhere down the line this bunch of tables calculated on the fly gets deemed as the Data Warehouse.

Neither of these examples represent a data warehouse.  But in both of these examples we see important needs being met.  In the first example we see the importance of data residing in a single location to simplify access to the tables.  In the second example we see the importance of simplifying query logic so the report writer can focus on building content.  At Intricity we see all sorts of combinations of these two stories playing out, but these fall short in delivering the benefits of leveraging a true Data Warehouse.

In a real Data Warehouse we take the best qualities of these two scenarios, by physically centralizing the data and simplifying the logic for its consumption. What might surprise you about the process of building a data warehouse is that the key to its success is really the Business Stakeholders and not so much the tools or technicalities. Building a Data Warehouse is much like peeling an onion, if you don’t start with the first layer you’re likely going to miss the big picture. The temptation to dive right into the detailed data and piece together what the business needs will ultimately cost you time and rework. By clicking on the link in the video description, we can take you through the 8 layers of the Data Warehousing onion that need to be peeled for a successful deployment.

Once these layers have been conquered the organization then has a pattern or rather a program for acquiring raw data and turning it into a shared decision making asset.

So what are some of the signs you really don’t have a data warehouse?

1.        Acquiring your data involves scouring multiple databases

2.        Acquiring your data might be very reliant on database views

3.        The data warehouse was created as an IT project and did not require Business Stakeholder attention

4.        You’re wanting to replace your Business Intelligence tool because you’re not getting the data you need

5.        All the logic for gathering data sits in the head of your report developer

6.        Business Analysts are independently creating their own analytical environments to get information out of the data

7.        Your management meetings are more about who’s data is right then what the actual performance gaps are

8.        There is a lot of query logic required in your reports to get the data to be right

9.        Analysts have to use spreadsheets with dozens of tabs to get the data to look right

10.     Managements reports or dashboards appear simple but on the back end they require a large team of resources doing repetitive data integration work

The list of symptoms goes far beyond the scope of this video, but hopefully you can relate to some of the challenges I’ve mentioned. Intricity has been implementing Data Warehouses for all sorts of organizations. I recommend you visit Intricity’s website and talk with one of our specialist about some of the symptoms you’re experiencing. We can help you take on both the business and technical challenges of these symptoms and help you build a best of breed data warehousing solution.

 

Related Post

Snowflake Data Breach... Now What?

Snowflake's data breach affected 165 customers through stolen credentials. Discover the security measures that could have prevented it and how to protect your data in the future.

Read More

Using AI for Code & Metadata Conversions on Data Systems

LLM-based code conversions can have challenges and successes. Explore real-world insights and best practices for navigating these projects.

Read More

Ness Digital Engineering Acquires Intricity

Ness Digital Engineering Acquires Intricity - a New York based company specializing in data strategy, governance, modernization, and monetization

Learn More