Why are analytics dying?

Hi, I’m Jared Hillam.

Back in Sep 2017 I released a video titled Why is Hadoop Dying? I got a ton of hate mail about it, but that video’s prediction was spot on. 

At the time we were seeing early indications this was the case, as decision-makers were pausing  their Hadoop decisions and looking to cloud native options.

But in 2023 I’m asking a new question, why are analytics dying? Now this might sound preposterous, but hear me out…

Think for a moment what analytics are. They represent a transformation from data to navigable information. They use data to tell a visual story that is easier to consume. And multiple companies came up with creative ways to organize that process… 

For example, Business Objects came up with a business view of data elements which users could manipulate to assemble their own reports. On the back end, when users assembled these business friendly data elements, it would dynamically write a SQL query for the underlying database. Then you had the advent of Tableau which infiltrated the business community by showing visualizations that were far more compelling than their governed BI platforms. They appeared to magically generate visualizations from excel spreadsheets, which made for killer BI demos. There are many many more honorable mentions in the BI space which all have made huge impacts. But just like Hadoop, BI and Analytics will be dying a slow death, and you can blame AI for it.

About 10 years ago Business Objects at their annual conference demonstrated impressive gimmick. They assembled a sentence which turned into a database query. Business Objects report developers were questioning whether their jobs might be at risk after the conference. But the demo never really saw the light of day. Then Thoughtspot made a splash with the same feature, with a lot more umph behind it. And they’ve have had a successful run with their tooling in creating a chat based experience for querying data.

As we all know, there is the world before ChatGPT and the world after ChatGPT. The collective awareness of Large Language Models has completely shifted… everything. Companies that were darlings in large data queries now only get one question. “What are you doing with AI?” And they’re getting the message. The outcome however, is not favorable for niche BI vendors. 

In times past, there was an unspoken understanding that the analytics side wouldn’t encroach on the database side and vice versa. That understanding didn’t always pan out, but with every customer asking “what about AI” what exactly should we expect from database vendors? More speed is a no brainer, but how do you show a prospective customer that wants to see AI in action? There’s one glaringly obvious way to go, and that's AI based answers to natural language questions. And… that’s what we’re seeing. Answers are being drummed up from LLM interactions with charts, graphs, tables, and other elements, and it's coming FROM the database vendors themselves. So where does this leave BI and Analytic companies?

Well, at least in the short term I don’t see customers dumping their BI and Analytic licenses. But just like in 2017, there’s a pause. The questions are getting asked, “Do we need this?” “Can an LLM do this for us?” And this is just the baby step. The real dystopian question is, Why are we even producing these analytics to begin with? If a decision maker is on the other end, why can’t the AI just execute the decision without the noise of the visualizations to begin with?

Just like McDonald’s replacing cashiers with touchscreens and people hardly caring to notice, organizations may be able to replace large swaths of micro-decision makers with AI. This already is taking place in automated sorting facilities which used to be powered by operational reporting platforms like Crystal Reports that told people what to do next. 

The one big expense will be the queries themselves. Making that data layer as efficient as possible will be a key in making the ROI work for such automation. For at least that function, there still is strategic work to do, to ensure the table structures are as wickedly efficient as possible. 

Now this prediction is very early, just as my Hadoop prediction was, so we’ll see how the future will treat me. 

If you’re on that journey right now, I recommend talking with an Intricity specialist on your database back end. We can help productionize your AI journey by ensuring that the factory that fuels the data to AI isn’t dragging its feet.

You’ll also find a whitepaper, which I actually did write, titled “Do we need BI anymore?” which you’ll find in the description of this video.