Data Governance, Videos, Article, Blog

Data Deduplication factory

Jared Hillam

Jared Hillam

August 24, 2017


We recently had a discussion with a financial services company which had several concerns about moving to the cloud. But surprisingly, these concerns did not come from the technical staff, but rather their business leaders. "I just don't trust it" was the statement from one of their executives. So, this article is for all those

business executives that are being pitched a move to the cloud, but feel uneasy about it.

In the magic land of Simpleville your entire company lives under 1 enterprise application system. All the customer records live in one place and everybody around you uses the same systems you do, and you get to ride a unicorn past all the traffic on your way home. At least that’s what the ERP vendors would have us believe.

And then there’s the real world. Customer records live in dozens of application systems. Your ERP is just 1 of 20 mission critical systems that operate your business which contain customer data, despite all your ERP standardization efforts. A single customer record could live in all of those systems at the same time, each containing a slight variation. Imagine needing to send a mailer to all your customers. How would you do it? Would you mail all of them, knowing that you’re going to end up wasting millions of dollars in repeat mailers? No doubt there are ivory tower efforts to solve this problem 3-10 years from now… but how do you address this issue now?


Intricity is here to propose a simple cloud solution to this duplicate record problem, with something called Data Deduplication Factory (DDF). When you first connect the DDF, its guided machine learning capabilities will present its best guess of duplicate records. As you provide the machine learning feedback on matches and misses, it will learn from each training loop, and ultimately perform autonomously. Once you have the trained results, DDF allows you to integrate the results via a RESTful web service which can be integrated into your enterprise applications and use cases. Additionally, you can pull the results in multiple forms like .csv and JSON.

Intricity prices the DDF by data volume. If you would like to get an estimate and a demo, click here. 

Related Post

When data gets creepy

Ever wonder how companies manage your data behind the scenes? Uncover how companies collect, enrich, and utilize your data, and the implications for privacy and marketing strategies.

Watch Now

Why are analytics dying?

As AI gains prominence in answering natural language queries directly from databases, the role of traditional BI and analytics for decision-making could become obsolete.

Watch Now

Widgets vs Code

Explore the advantages and disadvantages of using graphical widgets in traditional ETL tools versus coding in ELT. Shed light on aspects such as cost efficiency, performance tuning, and the...

Watch Now