Intricity has extensive experience migrating legacy Data Lakes and Data Warehouses. This is especially the case with migrations to Snowflake.
Both Data Lakes and Data Warehouses have certain nuances, along with their respective legacy platforms. The sheer breadth and depth of the Intricity's experience with these varied platforms has made us a unique player for data migrations. Perhaps more importantly, Intricity has developed assets to address some of the more complex Data Migration scenarios such as MPP database replacements.
The interpretation of the term "Data Lake" often varies from company to company, but if the Data Lake is serving its textbook purpose, it contains all the organizations data in one place. This solves the locality problem of having data dispersed in multiple repositories. Additionally, it means that replicating a Data Lake is relatively straight forward, as there usually isn't a lot of business logic which the data is conforming to. Thus the key events in migrating a data lake have to do with lifting and shifting the footprint, re-pointing source data by setting up streaming/replication pipelines and resetting the target applications connections. The difficulty lies in the size of the Data Lake, and the breadth of connections and targets.
The data warehouse is the nexus point of data turning into information which the business can act on. Much of the heavy lifting on how data elements tie together manifest themselves in a data warehouse. Therefore moving a data warehouse to a new platform requires a replication of the business logic which conforms the data, and this must often be expressed in the new database platform's code. Many "Big 4" consulting companies address this type of project by "throwing bodies at the problem." That is NOT the correct approach to doing a migration, particularly as the complexity levels increase. This is super obvious when it comes to migrating an MPP based Data Warehouse. These generally are vastly more complex and are not only leveraged for query speed, but also speeding up the data conformity pipeline, in an ELT or Push Down Optimization feature.
MPP Data Warehouses
Intricity has extensive experience with Netezza/Teradata. This experience has been an invaluable asset in helping customers migrate their current Netezza/Teradata footprint to Snowflake. Intricity has created a migration factory for making a Netezza/Teradata migration painless. There is no such thing as a "push button" migration, as each migration will have nuance in their development patterns. To address these nuances, Intricity leverages a pattern based micro-accelerator approach which is used to cycle through the Databases, Tables, Views, Stored Procedures, Functions, ETL Jobs, Orchestration Jobs, and Shell Scripts. This produces a pass/fail migration of objects, and the failed objects undergo yet another round of pattern identification and cycling for another pass/fail. This continues as Intricity moves through the migration, and with each pattern capture the migration speed accelerates until all the objects are successfully completed.
SEE A MPP MIGRATION PROJECT
If you would like to review some examples of where Intricity has conducted migrations, we recommend you reach out to Intricity and talk with a specialist by filling out the form below.
To read more about Intricity's MPP migration process click on your platform: