Case Study: Migration of Netezza and Informatica to Snowflake

About Client

The Client operates behind the scenes for over 50,000 customers by managing critical fleet, transportation and supply chain functions. Many of which make the products that consumers use every day. The Client is known for its fleet of commercial rental trucks, the expansive infrastructure of maintenance facilities and warehouses, and some of the most talented people in the industry where they operate in North America.

Challenge

The Client was met with an upcoming renewal deadline for its Netezza data warehouse appliance. The cost to renew would be in the millions and Snowflake’s capabilities offered far better scalability and dev ops mobility. The Client decided to migrate from the on-prem legacy Netezza to the cloud into Snowflake.

Navigating Constraints

In order to successfully migrate to Snowflake from Netezza, there were several constraints the Client and Intricity teams needed to address:

Time Crunch

The Client planned this move in advance, but the renewal created a clear line of demarcation for the project team to achieve. The consequence of lapsing the delivery date had real-world cost implications.

Multiple Technologies

Netezza was the end state of the legacy data pipeline; the Client also used Informatica to load its data into Netezza. This Informatica instance needed to be repointed to the new Snowflake instance. Rather than attempt to inject yet another code conversion step, the Client decided to keep PowerCenter, but repoint all the PowerCenter objects related to Netezza towards Snowflake. This included all the embedded Netezza SQL jobs and connection parameters to PowerExchange for Snowflake. Informatica had grandfathered the Client into using the PowerExchange framework for Snowflake.

Performance

The Client needed the performance to be as fast or faster than the legacy environment. While Snowflake was known for speed and efficiency, this had never been tried with the Client’s actual jobs.

Testing and Validation

The Client team needed to set up a significant testing and validation effort to ensure the converted Netezza and Informatica jobs were converted correctly.

Win 1: On-Time Conversion

Intricity worked with the Client to convert the jobs using BladeBridge Converter. The project was delivered on time and on budget. The Client was able to end the Netezza renewal to avoid the million-dollar fees and the project was highlighted as an IT success story with the Board of Directors.

Win 2: Co-Conversion

Unit Tests

The Intricity team iterated through the BladeBridge configuration files to generate the converted code and the Client ran the code against the future state environment. Any unit test error patterns that were in high volume were sent back to the Intricity team. The Intricity team updated the BladeBridge configuration files to address the error patterns, then re-submitted the converted jobs to the Client’s testing team. 

Low volume error patterns were tweaked by the Intricity team directly in SnowSQL or Informatica. This ensured that the Intricity team members conditioning the BladeBridge configuration files focused on high-output pattern conversions.

Data Tests

Data testing followed a similar pattern to Unit Testing. The Client ran a parallel query between the legacy database and the target database to ensure the results were the same. Where the data came up as different, the converted code would be assessed for pattern volume. High-volume patterns were pushed to the Intricity team that updated BladeBridge configuration files. If the patterns were small, the logic would be manipulated manually.

Win 3: 

The Client set up a highly responsive syntax and data testing team that rapidly generated feedback. This feedback loop enabled the joint teams to quickly identify changes that needed to be made to target logic.

Win 4: 

The Client today has a fully functional Snowflake environment with native Snowflake code in both the database and ETL layer.

Learn more about: 

• Netezza to Snowflake conversion
• BladeBridge Converter
• Improving your Snowflake environment