Videos, Snowflake, Blog

Code modernization

Jared Hillam

Jared Hillam

January 27, 2023

Watch our latest Video: Code Modernization

In the last 4 years, the data management space has seen more modernization than the prior 20 years combined. But with all this modernization comes a big question: What do we do with our legacy systems?

Read the Whitepaper: Practical Automation for Code Modernization 

Learn more about our conversion approach with BladeBridge

Talk with a Specialist

Hi I’m Jared Hillam, In the last 4 years, the data management space has seen more modernization than the prior 20 years combined. But with all this modernization comes a big question: What do we do with our legacy systems? 

Some have the luxury to start fresh and rewrite their data applications, but most have to deal with the prospect of modernizing their existing jobs into their newly-selected platforms.

There is no “magic button” for code modernization. Those claiming high first-run conversion percentages tend to shy away from putting that in a legal agreement without big strings attached. Why is that? Why is it that code conversion isn’t a push-button event? Simply put, it’s a complex system.

A complex system is described as a system composed of many components which may interact with each other. 

In fact for code conversion, it’s two complex systems: one legacy system and one future system. The longer a legacy system has been in operation, the more “interaction” it has with other systems. So for an ETL tool, these can be connections, embedded SQL jobs, enterprise schedulers, custom programs, etc. These interactions need to be mimicked in the future system. However, of course, the target has its own limitations and capabilities. So an equally architected understanding has to be established of the future system. Presuming you can capture all that, then you’re ready to convert your code.

So if converting code isn’t a push-button event, why not just convert code by hand?

There are 2 reasons converting code by hand can be disastrous. The first is something we call Code Sharding. See when you have an army of developers converting code, each person is going to convert the same objects differently. And even the same person might change the way they convert something from one day to the next. This means that the patterns in your legacy environment will cease to exist in the future environment. Those patterns are super critical for tuning and automating your code. Once they’re gone, there is no way to treat your code as a body anymore.

The second reason has to do with the project itself. As all this code sharding progresses towards the end of the project, there is a growing fragility to technical mistakes. During conversions such technical gaps and mishaps are common, because remember… these are complex systems. So what do you do if you discover that say… your code is missing some tags that an external system needs… or maybe there was a datatype setting that was executed incorrectly, and let's say you’re almost finished converting thousands of jobs by hand? If that’s the case, there is nothing but hard labor to globally edit all those jobs. And this… is why automation is critical. Not the magic button, but a facility to automate global patterns and that is the value of a code conversion platform.

So presuming you want to move forward with a code conversion project, where do you start?

One of the early things that you should get out of the way is an automated analysis of the legacy systems. This analysis should be able to inventory and categorize all the objects in your ETL tools and databases so that you can appropriately size the level of effort.

Intricity leverages a tool called BladeBridge Analyzer to make this happen. Analyzer is a portable command line tool that can be executed locally on a laptop and only touches pre-exported metadata and code, so it doesn’t touch any runtime systems.

If you’d like to have Intricity scan and size your conversion, I recommend clicking on the link in the video description. Now there is a lot more to this topic, so I’ve written a fairly extensive whitepaper about code conversion, which you’ll also find in the video description. As always, if you would like to talk with a specialist about your conversion project we’ll include a link for that too.

Related Post

Snowflake Data Breach... Now What?

Snowflake's data breach affected 165 customers through stolen credentials. Discover the security measures that could have prevented it and how to protect your data in the future.

Read More

Using AI for Code & Metadata Conversions on Data Systems

LLM-based code conversions can have challenges and successes. Explore real-world insights and best practices for navigating these projects.

Read More

Snowflake ProTalk: Snowflake Summit 2024 Summary

Explore newly announced features from Snowflake Summit and their impact on customers. Stay ahead with exclusive insights for future ProTalk sessions!

Watch Now