Snowflake ProTalk
Complex Data Pipelines

Data enters into Snowflake in many different modalities and frequencies. This often results in overly complex queries. Often people attempt to solve their data pipeline issues by creating views on views on views. However, this can turn into a chain of complexity that can also result in abstracting Snowflake’s optimization of the table. Snowflake will do it, but it has to break it down into processable chunks, which results in too many micro-partitions being scanned and slowed down queries. This is only one use case among others to be cautious about.

In this session, Rich Hathaway and Arkady Kleyner discuss a number of challenges that occur with complex data pipelines and how to take alternative approaches that don’t bog down the Snowflake optimizer.

You will learn: 

  • The architecture of a curated data pipeline
  • How to limit the view layers
  • Strategies in high churn data vs low churn
  • "Materialized views" the way Snowflake does them
  • Strategies on curation to a broad user community

Episode Highlights:

Rich Hathaway

RICH HATHAWAY

As Enterprise Data Solutions Architect at INTRICITY, Rich has deployed Snowflake in several dozen Fortune 500 accounts and designs and delivers orchestration, modeling, security and strategy to enable elastic cloud data warehousing.

Jared Round BW

JARED HILLAM

Jared acts as our Vice President of Emerging Technologies and is the host of our Snowflake ProTalk webinar series. Listed as one of the “Top Marketing Operations Leaders You Should Know” and is the author and creator of our INTRICITY101 video series.

Arkady circle

ARKADY KLEYNER

As Executive VP and Co-Founder of INTRICITY, Arkady has over 30 years of experience in engineering, architecting, developing and deploying the most intricate enterprise business solutions in America and Europe. 

TO CONTINUE READING
Register Here

appears invalid. We can send you a quick validation email to confirm it's yours