Secrets of Spark to Snowflake Migration Success: Customer Stories and Outcomes

Today’s business landscape is increasingly competitive — and the right data platform can be the difference between teams that feel empowered or impaired. I love talking with leaders across industries and organizations to hear about what’s top of mind for them as they evaluate various data platforms. 

In these conversations, there are a number of questions that I hear time and time again: Will my data platform be scalable and reliable enough? Will it be easy to use for my entire team? What will costs look like? How will my data stay secure and governed? 

A critical part of this decision is determining which foundational technology to build infrastructure on. Managed Apache Spark environments — such as Databricks, Amazon EMR, and certain Cloudera deployments — can present teams with a plethora of pain points, which may include complexity, unpredictable costs, security concerns, or performance issues. 

I see these factors as key reasons why organizations of all sizes and industries make the move to Snowflake. Helping organizations through a migration — and seeing the tremendous outcomes they achieve as a result — are some of the most rewarding parts of my job. And in the new book “Secrets of Apache Spark to Snowflake Migration Success,” we’re spotlighting some of these exciting stories from customers as varied as AMN Healthcare, IGS Energy, Intercontinental Exchange and the New York Stock Exchange. 

Here are just a few examples of leading organizations that are migrating from managed Spark environments to Snowflake to save millions of dollars, improve performance and get products to market faster to delight their customers sooner. 

Travelpass delivers more curated traveler experiences — while saving 65% in costs

Built on the idea of discovering common ground through exploration, Travelpass connects travelers with the best hotels and travel experiences to meet their needs. Data is the lifeblood of Travelpass’ business — yet the Travelpass data teams were spending a lot of time focusing on how to build, rather than what to build.

By moving from Databricks to Snowflake, Travelpass now empowers more people to work with data to deliver greater efficiency, more informed decision-making and a more tailored experience for travelers across the globe. Thanks to Snowflake’s ease of use and approachability, even non-data engineers at Travelpass now contribute to Snowflake data in a meaningful, quick way.

Benefits include: 

• 65% cost savings by switching from its previous platform, Databricks, to Snowflake

• 350% improved efficiency delivering data to business units, thanks to Snowflake Dynamic Tables

• Greater reliability and ecosystem stability by eliminating labor-intensive debugging of the previous system

Chicago Trading Company achieves 54% cost savings and meets daily SLA for the first time

Recognized as a leading derivatives trading firm, Chicago Trading Company (CTC) provides liquidity to markets around the world, helping drive efficient, stable and healthy markets by participating on both buy and sell sides. CTC’s research platform collects information from thousands of sources, including feeds from every exchange it trades on, historical trading prices and third-party data. But CTC was paying $800,000 a year just to move data from Snowflake to managed Spark for processing and back again. 

To overcome these hurdles, CTC moved its processing off of managed Spark and onto Snowflake, where it had already built its data foundation. With Snowflake and Snowpark, CTC has gained greater visibility and control over its costs while also drastically reducing data processing job failures — an invaluable improvement, given that the jobs are always running against a clock. Thanks to the reduction in costs, CTC now maximizes data to further innovate and increase its market-making capabilities.

Benefits include: 

• 54% cost savings — amounting to millions of dollars annually — by moving from managed Spark to Snowflake

• $800,000 saved annually by eliminating data movement out of Snowflake and back

• First time meeting the daily service-level agreement of having data available at least one hour prior to market open — a milestone it hadn't been in a position to track before Snowflake 

Swire achieves millions in cost savings and accelerates model deployment by weeks 

Swire Coca-Cola, USA is the local bottler for Coca-Cola and other beverage brands in 13 states across the American West, delivering refreshments to 31 million consumers every day. Swire had Snowflake as its single source of truth and a separately managed Spark platform for its AI/ML needs. But managing complex infrastructure diverted data teams from model building, causing delays. Spark clusters needed manual maintenance to avoid waste and took 10-15 minutes to spin up, while the managed Spark platform outside Snowflake raised data governance concerns, impacting data integrity and security.

Snowflake emerged as the ideal one-stop shop for Swire’s AI/ML needs, offering a singular platform that significantly reduced complexity, enhanced ease of use and provided a robust framework for improved data governance. With these improvements, Swire has optimized its planned logistics routes to significantly reduce costs related to fuel, driver expenses and overall cost to serve. The impact on time to market was equally remarkable, with Swire able to develop models on Snowflake notably faster.

Benefits include: 

• Millions of dollars in cost savings by optimizing planned logistics routes

• Faster time to market, resulting in weeks of time savings, by deploying critical AI/ML models faster 

• Lower total cost of ownership from streamlined, automated data management

More migration successes 

These stories are just the beginning of how organizations are moving to Snowflake to drive competitive advantage. 

Download the book “Secrets of Apache Spark to Snowflake Migration Success” to see the five key reasons companies are moving to Snowflake — and how these migrations are helping businesses slash costs, reduce complexity and improve reliability for their daily operations.