Snowpark Is Now Generally Available

Today we’re excited to announce the general availability of the Snowpark API for Scala and Java UDFs on AWS. Snowpark is the developer framework for Snowflake, bringing deep, language-integrated data programmability to users in the languages they love. 

At its core, Snowpark is all about extensibility. It was designed to let data engineers, data scientists, and other developers work with data more efficiently and effectively in their programming languages and tools of choice, including Scala, Python (in private preview), and Java, using familiar programming constructs such as DataFrames. And it was built to move that work right to where the data lives: in Snowflake’s scalable, secure compute engine.

Our goal was to eliminate inefficient data pipelines and optimize processes and tasks that companies may be using just to get everyone on the same (data) page. Ultimately, Snowpark enables teams with different skill sets to collaborate and work on the same data, process data faster and more easily, and make data security and governance a top priority. It also opens up data access and manipulation to the broader developer, data engineer, and data scientist communities.

Since we introduced Snowpark in preview in June 2021, many Snowflake customers and partners have been using it to build solutions for a wide array of vital tasks. For example, some customers are leveraging Snowpark for PII detection; others are converting data formats while loading, or performing graph exploration. Many companies are using Snowpark to bring machine learning models they’ve built into Snowflake so that they can be operationalized more easily. And our partners have been jumping on this as well through the. Snowpark Accelerated program.

This GA is just a beginning. Since Snowflake first began inviting early adopters to work in Snowpark during our public preview, we’ve also been expanding our support. We’ve added Java UDTFs and support for both Google and Microsoft’s clouds to our public preview. And we have a raft of features in limited preview that we’ll be opening up in the near future, including stored procedures, logging support, and support for unstructured file processing. And yes, we also have Python in the pipe (currently in private preview).

“IQVIA powers the healthcare and life sciences industries by connecting data and services to drive intelligent decision-making. Snowflake’s scalability and security are a great fit for our requirements, and Snowpark’s DataFrame API and support for custom Java code are powerful for data transformation,” said Mohit Sauhta, IT Architect Director at IQVIA. “Snowpark will allow us to modernize and consolidate our data engineering pipelines, simplify our architecture with an easy transition from Spark, and allow our data engineering team to continue working with their preferred development interface, a DataFrame API with lazy evaluation, despite changing underlying platforms.” 

“This is a win-win, reducing time to insight for our customers while making processes easier and cheaper to manage for us,” said Sauhta. “We’re excited to see these features move into GA, and look forward to unlocking more use cases with Snowpark in the future.” 

Welcome to Snowpark, now officially open on the Data Cloud.