Build Your Code in Snowflake Using Snowpark and Your Favorite Notebook

One of the biggest announcements during Snowflake Summit 2022 was Snowpark for Python reaching public preview. Now data engineers, data scientists, and developers can code with their language of choice, including Python, to execute pipelines, machine learning (ML) workflows, and build data apps faster and more securely in a single platform.

To develop and deploy their code with Snowpark, developers have always had the flexibility to work from their favorite integrated development environment (IDE) or notebook. Why is this useful? Well, let’s go through most common scenarios we see in organizations today:

• We already have a preferred local/hosted IDE or notebook solution. Great news! Snowpark for Python can run anywhere you can run a Python kernel. There is no need to adopt a completely new tool; simply install the Snowpark client API and establish a connection to your Snowflake account. Here is an example of how to get started with Jupyter, but the approach applies to any IDE. 
• We want a {insert Snowpark supported language} development interface in Snowflake. Among the many Python- and data science-related announcements at Summit, we announced the private preview of additional language support in Snowsight Worksheets. This enables developers to build and deploy data pipelines, ML models, and applications directly from Snowflake’s user interface. More details below!
• We are still looking for a development interface for our team. If this is the case, this blog is a great start to help you learn about what options are available. Check out the full list below. 

One option for building apps in Snowpark is to leverage notebooks. Notebooks are a popular choice as they enable rapid experimentation using cells. You can run a variety of notebook solutions with Snowpark. Jupyter Notebooks can be run locally while connected securely to Snowflake to execute data operations. Any machine running containers or Python can build and execute Snowpark apps (at the time of this blog’s publication, Snowpark supports Python 3.8). Instructions for setting up a Jupyter Notebook can be found in the Snowpark developer guide. A similar approach can be used for working with Snowpark in other notebook solutions, including Apache Zeppelin. Open-source notebook solutions are a great choice for data exploration.

Many developers prefer to build apps using code editors and IDEs. These bring capabilities like local debugging, autocomplete, and integration with source control. Snowpark works great in tools like VS Code, IntelliJ, PyCharm, and more. When building with Snowpark and IDEs, be sure to select the right language runtime (Python 3.8 or Java 11). VS Code even works with a Jupyter extension that allows  a notebook experience within the editor, bringing in breakpoints and debugging to the notebook experience, without requiring separate management of the Jupyter container or runtime. Code editors and IDEs are a great choice for rich development and testing experience for building apps.

While you may be familiar with working with worksheets within Snowsight’s web-based experience, did you know you’ll soon be able to write Python and Java code in Snowsight too? Although you can write basic Python and Java code in-line when creating functions and stored procedures right now, Snowflake is rolling out worksheet support for Python, Java, JavaScript, and Scala (all currently in private preview). These worksheets provide autocomplete for the Snowpark session and can be run directly from the browser as a stored procedure. Snowsight is a good option for teams looking for a zero-install editor for writing and running Snowpark-stored procedures.

In addition to the above options, many of our Snowpark Accelerated partners offer either hosted open-source notebooks and/or their own integrated experiences. The best thing about these partners is that their solutions come with out-of-the-box Snowpark API preinstalled and offer secure data connections. These deeply integrated experiences speed up the building and deploying of pipelines, models, and apps.

Here are some of the partners that have step-by-step guides to help you get started with Snowpark:

• Dataiku
• Deepnote
• Hex
• SageMaker Studio Lab

Other partners include DataRobot/Zepl, Domino Data Labs, and H2O.ai. 

Mobilize also recently released the technical preview of BlackDiamond Studio, a set of cloud-powered developer tools for Snowflake. BlackDiamond Studio includes a number of templates for getting started with Snowpark and provides a cloud-powered VS Code experience with frameworks for testing and deploying code to Snowflake.

Teams looking for differentiated and productive experiences for building on Snowpark and Snowflake should look toward partner solutions like the above.

No matter what tools work for your team, Snowflake is the best place for building the next generation of data apps. World class performance, security, and collaboration with data sharing and native apps enable teams to collaboratively build and deploy their solutions at scale. 

If you’re looking to get started building on Snowpark and Snowflake, we recommend one of our quickstarts like the Snowpark for Python quickstart for building a customer churn model using a local Jupyter Notebook or one of the guides from our Snowpark Accelerated partners highlighted above.

Have questions? Get fast answers in the Snowpark Forum of the Data Heroes Community!