Data Engineering School

Welcome to the AI Data Cloud Academy for Data Engineering. This academy is designed for data engineers at all levels to learn the latest trends and skill sets in the field.

This program features curated content from Snowflake Summit 2024 for architects and data engineers, including top product sessions, customer stories on building data pipelines with Snowflake, and best practices for Data Engineering across three tracks:

Guide to Build Continuous & Streaming Pipelines: Capabilities from streaming ingestion to declarative transformations with incremental processing
Guide for Python Developers: Perform data constructs such as DataFrames and pandas directly in Snowflake without moving data around
Operationalizing Your Pipelines: Productionize your work faster with debugging, deploying, scheduling and monitoring your pipelines easily

Start the Course Now!

Guide to Build Continuous & Streaming Pipelines

5 Sessions

Best Practices: Data Ingestion with Snowflake

Length: 24-MINUTE VIDEO Learn about the different mechanisms for ingesting data into Snowflake and get our product expert’s recommendation on what to use in different scenarios and for various use cases from batch, streaming to usage of connectors.
Demo: Using Snowflake’s Native Connector for PostgreSQL

Length: 15-MINUTE VIDEO Built with Snowpipe Streaming and Snowflake Native Apps, Snowflake’s PostgreSQL connector allows you to seamlessly ingest your PostgreSQL data securely and fast, with simple clicks directly via Snowflake Marketplace.
Intro: What’s New with Dynamic Tables

Length: 34-MINUTE VIDEO Dynamic Tables is the new default for change data and incremental processing. This session will provide an overview of Dynamic Tables and explore its key benefits, top use cases, latest improvements and newest capabilities.
Best Practices: Dynamic Tables

Length: 38-MINUTE VIDEO Learn the best practices of Dynamic Tables for optimized development, testing, monitoring, performance and more.
Advanced Usage of Kafka Connector

Length: 35-MINUTE VIDEO Use Kafka for your streaming data? Learn all the key concepts with Snowflake’s Kafka Connector from configuration, optimization, monitoring and error handling.

Guide for Python Developers

4 Sessions

Intro: What’s New with Python Data Pipelines

Length: 16-MINUTE VIDEO Learn about the latest capabilities built for the Python data engineers.
Demo: End-to-End Python Data Pipelines

Length: 17-MINUTE VIDEO Watch this end-to-end Python Pipeline demo using some of the latest features built for the Python data engineers including Snowflake Notebooks, Snowpark pandas API, Serverless Tasks and Git Integration.
Demo: Leveraging Snowpark to Connect with External Endpoint

Length: 16-MINUTE VIDEO Watch this demo to see how you can leverage some of the advanced capabilities of Snowpark to build connections with external endpoint and process unstructured data.
Best Practices: Migrating Data Engineering Pipelines

Length: 40-MINUTE VIDEO Learn from our data engineering experts the best practices to migrate data engineering pipelines including how to operate, tune and optimize pipelines at scale, validate new pipelines, and manage separate codebases during migration.

Operationalizing Your Pipelines

4 Sessions

Best Practices and Demo: Orchestrating Data Pipelines

Length: 16-MINUTE VIDEO Tasks: Snowflake’s orchestration capability has got a lot of great improvements. Watch this session to learn what are some best practices when choosing Tasks with user-managed compute, serverless option or with Serverless Tasks Flex to balance art of performance and cost effectiveness for your use cases.
Demo: DevOps with Snowflake

Length: 21-MINUTE VIDEO Learn about the latest features for simplifying and automating DevOps processes, including the Snowflake Python API, CLI and Git integration, as well as database change management capabilities.
Intro and Demos: Observability in Snowflake

Length: 25-MINUTE VIDEO Learn about Snowflake Trail, a set of observability capabilities in Snowflake to monitor, debug and troubleshoot easily for your pipelines and applications. Watch a series of demos to see them in action.
Demo: Setting up Alerts and Notifications in Snowflake

Length: 18-MINUTE VIDEO Watch this demo to learn how you can easily set up alerts and notifications directly in Snowflake.

Data Engineering School

Start the Course Now!

Guide to Build Continuous & Streaming Pipelines

Best Practices: Data Ingestion with Snowflake

Demo: Using Snowflake’s Native Connector for PostgreSQL

Intro: What’s New with Dynamic Tables

Best Practices: Dynamic Tables

Advanced Usage of Kafka Connector

Guide for Python Developers

Intro: What’s New with Python Data Pipelines

Demo: End-to-End Python Data Pipelines

Demo: Leveraging Snowpark to Connect with External Endpoint

Best Practices: Migrating Data Engineering Pipelines

Operationalizing Your Pipelines

Best Practices and Demo: Orchestrating Data Pipelines

Demo: DevOps with Snowflake

Intro and Demos: Observability in Snowflake

Demo: Setting up Alerts and Notifications in Snowflake