New Snowflake Features Released in May–July 2023
At Snowflake Summit, we announced a wave of product innovations: Snowpark ML Modeling API, Snowflake Native App Framework, Dynamic Tables and more. If you missed out, not to worry! Read our Summit recap blog for highlights across industries or watch Summit sessions now on-demand. Let’s dive into all the new releases in May, June and July.
Applications
Snowflake Native App Framework now available in AWS - public preview
Snowflake Native Apps are an entirely new way to put data to work. Consumers can discover and purchase Snowflake Native Apps on Snowflake Marketplace, then install and run them within their own Snowflake account. Developers can now start building and testing Snowflake Native Apps in their accounts in AWS. Distribution and monetization capabilities will be available in public preview on AWS later this year. The Snowflake Native App Framework remains available in private preview on Google Cloud Platform and Azure. Learn more here.
Build your own pricing strategy for your Snowflake Native Apps - public preview
Snowflake offers a range of monetization options, from subscription-based models to usage-based models priced per month or per query. Custom Event Billing capabilities enable you to build your own pricing strategy. You can charge customers based on their usage and specify billing events based on your preferences, such as consumed rows, ingested rows and more. Learn more here.
Streaming and Continuous Pipelines & Data Onboarding
Snowpipe Streaming - general availability
Streaming pipelines are often challenging to build. Data comes in a continuous manner, and often a separate architecture is required to handle streaming data. What remains challenging is how streaming data is brought together with batch data. That’s why we built Snowpipe Streaming, now generally available to handle row-set data ingestion. Snowpipe Streaming enables low-latency streaming data pipelines to support writing data rows directly into Snowflake from business applications, IoT devices or event sources such as Apache Kafka, including topics coming from managed services such as Confluent Cloud or Amazon MSK. Learn more here.
Better replication and schematization for streaming ingestion - public preview
With this release, we are adding the support of Snowpipe Streaming with Snowflake replication. Snowflake supports the replication and failover of Snowflake tables populated by Snowpipe Streaming, and its associated channel offsets from a source account to a target account in different regions and across cloud platforms with replication. Snowpipe streaming supports both database replication and group-based replication.
The new Kafka connector, built with Snowpipe Streaming, now supports schema detection and evolution. The structure of tables in Snowflake can be defined and evolved automatically to support the structure of new Snowpipe streaming data loaded by the Kafka connector.
Build incremental pipelines declaratively - public preview
Once the data is landed, building and maintaining streaming and continuous pipelines is a whole different story. This is where Dynamic Tables (currently in public preview) come in. Dynamic Tables is a new table type that drastically simplifies continuous data pipelines for transforming both batch and streaming data, declaratively.
Instead of defining data transformation steps as a series of tasks and having to monitor dependencies and scheduling, you can simply by defining the end state of the transformation with dynamic tables, and leave the complex pipeline management to Snowflake. Using just a few lines of SQL (or Snowpark Python), you can build continuous pipelines with just a few lines of code. Learn more here.
Onboard data easily with schema detection and evolution - pubic preview
We are pleased to announce a public preview of the schema detection feature for JSON and CSV, and table schema evolution, making data onboarding easier. Schema can be automatically detected for Apache Parquet, Apache Avro and ORC files, further expanding our support to include JSON and CSV files. Not only for detection, we are also making improvements for table schema evolution. The structure of tables in Snowflake can now evolve automatically to support the structure of new data received from the data sources.
Snowpark
Faster and more intuitive end-to-end ML development in Snowflake - public preview
Now in public preview, the Snowpark ML Modeling API scales out feature engineering and simplifies ML training execution natively in Snowflake. Learn more here.
New Python versions in Snowpark - public preview
Snowpark now supports Python versions 3.9 and 3.10. Visit Snowflake Documentation for more details.
Advanced Analytics
Enhance speed and decision quality with ML-Powered Functions - public preview
Enhance speed and quality of decisions with ML-Powered Functions, now in public preview. These familiar SQL functions abstract the complexity of ML frameworks and algorithms for time-series forecasting, anomaly detection and more. Now you can scale from one to millions of ML-powered insights with the elasticity and near-zero operations of Snowflake’s engine, and share insights in analytics or BI tools integrated with Snowflake’s consistent data governance across model inputs and outputs. Learn more about ML-Powered Functions in our blog or in Snowflake documentation.
Advance geospatial analysis with a new data type: GEOMETRY - general availability
GEOMETRY, a data type designed for those using local spatial reference systems (SRS), is now generally available in Snowflake. GEOMETRY uses planar geometry and supports thousands of projections, which is especially useful in representing local areas while minimizing distortion errors. Learn more about how to use high-precision geospatial data by watching this virtual hands-on lab.
Perform orientation shape transformations for geospatial - general availability
Streamline how you perform orientation transformations with our new set of functions, all in GA: ST_SIMPLIFY, ST_BUFFER, ST_AZIMUTH and ST_MAKEPOLYGONORIENTED. Learn more about the functions in this blog post.
Handle invalid geospatial shapes - general availability
Now, you can load and store invalid shapes in our GEOGRAPHY and GEOMETRY columns, and either fix them after the import or keep using them as they are. Additionally, we added a new function, ST_ISVALID, to verify whether a shape is valid. Learn more in Snowflake documentation.
Collaboration and Marketplace
Accelerate purchases on Snowflake Marketplace by paying with Snowflake Capacity - general availability
Eligible Snowflake customers can now purchase third-party data and Snowflake Native Apps seamlessly on Snowflake Marketplace from across clouds with their Snowflake Capacity commitment, accelerating access to third-party content for data teams and streamlining procurement. Check out this blog post to see how you can enroll in the program.
Automated fulfillment of data across regions and clouds - general availability
Listing providers in Snowflake can now ensure their consumers always have fresh, up-to-date data irrespective of their region or cloud. Providers set a replication frequency and select targeted cloud regions, in which their public listings are automatically replicated and kept in sync as soon as there is demand for their listing in a new region. For listings targeted at specific accounts, providers don’t need to specify anything but the replication frequency—Snowflake ensures the data is automatically fulfilled in the respective target accounts’ cloud regions. See Snowflake documentation on how to set up Cross-Cloud Auto-Fulfillment on your listings.
Manage costs of automatically fulfilling data across regions and clouds - public preview
Two new Views have been added to the data sharing usage schema to help Listing Providers manage the cost of Cross-Cloud Auto-Fulfillment; that is, the automated replication of data across regions and clouds. The new Views provide visibility into storage costs associated with replicated data in remote Snowflake regions for the purpose of fulfilling consumer demand for a listing’s data product in a region, as well as visibility into compute costs associated with refreshing the data in remote Snowflake regions. See more details on understanding and managing related costs here as well as schema definitions in Snowflake documentation.
Observability
Application observability gets better with logging and tracing via event tables - public preview
We are improving application observability for developers and data engineers. Prior to Snowflake logging and tracing via event tables, there was no easy way to capture application logs and traces, and no centralized location to consume them. Now with logging and tracing via event tables, developers can better monitor and observe in order to build applications more effectively.
With this new feature, developers and data engineers can easily instrument their code to capture logs and traces across UDFs, UDTFs and stored procedures for all languages: Java, Scala, JavaScript, Python and Snowflake Scripting. Learn more here.
Data Lake
Connect Snowflake to s3-compatible storage devices on-premises or in the cloud - general availability
Customers can use Snowflake to access data in s3-compatible storage devices while getting the ease of use, elasticity, unified governance, resilience and connectivity of Snowflake’s platform. Use cases could include performing analytics on data lakes with External Tables, simplified ingestion of on-premises files to tables in the cloud, or even using Snowpark Python, Java or Scala to process files stored externally. For more information, including a list of supported storage providers and our public test suite, please read our product documentation.
Snowsight
More reasons to upgrade to Snowsight
Snowsight, an updated web interface for Snowflake, has become the default destination for many new and longtime Snowflake users. Most recently, all on-demand customers (those that pay month-to-month based on their usage with a credit card) now see Snowsight by default when they log in. With big new functionality such as new Python Worksheets, the ability to upload a CSV to smaller usability enhancements like Worksheet tabs, and the ability to create named stages, Snowsight gets better every day. Learn more in our Snowsight documentation.
Snowflake Marketplace
Snowflake customers can tap into Snowflake Marketplace for access to more than 1,800 live data sets, packages of data sets, or data services (applications are currently in private preview) from over 430 third-party data providers and data service providers (as of April 30, 2023), as well as market their own products across the Snowflake Data Cloud. Visit Snowflake Marketplace. Here are all the providers who posted new listings in May, June and July:
Transportation
Government
California State Water Resources Control Board
Security
Financial
Bloomberg Data Management Services Limited
Marketing
Geospatial
PTV Planung Transport Verkehr GmbH
Commerce
Demographics
Healthcare and Life Sciences
Connectors
Energy
Lookup Tables
Environment
Media
Identity
Sports
—–
Forward-Looking Statements
This post contains express and implied forward-looking statements, including statements regarding (i) Snowflake’s business strategy, (ii) Snowflake’s products, services, and technology offerings, including those that are under development or not generally available, (iii) market growth, trends, and competitive considerations, and (iv) the integration, interoperability, and availability of Snowflake’s products with and on third-party platforms. These forward-looking statements are subject to a number of risks, uncertainties, and assumptions, including those described under the heading “Risk Factors” and elsewhere in the Quarterly Reports on Form 10-Q and Annual Reports of Form 10-K that Snowflake files with the Securities and Exchange Commission. In light of these risks, uncertainties, and assumptions, actual results could differ materially and adversely from those anticipated or implied in the forward-looking statements. As a result, you should not rely on any forward-looking statements as predictions of future events.
© 2023 Snowflake Inc. All rights reserved. Snowflake, the Snowflake logo, and all other Snowflake product, feature, and service names mentioned herein are registered trademarks or trademarks of Snowflake Inc. in the United States and other countries. All other brand names or logos mentioned or used herein are for identification purposes only and may be the trademarks of their respective holder(s). Snowflake may not be associated with, or be sponsored or endorsed by, any such holder(s).