OCT 30, 2023
Collect Logs and Traces From Your Snowflake Applications With Event Tables
We are excited to announce the general availability of Snowflake
Event Tables
for logging and tracing, an essential feature to boost application observability and supportability for Snowflake developers.
In our conversations with developers over the last year, we’ve heard that monitoring and observability are paramount to effectively develop and monitor applications. But previously, developers didn’t have a centralized, straightforward way to capture application logs and traces.
Enter the new Event Tables feature, which helps developers and data engineers easily instrument their code to capture and analyze logs and traces for all languages: Java, Scala, JavaScript, Python and Snowflake Scripting.
With Event Tables, developers can instrument logs and traces from their UDFs, UDTFs, stored procedures, Snowflake Native Apps and Snowpark Container Services, then seamlessly route them to a secure, customer-owned Event Table. Developers can then query Event Tables to troubleshoot their applications or gain insights into performance and code behavior.
Logs and traces are collected and propagated via Snowflake’s telemetry APIs, then automatically ingested into your Snowflake Event Table.
Simplify troubleshooting in Snowflake Native Apps
Event Tables are also supported for Snowflake Native Apps. When a Snowflake Native App runs, it is running in the consumer’s account, generating telemetry data that’s ingested into their active Event Table.
Once the consumer enables event sharing, new telemetry data will be ingested into both the consumer and provider Event Tables. Now the provider has the ability to debug the application that’s running in the consumer’s account. The provider only sees the telemetry data that is being shared from this data application—nothing else.
Improve reliability across a variety of use cases
You can use Event Tables to capture and analyze logs for various use cases:
- As a data engineer building UDFs and stored procedures within queries and tasks, you can instrument your code to analyze its behavior based on input data.
- As a Snowpark developer, you can instrument logs and traces for your Snowflake applications to troubleshoot and improve their performance and reliability.
- As a Snowflake Native App provider, you can analyze logs and traces from various consumers of your applications to troubleshoot and improve performance.
Snowflake customers ranging from Capital One to phData are already using Event Tables to unlock value in their organization. “The Event Tables feature simplifies capturing logs in the observability solution we built to monitor the quality and performance of Snowflake data pipelines in Capital One Slingshot,” says Yudhish Batra, Distinguished Engineer, Capital One Software. “Event Tables has abstracted the complexity associated with logging from our data pipelines—specifically, the central Event Table gives us the ability to monitor and alert from a single location.”
As phData migrates its Spark and Hadoop applications to Snowpark, the Event Tables feature has helped architects save time and hassle.
“When working with Snowpark UDFs, some of the logic can become quite complex. In some instances, we had thousands of lines of Java code that needed to be monitored and debugged,” says Nick Pileggi,
Principal Solutions Architect at phData
. “Before Event Tables, we had almost no way to see what was happening inside the UDF and correct issues. Once we rolled out Event Tables, the amount of time we spent testing dropped significantly and allowed us to have debug and info-level access to the logs we were generating in Java.”
One large communications service provider also uses logs in Event Tables to capture and analyze failed records during data ingestion from various external services to Snowflake. And a Snowflake Native App provider offering geolocation data services uses Event Tables to capture logs and traces from their UDFs to improve application reliability and performance.
With Event Tables, you now have a built-in place to easily and consistently manage logging and tracing for your Snowflake applications. And in conjunction with other features such as Snowflake Alerts and Email Notifications, you can be notified of new events and errors in your applications.
Try Event Tables today
To learn more about Event Tables, join us at BUILD
,
Snowflake’s developer conference. Or get started with Event Tables today with a tutorial
and quickstarts for logging
and tracing
. For further information about how Event Tables work, visit Snowflake product documentation
.
Authors
Data Cloud Glossary
SIEM is a technology solution that ingests and logs security events and logs from a wide range of data sources across the IT...
Data Mining for Business Analytics
Data mining for business analytics allows companies to comb through large data sets to identify patterns and relationships that point to solutions.
Observe
Observe's SaaS Observability product ingests anything with a timestamp — logs, metrics, traces - and then curates and...