How Dropbox Implemented a Modern SIEM with Snowflake and Panther
As a file hosting service, Dropbox offers cloud storage, file synchronization, personal cloud, and client software. With 700 million users as of 2021 and hosting over 550 billion pieces of content in total, security and governance are top priorities for the company.
Securing and storing security data in its entirety poses a challenge—how can security teams manage visibility en masse while maintaining scale? Legacy systems and processing tend to result in extremely high cost for ingestion alone. To counter that cost, security teams typically retain data for 30, 60, maybe 90 days maximum. As a result, security teams are forced to stack rank data sources, only ingesting what’s deemed most important into the security tools while the remaining data gets dumped into storage, resulting in data silos and visibility issues.
With such a legacy system, the onus of tying together this disparate data falls on the security teams. In turn, they spend a lot of time on manual processes. A security posture spanning data silos forces security professionals to make decisions based on limited information. Instead of decisions being entirely data-driven, time-sensitive decisions are forced to be based on gut feelings. Many of the security tools on the market require specialized skill sets and security leaders often struggle to hire, train, and retain security professionals.
Dropbox faced these common security challenges and set out to improve processes with its new detection and response tool, which it used to:
- Eliminate data silos and reduce cost
- Achieve visibility across cloud and on-prem sources
- Retain terabytes of security log for longer periods of time
- Implement near real-time alerting
- Create built-in modular detection frameworks
- Create detections as code with testing facilities
- Perform threat intelligence and data enrichment with low false positive rates
Snowflake connected with Jeff Puchalski, Senior Cloud Security Architect at Dropbox, to learn how the company leverages Snowflake and Panther to scale threat detection and response. Watch the full Dropbox webinar here.
Security challenges
Why Snowflake for cybersecurity?
Snowflake is cross-cloud, available on any of the major cloud providers, and a global solution. With usage-based pricing you can have as many Snowflake warehouses as you need, all while adhering to global privacy requirements like GDPR. Here are four of the key reasons Jeff Puckalski, Senior Cloud Security Architect at Dropbox, shared for why Dropbox switched to Snowflake:
1. Ability to store a near-infinite amount of data at a low cost with Snowflake’s architecture that separates storage from compute: “With Snowflake compression, we typically see a 10-to-1 compression ratio. Snowflake also automates the encryption of that data in transit, and at rest can provide role-based access, dynamic data masking, making it more than just storage of that data, but actually protecting that data as it sits. And again, due to compression and the low cost, this makes it extremely affordable to store,” said Puchalski.
2. Return fast query results on always hot data: “Our organization typically had to go through a tiering strategy of hot, warm, cold, and a rehydration process to access data over X months old,” said Puckalski. “That does not exist within Snowflake. You can search everything that’s there, which becomes extremely handy when doing investigations or doing things like applying the IOC to historical data.”
3. Scale your warehouses up and down as you need, and only pay for compute you use: Investigating and scanning a year’s worth of historical data can be done with a click of a button and is not disruptive to the end user. With Snowflake’s consumption model, you can run queries as needed, and scale back when not in use. For example, if a team needs to find one IP address across a year or two years’ worth of data, you have the option to crank up your Snowflake warehouse to a double X, triple X, or quadruple XL and look for that IP, then scale back to everyday usage. According to Puchalski, “Frequently, the difference between a successful remediation event and a non-successful remediation event is how fast we can get the answers that we need when the pressure is on.”
4. Reduce workload contention: With Snowflake, separation of compute and storage enables you to load data, perform analytics, and perform tests on a single source of data without impacting performance. So your detection engineering team can be working on new correlations while another team is working on investigations simultaneously. Each of the teams can work within their own environment. This eliminates the worry that someone writing a large query spanning months of data will slow the system down for everybody else.
Why Panther: A modern rules engine on Snowflake
Panther, a Powered by Snowflake solution, is a modern system built for security operations at scale that leverages a number of serverless services in order to process data in large volumes very quickly.
The cloud-native architecture enables downstream benefits for the customers, including real-time alerting. Historically with traditional SIEM, security data is loaded into a database and then crawled over and over to look for anomalies. While this can be effective for certain types of correlation and threat patterns, it is fairly inefficient. It’s a lot more efficient to analyze that data as it comes through the engine and raise alerts in near real time about suspicious activities.
Panther provides “detection as code,” which is a modern, flexible, and structured approach to writing detections that applies software engineering best practices to security. By treating detections as code that can be tested, checked into source control, and code-reviewed by peers, teams can produce higher-quality alerts that reduce false positives and accurately flag suspicious activity.
As a connected application to Snowflake, Panther customers such as Dropbox enjoy Snowflake’s scalability and performance for storing and analyzing security data, while seamlessly using Panther for its out-of-the-box detection and detection-as-code features.
The Snowflake and Panther solution turned out to be a very highly scalable managed solution for us. Managed well in that we don’t have to deal with running most of the infrastructure, [which] takes a lot of the burden off our security engineers from having to deal with the infrastructure itself and being able again to focus on the important parts of the problem.”
Jeff Puckalski
Learn more about Snowflake for cybersecurity.
Authors
