Optimizing Snowflake for Efficiency and Cost Reduction at City of Hope National Medical Center.
City of Hope National Medical Center shares their journey of transitioning to Snowflake, optimizing performance and costs.
00:00:00 City of Hope National Medical Center discusses their journey of transitioning from on-prem data warehouse to Snowflake, highlighting the challenges of cost estimation and cost optimization.
🏥 City of Hope is a non-profit Cancer Research Center that is transitioning from on-prem to the cloud using Snowflake.
💰 They conducted performance testing on Snowflake to determine the budget needed for their data warehouse migration.
🔍 They emphasize the importance of cost optimization and monitoring when using Snowflake to avoid unexpected expenses.
00:06:28 This video discusses the process of right-sizing a cloud-based data warehouse for increased workloads, focusing on performance evaluation and response times.
🔑 The workload at City of Hope National Medical Center is increasing due to the addition of new hospitals.
💰 City of Hope uses a pricing calculator to determine the cost of scaling their infrastructure on Snowflake.
🖥️ City of Hope conducted performance evaluations on Snowflake to estimate response times and optimize their workload.
00:12:54 This video discusses how City of Hope National Medical Center optimized the use of Snowflake for their workload. They found that adding clusters helped maintain response time, but increasing warehouse size didn't have much impact. They also developed a method to calculate the number of clusters needed based on concurrency and user count.
🔑 Using a more stringent metric, City of Hope National Medical Center achieved faster response times and ensured user satisfaction.
🔍 Testing the scalability of Snowflake, the center found that adding clusters maintained the same response time, allowing for easy scaling of concurrency.
⚖️ Scaling up query performance did not significantly rely on larger warehouses, but rather on increasing concurrency within clusters.
💰 By analyzing costs based on different levels of workload, the center determined the machine requirements needed to meet response time SLA and user concurrency.
00:19:22 This video discusses how City of Hope National Medical Center optimized their cloud computing costs with Snowflake and found the best performance options to meet their SLA.
⭐️ Determining the most cost-effective and SLA-meeting cluster size involves analyzing different options and considering trade-offs with stakeholders.
💰 Creating a performance curve matrix helps estimate costs and budget for workload requirements, even before implementing auto scaling.
🔀 Utilizing partition pruning and join elimination can further optimize performance and reduce costs in Snowflake.
00:25:51 Learn how to optimize your Snowflake account for cost and performance. Explore cost allocation techniques and new features like query operator stats. Gain insights into storage consumption, query consumption, and other cost areas. Focus on query consumption for the most cost optimization.
The speaker has been working with Snowflake since 2018 and has extensive experience in data warehouses.
Snowflake is a revolutionary technology that offers scalability for various data-related tasks.
The speaker and his team have developed a tool called 'snoptimizer' for right-sizing Snowflake accounts to find a balance between cost and performance.
00:32:20 The video discusses the importance of correctly using resource monitors in Snowflake to avoid excessive costs. It also mentions the new crate and monitor budget functionality and warehouse utilization in private preview.
🔑 Proper resource monitoring and control are essential to prevent unnecessary costs and risks in using Snowflake.
💼 Balancing performance and cost savings is crucial for optimizing Snowflake usage.
📊 Implementing a cost attribution model can help businesses track and allocate Snowflake costs to different units.
00:38:46 How City of Hope National Medical Center Optimizes Snowflake for Efficiency and Cost Reduction
🔑 The new user function in City of Hope National Medical Center's implementation of Snowflake is a game-changer for data professionals, providing deep insights into query optimization and automation.
🔍 The detailed visualization and metrics offered by Snowflake's user function allows for in-depth analysis of query performance, enabling the identification of optimizations and the provision of recommendations.
💡 Snowflake's optimization tools help improve resource utilization, reduce costs, enhance security, and ensure efficient performance, leading to significant savings and continuous monitoring of account usage.
Want to deep dive into this video?
