SQLBits 2023
Spark Execution Plans for Databricks
Introduction into Spark Execution Plans for Databricks for optimizing code and execution.
Databricks is a powerful data analytics tool for data science and data engineering, but understanding how code is executed on cluster can be daunting.
Using Spark execution plans allows you to understand the execution process and flow, this is great for optimizing queries and identifying bottlenecks.
This session will introduce you to Spark execution plans, the execution flows and how to interrogate the different plans.
By the end of this session, you will have everything you need to start optimizing your queries.
Using Spark execution plans allows you to understand the execution process and flow, this is great for optimizing queries and identifying bottlenecks.
This session will introduce you to Spark execution plans, the execution flows and how to interrogate the different plans.
By the end of this session, you will have everything you need to start optimizing your queries.
Speakers
Falek Miah's previous sessions
Value of DevOps Release Process in Data Teams
Have you ever wondered why release plans, approaches, and environments are important in the world of data operations? Many data professionals come from various backgrounds without prior software development experience, leading to questions about the necessity of these concepts.
In this session, we will discuss the significance of DevOps Release Processes for data teams. We will explore how insufficient processes can lead to delays in deployment, introduce breaking changes, hinder team collaboration and result in multiple releases.
Introduction to the wonders of Azure DevOps
Azure DevOps is the leading deployment tool for build and release solutions end to end. It helps you plan your Agile project, manages Git code, and deploys solutions using Continuous Integration (CI) and Continuous Deployment (CD) pipelines.
In this session we will cover some of the core components of Azure DevOps and show you how to implement a secure deployment pipeline, using unit tests and gating with your CI builds and CD releases.
Spark Execution Plans for Databricks
Introduction into Spark Execution Plans for Databricks for optimizing code and execution.
Building Your Data Analytics Team - live RunAs Radio episode
Panel Discussion moderated by Richard Campbell of RunAs Radio!
Automate the deployment of Databricks components using Terraform
Introduction into Terraform, Databricks provider and steps required to build an automated solution to provision Databricks workspace and resources into Azure cloud platform using Terraform.