22-25 April 2026
Video unavailable
SQLBits 2020

DevOps your Databricks Workspace

Learn how to take your Azure Databricks Workspace and store all you care about in a git repo, and maybe even have some tests running as part of your release pipelines!

Azure Databricks is amazing: within a very short space of time you can get a Workspace deployed and start writing notebooks in a variety of languages, attach those notebooks to clusters and access large volumes of data at scale on clusters.

Inevitable though, you will want to share your code with other team members, deploy versions of the notebooks to different workspaces and configure jobs and secrets. With a simple walk-through of the constituent parts of what makes a workspace, we cover how to get your workspace into a repo, how to develop "locally in the cloud", and some of the challenges of the out-of-the-box source control features that Azure Databricks provides.

We will also touch on the challenge of monitoring and automating tests.
Note this talk is less about development best practices in Azure Databricks from a code perspective: we will be focusing on software development lifecycle and how it applies to Azure Databricks.