The Agenda
Agenda HomeSQLBits 2024 runs from Tuesday 19th – Saturday 23rd March.
Big Data & Data Engineering
Automate the rollout of a complete Azure based data warehouse solution in a day
Description
In just one day we will create a completely automated rollout of a data warehouse solution consisting of Azure SQL DB, Data Factory, Databricks, Storage, Log Analytics, Keyvault and Synapse Serverless.
We will use Bicep, Github Actions, Powershell and API calls to glue everything together. The end result will be a working end to end flow of data, all technologies
will be able to talk to each other via azure active directory where possible, and creating the needed Managed Identities and IAM rights will be part of our automation.
And of course we will have separate Dev, Test and Production environments.
Besides the "how" part, where we'll dive deep into how Bicep works for instance we are taking time to discuss what kind of rights your pipeline needs and what to do when your organisation doesn't grant you everything.
We'll also discuss the differences between Github Actions vs Azure DevOps and Bicep vs Terraform. We'll mix in as much as possible notes from the field to make this day not just a
"start to finish" technical how-to but also a lessons learned. After this training you will not just be able to take the code and run it yourself but also understand all the bits and pieces and
adapt it to your own situation.
We will use Bicep, Github Actions, Powershell and API calls to glue everything together. The end result will be a working end to end flow of data, all technologies
will be able to talk to each other via azure active directory where possible, and creating the needed Managed Identities and IAM rights will be part of our automation.
And of course we will have separate Dev, Test and Production environments.
Besides the "how" part, where we'll dive deep into how Bicep works for instance we are taking time to discuss what kind of rights your pipeline needs and what to do when your organisation doesn't grant you everything.
We'll also discuss the differences between Github Actions vs Azure DevOps and Bicep vs Terraform. We'll mix in as much as possible notes from the field to make this day not just a
"start to finish" technical how-to but also a lessons learned. After this training you will not just be able to take the code and run it yourself but also understand all the bits and pieces and
adapt it to your own situation.
Learning Objectives
Delegates will learn Bicep and will be ready to use the provided scripts and adapt or even build there own from scratch. They will learn Github Actions, and use Powershell to deploy Data Factory pipelines. Besides the tech stack, they will also learn how to adapt if certain things are different or even not possible in their environment, we'll go into which access rights are needed for what and throw in a lot of practical experience.
Previous Experience
real prior experience is not really needed because we're going to go through everything form the start and we're building a complete demo ready system. But having some experience with Azure, knowing the basic principals of data warehousing and having a dislike for DRY, or "Don't repeat yourself" is useful
Tech Covered
Azure, Synapse Analytics, Azure SQL Database, deployment, Data Bricks, Big Data & Data Engineering, DataOps & Automation