The Agenda

In this session we will be diving into most of the major moving parts of an automated enterprise BI solution, as per Microsoft reference architecture (Enterprise business intelligence - Azure Reference Architectures | Microsoft Docs). Azure Active Directory, Blob storage, Azure Monitor, Azure Synapse, Azure Data Factory, Azure Analysis Services and Power BI serving as key pillars in building a solid custom framework for automated data ingestion and analysis. Learn how to setup each of these services, how they interact and how to benefit from built in automation in Azure.
Leading by example, we will be tracing data from various sources on its journey through the Azure services, configuring them configure as we go along.

The day is comprised of a number of parts, one for each major aspect of the architecture.

Introduction (1 hr)

Break - 15 min

Sources (1 hr)
In this section, we will go over some of the options available in Azure, and how we would go about sourcing our data.
The most common sources we will cover are:
- Azure SQL Database
- (S)FTP folder(s)
- Blob storage
- WebService

Break - 15 min

Ingestion (2 hrs)
The staging area will be covering the landing zone in our Azure Data Lake Storage Gen2, discussing the various options we have for supporting Delta lake etc. A huge part of this will also focus on setting up Azure Synapse Analytics, so that we can examine which offering is the most appropriate for our scenario.

Break - 1 hr - Lunch

Automation (1 hrs)
The automation part in Azure concentrates about Azure Data Factory/Azure Synapse pipelines and the number of ways the platform offers to invoke pipelines/notebooks/data flows.

Break - 15 min

Model (1 hr)
In part we will be creating a Power BI model, where we will investigate how to configure partitions with incremental data refresh and what options we have for deployment.

Break - 15 min

Wrapping it all up (1 hr)
Rounding off the day by going over all of the major moving parts and how they interact. We will also briefly touch some of the pitfalls to avoid.

Prerequisites:
A basic knowledge about Azure and some data platform technology is helpful