Video unavailable
SQLBits 2024
Microsoft Data Integration Pipelines – The Fundamentals to Level 300
Start this full day of training with the fundamentals of data pipelines before building the skills needed to orchestrate an entire data platform solution. We’ll focus Azure Data Factory as the mature resource, but also cover Synapse Analytics and Microsoft Fabric pipelines.
In this full day of training, we will start with the very basics of data integration before building up the fundamental skills and artifacts needed to orchestrate your data platform end-to-end. We will do this using Azure Data Factory pipelines, not exclusively, but as the basis for our learning journey. The maturity of the Data Factory resource offers a set of good foundations to understand the technical capabilities needed for orchestration, before applying our knowledge to other tools such as Microsoft Fabric and Azure Synapse Analytics.
Through a blend of theory, demonstration, and practical labs we will explore the components needed to orchestration cloud-based data processing workloads from source system ingestion, through to data model delivery. Focusing on the control flow plane, with supplementary options for building out low code data flow packages available as part of our integration pipelines.
Start the day knowing nothing about data integration pipelines, or as an experienced SQL Server SSIS developer, and leave with the knowledge and resources to apply these skills to your role as a data professional working with Microsoft cloud native tools.
Through a blend of theory, demonstration, and practical labs we will explore the components needed to orchestration cloud-based data processing workloads from source system ingestion, through to data model delivery. Focusing on the control flow plane, with supplementary options for building out low code data flow packages available as part of our integration pipelines.
Start the day knowing nothing about data integration pipelines, or as an experienced SQL Server SSIS developer, and leave with the knowledge and resources to apply these skills to your role as a data professional working with Microsoft cloud native tools.
Speakers
Paul Andrew's previous sessions
An Evolution of Data Architectures - Lambda, Kappa, Delta, Mesh & Fabric
How has advancements in highly scalable cloud technology influenced the design principals we apply when building data platform solutions?
Building an Azure Data Analytics Platform End-to-End
Based on real world experience let’s think about just how far the breadth of our knowledge now needs to reach when starting from nothing and building a complete Microsoft Azure Data Analytics solution.
Creating a Metadata Driven Orchestration Framework Using Azure Data Integration Pipelines
We'll explore delivering this framework within an enterprise and consider an architect’s perspective on a wider platform of ingestion/transformation workloads with multiple batches and execution stages.
ETL in Azure Made Easy with Data Factory Data Flows
What happens when you combine a cloud orchestration service with a Spark cluster?! The answer is a feature rich, graphical, scalable data flow environment to rival any ETL tech we’ve previously had available in Azure.
Using Azure DevOps for Azure Data Factory
DevOps as a concept does not always translate to the technology when implemented. In this session we'll explore that problem when working with Azure Data Factory and what the different cloud only CI/CD options are.
Complex Azure Orchestration w Dynamic Data Factory Pipelines
If you have already mastered the basics of Azure Data Factory (ADF) and are now looking to advance your knowledge of the tool this is the session for you.
Building an End to End IoT Solution Using Pi Sensors & Azure
Demonstrating an end to end IoT solution providing real-time sensor data from a Raspberry Pi into an Azure IoT Hub, through Stream Analytics, then with outputs to Power BI and SQL DB. Learn how to build this simplified IoT solution from scratch.