22-25 April 2026 
Niall Langley
Niall is a independent consultant specialising in data engineering & platform architecture. He has been working with the Microsoft Data Platform tools for over 13 years, these days Niall helps clients build robust, scalable data pipelines built around Databricks and the Lakehouse architecture.
Niall is actively engaged in the data community, blogging occasionally, and regularly speaking at user groups and conferences in the UK, including SQLBits in 2019 & 2020. Niall is a committee member for Data Relay, and has been a helper at SQL Bits for many years.
Niall Langley's Sessions
Better ETL with Managed Airflow in ADFSQLBits 2024
Building complex data workflows using Azure Data Factory can get a little clunky, especially as you orchestration needs get more complex. Recently another option become available, Managed Airflow in ADF.
Managed Airflow brings Apace Airflow to Azure as a PaaS service. In this session we discover what Airflow is, why we might want to use it for ETL orchestration, and see how it works with lots of demos.
Ask the Expert - Group B (Thu 10:10 to 11:00)SQLBits 2024
Come and see to some of our amazing speakers on one of these topics:
5. Data Science and Analytics:
6. Microsoft Tools and Technologies:
7. Programming and Development:
8. Data Management and Governance:
9. User Experience and Design:
Ask the Expert - Group B (Fri - 10:10 to 11:00)SQLBits 2024
Come and see to some of our amazing speakers on one of these topics:
5. Data Science and Analytics:
6. Microsoft Tools and Technologies:
7. Programming and Development:
8. Data Management and Governance:
9. User Experience and Design:
Ask the Expert - Group B (Sat: 16:00 to 16:50)SQLBits 2024
Come and see to some of our amazing speakers on one of these topics:
5. Data Science and Analytics:
6. Microsoft Tools and Technologies:
7. Programming and Development:
8. Data Management and Governance:
9. User Experience and Design:
Introduction to Databricks Delta Live TablesSQLBits 2023
Delta Live Tables is a new framework available in Databricks that aims to accelerate building data pipelines by providing out of the box scheduling, dependency resolution, data validation and logging.
We'll cover the basics, and then get into the demo's to show how we can:
- Setup a notebook to hold our code and queries
- Ingest quickly and easily into bronze tables using Auto Loader
- Create views and tables on top of the ingested data using SQL and/or python to build our silver and gold layers
- Create a pipeline to run the notebook
- See how we can run the pipeline as either a batch job, or as a continuous job for low latency updates
- Use APPLY CHANGES INTO to upsert changed data into a live table
- Apply data validation rules to our live table definition queries, and get detailed logging info on how many records caused problems on each execution.
By the end of the session you should have a good view of whether this can help you build our your next data project faster, and make it more reliable.
Slowly Changing Dimensions made Easy with Durable KeysSQLBits 2020
In this session we look at a simple way to implement Kimball durable keys on a SCD2 dimension. This provides an easy, performant, way to support reporting on data using historical and current hierarchies.
SQL Server Encryption for the LaymanSQLBits 2019
With GDPR and the number of data breaches we see in the news, encrypting sensitive data is incredibly important. In this talk we start with the basics of encryption, moving on to look at the ways we can encrypt data in SQL Server and Azure SQL DB.
Better ETL with Managed Airflow in ADFSQLBits 2024
Building complex data workflows using Azure Data Factory can get a little clunky, especially as you orchestration needs get more complex. Recently another option become available, Managed Airflow in ADF.
Managed Airflow brings Apace Airflow to Azure as a PaaS service. In this session we discover what Airflow is, why we might want to use it for ETL orchestration, and see how it works with lots of demos.
Ask the Expert - Group B (Thu 10:10 to 11:00)SQLBits 2024
Come and see to some of our amazing speakers on one of these topics:
5. Data Science and Analytics:
6. Microsoft Tools and Technologies:
7. Programming and Development:
8. Data Management and Governance:
9. User Experience and Design:
Ask the Expert - Group B (Fri - 10:10 to 11:00)SQLBits 2024
Come and see to some of our amazing speakers on one of these topics:
5. Data Science and Analytics:
6. Microsoft Tools and Technologies:
7. Programming and Development:
8. Data Management and Governance:
9. User Experience and Design:
Introduction to Databricks Delta Live TablesSQLBits 2023
Delta Live Tables is a new framework available in Databricks that aims to accelerate building data pipelines by providing out of the box scheduling, dependency resolution, data validation and logging.
We'll cover the basics, and then get into the demo's to show how we can:
- Setup a notebook to hold our code and queries
- Ingest quickly and easily into bronze tables using Auto Loader
- Create views and tables on top of the ingested data using SQL and/or python to build our silver and gold layers
- Create a pipeline to run the notebook
- See how we can run the pipeline as either a batch job, or as a continuous job for low latency updates
- Use APPLY CHANGES INTO to upsert changed data into a live table
- Apply data validation rules to our live table definition queries, and get detailed logging info on how many records caused problems on each execution.
By the end of the session you should have a good view of whether this can help you build our your next data project faster, and make it more reliable.
A Guide to Community Tools for Data Engineering on AzureSQLBits 2022
This session looks at some of the great tools produced by the data community that can help speed up building a data engineering project on Azure, as well as providing great examples of how to build reliable components and simple ways to solve common data engineering problems.
An Introduction to Azure for Data ProfessionalsSQLBits 2022
The first time you open the Azure portal, there are so many new icon and services that it can be a bit overwhelming. If you are just starting out with Azure, this session is for you.
Building a Modern Data Platform Using a Test Driven ApproachSQLBits 2022
In data engineering, testing our pipelines and data flows is all too often a difficult, manual and time consuming process. In this session we will look at how we can use Azure DevOps and Azure Dev Test Labs to auto-magically build out a whole data platform environment, both for developers to use and for automated tests to run.
Metadata Driven Pipelines with Python and DatabricksSQLBits 2022
In this demo filled session we'll take a journey from data pipelines built with notebooks to building scaleable, metadata driven pipelines using python functions.
Slowly Changing Dimensions made Easy with Durable KeysSQLBits 2020
In this session we look at a simple way to implement Kimball durable keys on a SCD2 dimension. This provides an easy, performant, way to support reporting on data using historical and current hierarchies.
SQL Server Encryption for the LaymanSQLBits 2019
With GDPR and the number of data breaches we see in the news, encrypting sensitive data is incredibly important. In this talk we start with the basics of encryption, moving on to look at the ways we can encrypt data in SQL Server and Azure SQL DB.