Mastering Direct Lake

This training day is for anyone who wants to master all aspects of Direct Lake. Direct Lake revolutionizes the way we do data modeling, but also comes with its own set of limitations, quirks, and principles to understand.

Mastering Direct Lake is a key skill set that sets you apart from other data modelers by enabling you to blend Import-like query speed with constantly updated data. Performant, flexible, and fast, even for very large tables.

In this immersive hands-on workshop, we'll cover every facet of Direct Lake, from getting the basics right to advanced modeling and performance tuning - turning you into a pro Direct Lake modeler on the way.

Modules:

1. Getting started with Direct Lake
- Direct Lake prerequisites
- Variations and use cases for Direct Lake
- Creating a Direct Lake model

2. Data prep for Direct Lake
- Examining Parquet, Delta, and compression engines
- Pushing transformations upstream
- Analyzing Delta Lake for optimal Direct Lake performance

3. From Basics to Advanced Data Modeling with Direct Lake
- Data modeling best practices
- Massive data volumes in Direct Lake
- Composite models (Import + Direct Lake)

4. Performance tuning for Direct Lake
- Transcoding
- Cold/hot cache management
- Framing, Partitioning and incremental updates
- Fine-tuning table performance

5. Security for Direct Lake
- Row-level security for Direct Lake
- Lakehouse security implications for Direct Lake
- Setting up security in practice

6. Migration pathways
- From Import to Direct Lake
- From Direct Lake on SQL to Direct Lake on OneLake

- Understand the full scope of Direct Lake - Practice developing and finetuning Direct Lake models - Learn about migration and security implications