Have you ever found yourself at the start of an Azure data engineering project, unsure about what tool to choose? Speak no more! In this session we will discuss three often used data engineering tools on Azure: - Azure Data Factory - Azure Databricks - Azure Synapse Analytics
The video is not available to view online.
In this session, seasoned data engineer and youtube grumbler Simon Whiteley takes us on a journey through the current industry trends and buzzwords, carving through the hype to get at the underlying ideals.
The video is not available to view online.
Based on real world experience let’s think about just how far the breadth of our knowledge now needs to reach when starting from nothing and building a complete Microsoft Azure Data Analytics solution.
The video is not available to view online.
In this session, we will discuss the evolution of the big data landscape, to the data lakehouse concept. We will focus on how an Open Data Lakehouses are developed, and how to secure, govern and augment using Dremio.
The video is not available to view online.
Are you able to fully understand the state of your data in your systems? Are you able to evaluate the quality of your data in all processes? In this session you will learn how easy it is to integrate HEDDA.IO into your existing processes, how it can provide you with a continuous view of the quality of your data and how it can inform you directly in the event of errors, warnings or major changes to individual parameters. HEDDA.IO brings data observability to your process and platforms.
The video is not available to view online.
Modern software engineering and management for ETL, so data analysts and engineers can spend less time on tooling and focus on getting value from data.
The video is not available to view online.
In this session, we will see how to use Delta Live Tables to build fast, reliable, scalable, and declarative ETL pipelines on Azure Databricks platform.
The video is not available to view online.
Gain insight into how data can be loaded, processed, and uploaded to other environments in a self-service process from a wide variety of environments in a fully controlled and audited manner. Whether SQL Server, Data Lake, Delta Lake, SAP HANA or many other data sources and targets, WOODY.IO helps you implement clear governance processes into your self-service integration processes.
The video is not available to view online.
HEDDA.IO is a central data quality management solution that connects departments, data stewards and data engineers. It helps to easily integrate standardization, cleansing, matching and enrichment tasks into existing processes.
The video is not available to view online.
Delta Live Tables is a new framework available in Databricks that aims to accelerate building data pipelines by providing out of the box scheduling, dependency resolution, data validation and logging. We'll cover the basics, and then get into the demo's to show how we can: - Setup a notebook to hold our code and queries - Ingest quickly and easily into bronze tables using Auto Loader - Create views and tables on top of the ingested data using SQL and/or python to build our silver and gold layers - Create a pipeline to run the notebook - See how we can run the pipeline as either a batch job, or as a continuous job for low latency updates - Use APPLY CHANGES INTO to upsert changed data into a live table - Apply data validation rules to our live table definition queries, and get detailed logging info on how many records caused problems on each execution. By the end of the session you should have a good view of whether this can help you build our your next data project faster, and make it more reliable.
The video is not available to view online.
Be part of a live audience recording of an episode of Knee-Deep in Tech!
The video is not available to view online.
In this demo-rich session you will learn how you can implement a framework for data quality validation and monitoring, spanning your end-to-end data platform including Power BI!
The video is not available to view online.
In this session, you will get to learn how to enrich your data in spark tables with models created using Automated ML in Azure Synapse Analytics. We will be looking at a demo where we will create regression and classification models using Automated ML and how to use these models for prediction in Azure Synapse Analytics.
The video is not available to view online.
Overview to some of the more popular Azure Data Engineering services used to analyze data
A intro session showing the whys, how's and what's for building a Data Lakehouse in Azure Synapse Analytics
In this session, we will cut through the marketing buzzwords to share experiences, tips, and tricks on how to be successful with Data Science and Analytics in the real world. Tune in to hear the team share real-world experience and get takeaways from industry insiders on real projects with impact. We will also discuss the ethics and fairness of Data Science and Analytics projects and how we can be more inclusive from a technology, people, and process standpoint.
The video is not available to view online.
Building a Data Quality implementation in Purview using the Glossary and Databricks
This session will discuss a custom solution we have built to solve the problem of row level encryption in a highly complex data lake.
Join me to learn how to migrate a traditional DW transformation process to Spark/Scala based platform and reuse existed experience
<<12>>