Have you ever found yourself at the start of an Azure data engineering project, unsure about what tool to choose? Speak no more! In this session we will discuss three often used data engineering tools on Azure: - Azure Data Factory - Azure Databricks - Azure Synapse Analytics
The video is not available to view online.
In this session, seasoned data engineer and youtube grumbler Simon Whiteley takes us on a journey through the current industry trends and buzzwords, carving through the hype to get at the underlying ideals.
The video is not available to view online.
Based on real world experience let’s think about just how far the breadth of our knowledge now needs to reach when starting from nothing and building a complete Microsoft Azure Data Analytics solution.
The video is not available to view online.
Modern software engineering and management for ETL, so data analysts and engineers can spend less time on tooling and focus on getting value from data.
The video is not available to view online.
In this session, we will see how to use Delta Live Tables to build fast, reliable, scalable, and declarative ETL pipelines on Azure Databricks platform.
The video is not available to view online.
Delta Live Tables is a new framework available in Databricks that aims to accelerate building data pipelines by providing out of the box scheduling, dependency resolution, data validation and logging. We'll cover the basics, and then get into the demo's to show how we can: - Setup a notebook to hold our code and queries - Ingest quickly and easily into bronze tables using Auto Loader - Create views and tables on top of the ingested data using SQL and/or python to build our silver and gold layers - Create a pipeline to run the notebook - See how we can run the pipeline as either a batch job, or as a continuous job for low latency updates - Use APPLY CHANGES INTO to upsert changed data into a live table - Apply data validation rules to our live table definition queries, and get detailed logging info on how many records caused problems on each execution. By the end of the session you should have a good view of whether this can help you build our your next data project faster, and make it more reliable.
The video is not available to view online.
In this demo-rich session you will learn how you can implement a framework for data quality validation and monitoring, spanning your end-to-end data platform including Power BI!
The video is not available to view online.
In this session, you will get to learn how to enrich your data in spark tables with models created using Automated ML in Azure Synapse Analytics. We will be looking at a demo where we will create regression and classification models using Automated ML and how to use these models for prediction in Azure Synapse Analytics.
The video is not available to view online.
Overview to some of the more popular Azure Data Engineering services used to analyze data
A intro session showing the whys, how's and what's for building a Data Lakehouse in Azure Synapse Analytics
In this session, we will cut through the marketing buzzwords to share experiences, tips, and tricks on how to be successful with Data Science and Analytics in the real world. Tune in to hear the team share real-world experience and get takeaways from industry insiders on real projects with impact. We will also discuss the ethics and fairness of Data Science and Analytics projects and how we can be more inclusive from a technology, people, and process standpoint.
The video is not available to view online.
Building a Data Quality implementation in Purview using the Glossary and Databricks
This session will discuss a custom solution we have built to solve the problem of row level encryption in a highly complex data lake.
Join me to learn how to migrate a traditional DW transformation process to Spark/Scala based platform and reuse existed experience
Curious what Azure Synapse Analytics brings to the table? Bring your scuba gear, as we take a dive and explore everything it has to offer!
Azure Synapse Analytics combines the power of Data Lakes with Data Warehouses, empowering the organizations to build Big Data, Advanced Analytics and Business Intelligence in one single platform.
AI and data is at the center of the digital feedback loops. We have invested in a comprehensive portfolio of AI tools, infrastructure and services. Come to this session to get an update of Azure AI with demos.
Come learn how to use .NET and Spark together in Azure Synapse and elsewhere to cook and analyze your data!
Azure Synapse Analytics combines the power of Data Lakes with Data Warehouses, empowering the organizations to build Big Data, Advanced Analytics and Business Intelligence in one single platform.
Common Data Model as the foundation of Power BI Dataflows and as part of the Open Data Initiative with SAP and Adobe, seems to be a pretty good move from Microsoft. We want to take a closer look to this approach
Taken from the 20+ years of field experiences, many common statistical and data science mistakes have been detected. Session will tackle couple of them.
The video is not available to view online.
Learn how Azure supports interactive, exploratory notebooks (e.g. Jupyter) for data processing and experimentation across a range of scales from simple single-computer work up to massively parallel Databricks clusters.
Learn to make better analytic solutions by following current thoughts on data modeling.
The video is not available to view online.
Databricks, Lakes & Parquet are a match made in heaven, but explode with extra power when using Delta Lake. This session will dive into the details of how Databricks Delta works and how to make the most of it.
What happens when you combine a cloud orchestration service with a Spark cluster?! The answer is a feature rich, graphical, scalable data flow environment to rival any ETL tech we’ve previously had available in Azure.
Come learn how you can leverage the new and advanced features of Delta Lake on Azure Databricks to easily transform your big data analytics and machine learning workloads
In this session we focus on how Spark implements Machine Learning at Scale with Spark ML.
Are you looking to take your career to the next step? Microsoft Certifications help validate knowledge and ability required to perform current and future industry job-roles in a modern digital business. Our certification
Azure Databricks has become one of the staples of big data processing. See how to make the most of it by understanding how Spark works under the covers.
This session provides an end-to-end walk through of how to use Azure Synapse for cloud-hosted advanced analytics, based around a real-world predictive maintenance use case.
An opportunity to explore Scala, and why it is truly a “Data Engineers language”. Using Azure Functions, Data Factory, Azure Data Lake Gen2 and Databricks the basics will be explored, followed by real world examples
Databricks Delta Lake brings new levels of reliability and transformation abilities to big data solutions. Come and learn how you can simplify your ETL and maximise data consistency and resilience, no matter the size
Azure now has two slick, platform-as-a-service spark offerings, but which one should you choose? A separate specialist tools or a one-size-fits-all solution? Join Simon as he compares and contrasts the spark offerings.
<<1>>