Using log data in Fabric Real Time Intelligence - a practical example with Spark
2025TL; DR
Want to know how to use Fabric Real Time Intelligence to make logs data? If so - this is the session for you. In this demo-centric session we will go through the end-to-end set-up process to get more visibility into your processes. We will use Fabric events and Fabric Spark Logs to demonstrate this, but any logging can be used.
Session Details
In this session I will discuss how to use Fabric's Real Time Intelligence capabilities to ingest, analyse and action logs from any service.
During this session I will use logs from Fabric events and Fabric Spark as a practical example of how Fabric Real Time Intellgence enables a user to work with logs and make sense from them.
In this session we will:
1 - step through the set-up process
2 - walkthrough the different logging options
3 - ways to analyse the logs with KQL
4 - how to extend the logs to enable actions and alerts from the logs.
This session will be primarily demo-centric, and requires no knowledge of Spark/Data Engineering and/or Real-Time Intelligence. A basic understanding of Fabric concepts is preferable.
During this session I will use logs from Fabric events and Fabric Spark as a practical example of how Fabric Real Time Intellgence enables a user to work with logs and make sense from them.
In this session we will:
1 - step through the set-up process
2 - walkthrough the different logging options
3 - ways to analyse the logs with KQL
4 - how to extend the logs to enable actions and alerts from the logs.
This session will be primarily demo-centric, and requires no knowledge of Spark/Data Engineering and/or Real-Time Intelligence. A basic understanding of Fabric concepts is preferable.
3 things you'll get out of this session
1 - Understanding of how real-time intelligence can be used for logging scenarios.
2 - Understanding of the detailed logging options for Data Engineering in Fabric
3 - Practical guidance on how to configure data engineering logging through real time intelligence.
Speakers
Luke Moloney's other proposed sessions for 2026
Airflow in Fabric - an introduction informed by experience - 2026
Fabric for ISVs - Develop multi-tenancy apps on Fabric - 2026
Lessons from large-scale Fabric deployment - 2026
Practical lessons in optimizing Data Engineering with Spark - Part 1 - 2026
Practical lessons in optimizing Data Engineering with Spark - Part 2 - 2026
Luke Moloney's previous sessions
Flying High with Data Engineering in Microsoft Fabric
In this session demo centric session we will walk through Data Engineering in Microsoft including: Getting started with Notebooks, Learning how Fabric Lakehouse's provides flexibility to use the tools you prefer, The flexibility of data engineering in Spark - including Cluster Configuration, Environments and Library Management and finally, How Copilot enables all developers to be more efficient with Spark.
Harnessing Data Science and AI in Fabric
In this session you will learn about all the latest data science developments in Microsoft Fabric. We will show you how Fabric supports the end-to-end data science workflow and how we plan to evolve these capabilities. We will also look at exciting new AI capabilities including Copilot experiences and more!
Synapse Q&A with PG
SQLBits' has brought members of the Azure Synapse Program Group to Wales to ask YOUR questions. Bring your questions for: Dedicated SQL Pools, Serverless SQL Pools, Spark Pools, Pipelines, Kusto and everything else Synapse related.
The fundamentals of building a lakehouse with Synapse
The fundamentals of building a lakehouse with Synapse