Paul Andrew
mrpaulandrew.com
Microsoft Data Platform MVP. 10+ years’ experience working with the complete on premises SQL Server stack. Now working as a principal consultant and solution architect dealing with data platform deliveries in Azure.
SQLBits 2019
Complex Azure Orchestration w Dynamic Data Factory Pipelines
If you have already mastered the basics of Azure Data Factory (ADF) and are now looking to advance your knowledge of the tool this is the session for you.
If you have already mastered the basics of Azure Data Factory (ADF) and
are now looking to advance your knowledge of the tool this is the
session for you. Yes, Data Factory can handle the orchestration of our
ETL pipelines. But what about our wider Azure environment? In this
session we’ll go beyond the basics looking at how we build custom
activities, metadata driven dynamic design patterns for Data Factory.
Plus, considerations for optimising compute costs by controlling other
service scaling as part of normal data processing. Once we can hit a
REST API with an ADF web activity anything is possible, extending our
Data Factory and orchestrating everything.
are now looking to advance your knowledge of the tool this is the
session for you. Yes, Data Factory can handle the orchestration of our
ETL pipelines. But what about our wider Azure environment? In this
session we’ll go beyond the basics looking at how we build custom
activities, metadata driven dynamic design patterns for Data Factory.
Plus, considerations for optimising compute costs by controlling other
service scaling as part of normal data processing. Once we can hit a
REST API with an ADF web activity anything is possible, extending our
Data Factory and orchestrating everything.