An Evolution of Cloud Data Architectures - Lambda, Kappa, Delta, Mesh & Fabric

How has advancements in highly scalable cloud native technology influenced the design principles we apply when building data platform solutions? Are we designing for just speed and batch layers or do we what more from our platforms, and who says these patterns must be delivered exclusively?

Let’s disrupt the theory and consider the practical application of all things Microsoft now has to offer, where concepts, patterns, and best practice meet/clash with technology. Can we now utilise cloud technology to build architectures that cater for lambda, kappa, and mesh concepts in a complete stack of services? And should we be considering a solution that offers all these principals in a nirvana of data insight high scalable, decoupled perfection? Lastly, how does Data Fabric as a concept fit with Microsoft Fabric as a product and should we decentralise everything as suggested by the data mesh!?

In this session we’ll explore the answer to all these questions and more in a thought provoking, argument generating look at the challenges every data platform engineers/architects face.

• Examine how advancements in cloud-native technology influence data architectures, specifically the Lambda, Kappa, and Mesh models. • Discuss the integration of various architectural patterns and best practices into a cohesive, highly scalable data platform solution utilizing Microsoft technologies. • Explore the concepts of Data Fabric and Data Mesh, and their practical application in achieving decentralized, highly scalable, and insightful data platforms.