The introduction of Generative AI in the enterprise heralds a new era of advanced analytics and operational efficiency.

In our data community, we tend to use a lot of technical jargon that is meaningless to business executives seeking outcome-oriented solutions. Instead of your business cases getting shuffled into technology budgets, bring your AI initiative to the forefront by focusing on business priorities and value. Data mesh, data fabric, data lakehouse projects and others have failed to do this, and have taken a toll on the rigor required to make your AI case. In this session you will learn to flip the script - talk value first, educate and provide data literacy to your executive team and stakeholders, and make your AI solutions a reality in record time, with the right level of investment.

How to unlock business knowledge to accelerate data value.

Big data has moved beyond being just a buzzword; it's now at the heart of modern business strategies. When used effectively and efficiently, data can open up new revenue opportunities, provide deep insights, and even drive social impact. As digital transformation accelerates, data is no longer just a tool—it's woven into the fabric of every part of an organization. Designing and maintaining a tier 1 data platform has become essential to staying ahead of the competition. 

Especially with AI-driven applications on the rise, the convergence of DevSecOps and DataOps is becoming increasingly critical. The recent global disruption caused by a security company's mistake was a wake-up call—highlighting just how high the stakes can be. Building and scaling data platforms isn't enough; security and scalability need to be integral to the entire data lifecycle. 

Bringing more than a decade of SRE experience to maintaining and managing top enterprise software, we will discuss how to tear down silos and encourage collaboration among development, security, operations, and data teams. By doing so, organizations can achieve unprecedented levels of reliability and security. Integrating DevSecOps with DataOps doesn't just automate and protect data operations—it also safeguards data integrity, privacy, and compliance, even as data environments expand in size and complexity. In today's competitive market, this proactive stance is what will set the leaders apart from the rest.

Main Actionable Takeaways:

• Cultivate a Collaborative Culture

• Prioritize Resilience and Recovery

• Integrate Security Seamlessly into Data Pipeline

In today's data-centric business landscape, robust governance, comprehensive auditing, and resilient disaster recovery are paramount for ensuring data integrity, availability, and compliance. This session will explore best practices and advanced strategies for managing and securing your Power BI and Microsoft Fabric environments. Discover how to mitigate risks, optimize operational efficiency, and derive maximum value from your data assets.

Improve your data infrastructure with governance and security, using proven methods and best practices. Break down data silos, foster collaboration, and optimise data accessibility, empowering your business units with the data and technologies they need. Learn how AI improves efficiency and streamlines data product development. And see how Microsoft Fabric simplifies data estate modernization with a focus on unifying your data in an open and governed foundation.

Join us Join us for an engaging and insightful session as we delve into the innovative patterns of mesh, fabric, and knowledge hubs, all grounded in federated operating principles. We’ll explore the common pitfalls encountered on the data journey, key considerations for success, and how Microsoft’s cutting-edge solutions can drive your transformation forward. 

Generative AI (GenAI) has garnered significant attention for its potential to revolutionize various industries, from creative arts to data analysis. However, organizations are realizing that implementing GenAI is not as easy as just asking ChatGPT a few questions. Providing the most relevant and accurate contextual data to the LLM is critical if organizations are going to realize the full benefits of GenAI. Retrieval Augmented Generation, or RAG, is a well understood and effective technique for augmenting the original user prompt with additional, contextual data. However, many examples of RAG grossly oversimplify the reality of enterprise data ecosystems. In this session, we will examine how a Logical Data Fabric can make RAG a practical reality in large, complex organizations and deliver AI-ready data that make RAG effective and accurate.

As many organizations strive to harness the transformative power of Generative AI, implementing a data fabric has emerged as the solution of choice to manage and leverage vast amounts of data effectively.

Join Kaycee Lai, Founder of Promethium, and Matt Clark, who leads data at National Grid Energy Transmission (NGET), as they explore how National Grid is deploying a data fabric to accelerate time-to-insight, enhance data value, and empower AI initiatives. They will discuss the specific challenges their team aimed to address, the strategic approach they took to implement the data fabric, and the critical role that data products play in delivering rapid analytics and enabling Generative AI.

Don’t miss this insightful session to learn how the company is pioneering data fabric technology to drive innovation and efficiency in the energy sector.

As many organizations strive to harness the transformative power of Generative AI, implementing a data fabric has emerged as the solution...

How about a workplace where generative AI accelerates every data management task, transforming routine into innovative experiences? A vision which can be in production for the AWS customers in just 60 days through a combination of Amazon Bedrock, which enables rapid development and deployment of AI applications, and Stratio Generative AI Data Fabric, which provides accurate output based on quality data with business meaning. Join us to learn how a combination of these products is empowering data managers and chief data officers to drive innovation and efficiency across their organizations. 

Send us a text Talking data with Dima Spivak, Director of Product Management, StreamSets.  Data integration and real-time decision making.

02:02 Dima Spivak is Here! 04:19 Why StreamSets?06:00 What is StreamSets?09:48 On Demand Expense11:34 Regulated Industries12:36 Secret Sauce14:41 A Competitive View15:50 Data Fabric and StreamSets18:25 StreamSets and AI21:12 Use Cases24:02 The Future of Streaming25:48 Quality and Testing31:19 For FunLinkedin: linkedin.com/in/dmitryspivak Website: https://www.ibm.com/blog/announcement/ibm-acquires-streamSets/ Want to be featured as a guest on Making Data Simple?  Reach out to us at [email protected] and tell us why you should be next.  The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.  Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

As data continues to grow and become more complex, organizations seek innovative solutions to manage their data effectively. Data mesh is one solution that provides a new approach to managing data in complex organizations. This practical guide offers step-by-step guidance on how to implement data mesh in your organization. In this book, Jean-Georges Perrin and Eric Broda focus on the key components of data mesh and provide practical advice supported by code. Data engineers, architects, and analysts will explore a simple and intuitive process for identifying key data mesh components and data products. You'll learn a consistent set of interfaces and access methods that make data products easy to consume. This approach ensures that your data products are easily accessible and the data mesh ecosystem is easy to navigate. This book helps you: Identify, define, and build data products that interoperate within an enterprise data mesh Build a data mesh fabric that binds data products together Build and deploy data products in a data mesh Establish the organizational structure to operate data products, data platforms, and data fabric Learn an innovative architecture that brings data products and data fabric together into the data mesh About the authors: Jean-Georges "JG" Perrin is a technology leader focusing on building innovative and modern data platforms. Eric Broda is a technology executive, practitioner, and founder of a boutique consulting firm that helps global enterprises realize value from data.

This comprehensive guide will teach you how to build an income statement semantic model, also known as the profit and loss (P&L) statement. Author Chris Barber— a business intelligence (BI) consultant, Microsoft MVP, and chartered accountant (ACMA, CGMA)—helps you master everything from designing conceptual models to building semantic models based on these designs. You will learn how to build a re-usable solution based on the trial balance and how to expand upon this to build enterprise-grade solutions. If you want to leverage the Microsoft BI platform to understand profit within your organization, this is the resource you need. What You Will Learn Modeling and the income statement: Learn what modelling the income statement entails, why it is important, and how income statements are constructed Calculating account balances: Learn how to optimally calculate account balances using a Star Schema Producing external income statement semantic models: Learn how to produce external income statement semantic models as they enable income statements to be analyzed from a range of perspectives and can be explored to reveal the underlying accounts and journal entries Producing internal income statement semantic models: Learn how to create multiple income statement layouts and further contextualize financial information by including percentages and non-financial information, and learn about the various security and self-service considerations Who This Book Is For Technical users (solution architects, Microsoft Fabric developers, Power BI developers) who require a comprehensive methodology for income statement semantic models because of the modeling complexities and knowledge needed of the accounting process; and finance (management accountants) who have hit the limits of Excel and have started using Power BI, but are unsure how income statement semantic models are built

Discover how to harness the full potential of Microsoft Power BI in "Microsoft Power BI Cookbook". Through its recipe-based structure, this book offers step-by-step guidance on mastering data integration, crafting impactful visualizations, and utilizing Power BI's latest features like Hybrid tables and enhanced scorecards. This edition equips you with the skills to transform raw data into actionable insights for your organization. What this Book will help me do Turn business data into actionable insights by utilizing Microsoft Data Fabric effectively. Create engaging and clear visualizations through Hybrid tables and advanced reporting techniques. Gain competence in managing real-time data accuracy and implementing dynamic analytics in Power BI. Ensure robust data compliance and governance integrated seamlessly into business reporting workflows. Leverage cutting-edge Power BI features to prepare for emerging trends in data intelligence. Author(s) Greg Deckler and None Powell, both esteemed professionals in the Power BI and data analytics domain, co-author this comprehensive guide. With decades of experience, they bring vast knowledge and practical skills to this work, presenting it in a structured and approachable manner. Both are dedicated to empowering learners of all levels to excel with Power BI. Who is it for? This book is ideal for professionals like data analysts, business intelligence developers, and IT specialists focused on reporting. It suits readers with a basic familiarity with Power BI, looking to deepen their understanding. If you aim to stay current with Power BI's most modern practices and features, this book will help you achieve that. Additionally, it supports those aiming to enhance business decision-making through better visualizations and advanced analysis.

The "MuleSoft Platform Architect's Guide" is your essential resource for mastering API-driven solutions using MuleSoft Anypoint Platform. This book enables you to design, deploy, and operate scalable, secure, and high-performance API architectures in enterprise settings while preparing for MuleSoft Platform Architect certification. What this Book will help me do Design robust API integration solutions using MuleSoft Anypoint Platform. Successfully deploy applications to CloudHub and Runtime Fabric environments. Monitor and operate APIs with advanced management tools. Implement scalable solutions aligned with business outcomes. Prepare confidently for the MuleSoft Platform Architect certification. Author(s) Jitendra Bafna is a Senior Solution Architect with years of experience optimizing MuleSoft implementations. Jim Andrews, a MuleSoft Evangelist, has dedicated his career to guiding others in achieving enterprise-ready API solutions. Together, they share practical knowledge, step-by-step guidance, and expertise in API and integration mastery. Who is it for? This book is perfect for IT architects and senior developers experienced in API development, especially those familiar with MuleSoft. It's tailored for professionals aiming to master Anypoint Platform or pursue MuleSoft Platform Architect certification. Readers should have basic experience with integration platforms and a willingness to explore advanced API design.

In this session Steve Sawyer will discuss a case study for how IBM Data Observability with Databand, collects metadata to build historical baselines, detect anomalies and triage alerts to remediate data quality issues for you data pipelines and warehouses. Additionally, he will provide a Product perspective on the technologies IBM is building to meet the data observability needs across the enterprise, and how it relates to our investments in AI and Data Fabric.

Summary

Data lakehouse architectures have been gaining significant adoption. To accelerate adoption in the enterprise Microsoft has created the Fabric platform, based on their OneLake architecture. In this episode Dipti Borkar shares her experiences working on the product team at Fabric and explains the various use cases for the Fabric service.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst is an end-to-end data lakehouse platform built on Trino, the query engine Apache Iceberg was designed for, with complete support for all table formats including Apache Iceberg, Hive, and Delta Lake. Trusted by teams of all sizes, including Comcast and Doordash. Want to see Starburst in action? Go to dataengineeringpodcast.com/starburst and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino. Your host is Tobias Macey and today I'm interviewing Dipti Borkar about her work on Microsoft Fabric and performing analytics on data withou

Interview

Introduction How did you get involved in the area of data management? Can you describe what Microsoft Fabric is and the story behind it? Data lakes in various forms have been gaining significant popularity as a unified interface to an organization's analytics. What are the motivating factors that you see for that trend? Microsoft has been investing heavily in open source in recent years, and the Fabric platform relies on several open components. What are the benefits of layering on top of existing technologies rather than building a fully custom solution?

What are the elements of Fabric that were engineered specifically for the service? What are the most interesting/complicated integration challenges?

How has your prior experience with Ahana and Presto informed your current work at Microsoft? AI plays a substantial role in the product. What are the benefits of embedding Copilot into the data engine?

What are the challenges in terms of safety and reliability?

What are the most interesting, innovative, or unexpected ways that you have seen the Fabric platform used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on data lakes generally, and Fabric specifically? When is Fabric the wrong choice? What do you have planned for the future of data lake analytics?

Contact Info

LinkedIn

Parting Question

From your perspective, what is the biggest gap in the tooling or technology for data management today?

Closing Announcements

Thank you for listening! Don't forget to check out our other shows. Podcast.init covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast helps you go from idea to production with machine learning. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.

Links

Microsoft Fabric Ahana episode DB2 Distributed Spark Presto Azure Data MAD Landscape

Podcast Episode ML Podcast Episode

Tableau dbt Medallion Architecture Microsoft Onelake ORC Parquet Avro Delta Lake Iceberg

Podcast Episode

Hudi

Podcast Episode

Hadoop PowerBI

Podcast Episode

Velox Gluten Apache XTable GraphQL Formula 1 McLaren

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA Sponsored By: Starburst:

This episode is brought to you by Starburst - an end-to-end data lakehouse platform for data engineers who are battling to build and scale high quality data pipelines on the data lake. Powered by T

Introductory session on Real-Time Analytics in Microsoft Fabric.