talk-data.com

# Building a Unified Medallion Architecture in Microsoft Fabric: From Raw Data to AI-Ready Insights

As data platforms evolve toward unified and intelligent ecosystems, the Medallion Architecture has emerged as a foundational design pattern for building scalable, governed, and AI-ready analytics environments. Microsoft Fabric brings this vision to life by seamlessly integrating ingestion, transformation, governance, and visualization within a single, end-to-end platform.

In this session, Rajesh Vayyala will share an architecture-driven perspective on how organizations can conceptualize and operationalize a Medallion Architecture in Microsoft Fabric. We will focus on the architectural blueprint, governance principles, and organizational strategies required to establish Bronze, Silver, and Gold layers that ensure data consistency, lineage, and trust across data domains. 🏗️

Attendees will gain practical insights into designing resilient data estates, aligning data architecture with business goals, and enabling self-service analytics and AI through Fabric’s semantic layer and Power BI integration.

## Learning Objective

By the end of this session, attendees will be able to: 🔹 Recognize the key principles of the Medallion Architecture and understand how it supports scalable, governed, and AI-ready data ecosystems. 🔹 Map organizational data flows into Bronze, Silver, and Gold layers within Microsoft Fabric using an architecture-driven approach rather than deep technical coding. 🔹 Apply governance best practices for metadata management, lineage, and schema evolution using Fabric’s unified data foundation (OneLake, Data Activator, and Dataflows Gen2). 🔹 Align performance and reusability goals with enterprise data strategy—ensuring consistent, high-quality data delivery across analytics and AI initiatives. 🔹 Leverage the semantic layer and Power BI integration in Fabric to enable self-service analytics and informed decision-making across business domains.

Session Level: Intermediate

Presentation Title: Understanding file formats within the Fabric Lakehouse - John Miner

Description Microsoft Fabric has OneLake Storage at the center of all services. Storage is based upon existing Azure Data Lake Storage and can be accessed with tools that you are familiar with. Many different file formats have been used over time. Understanding the pros and cons of each file type is important.

I will be exploring several different datasets during the talk: zip files, stock data, earthquake data, NASA website data, and Fisher Iris dataset. Our exploration will start with the CSV format which is widely used. However, there are many other formats that you might encounter. Web services typically use a JSON document as the input and/or output to REST API calls. The Apache foundation projects came up with three different file formats: AVRO shines at data deserialization for RPC calls, ORC is suited for Hadoop processing, and PARQUET is optimized for Spark processing.

There are edge cases in which a file is in a special format. One can always use the TEXT format to parse out the data row by row. All of the above formats do not support the ACID properties of a database. That is why Databricks developed the DELTA file format which was opened source in 2019. This format is the foundation of most files in OneLake.

The Fabric Lakehouse is an implementation of Apache Spark which can have both managed and unmanaged Tables. I suggest using managed tables since they are support by both the SQL Analytics endpoint and the semantic model.

At the end of this talk, the developer will have a full understanding of all the file formats than can be managed by Fabric.

Join our Microsoft Fabric - UK User Group for our webinar in association with Onyx Data (https://onyxdata.co.uk).

This session we are excited to announce our host Microsoft MVP Leon Gordon and co-host Microsoft MVP Pragati Jain will be introducing Steve Campbell.

Join the Microsoft Fabric UK - User Group on -

Microsoft Community - https://community.powerbi.com/t5/Power-BI-UK-User-Group/gh-p/pbi_UK_usergroup LinkedIn - https://www.linkedin.com/groups/8909321/ MeetUp - https://www.meetup.com/microsoft-power-bi-uk/

Session Abstract -

Are you curious about Microsoft Fabric but find the technical jargon overwhelming? Join us for a session that breaks down Microsoft Fabric in a way that requires no prior coding knowledge or tech-savvy background.

This session will use simple language, relatable examples, and visual aids to explain the core concepts behind Microsoft Fabric. We'll cover concepts of OneLake, the foundation of Fabric. It will explain data lakes, delta, and parquet. All without delving into intricate technical details, covering terminology at an introductory level so you won’t get left behind

Whether you're a business professional seeking to make informed decisions about adopting Microsoft Fabric, a developer curious about the evolving tech landscape, or simply someone interested in demystifying complex technology, this session is tailored for you. Prepare to gain valuable insights that will demystify Microsoft Fabric and leave you feeling confident in your grasp of this transformative technology.

Biography

I’m Steve Campbell.

For my job, I run Sunny BI, a Microsoft data training and consulting firm focused on analytics and reporting.

Outside of this, I love to blog on PowerBI.tips, present at conferences and webinars, and co-run the Brew City Power Platform user group. It’s a fantastic community and I recommend being a part of it!

Social Media / Website Links

https://www.sunnybitraining.com/

We are excited to invite you to our upcoming community session featuring Pawel Potasinski,Senior Program Manager at Microsoft.

During this session, Pawel will delve into how Microsoft Fabric users can leverage OneLake as the storage foundation of the platform.

Don't miss this opportunity to enhance your understanding of Microsoft Fabric and the transformative power of OneLake!

"OneLake - the foundation of Microsoft Fabric, OneDrive for data and the data silo killer"

Session Details:

Title: OneLake - the foundation of Microsoft Fabric, OneDrive for data and the data silo killer

Date: 30th August 2023

Time:18:00 hrs CET

Speaker: Pawel Potasinski,Microsoft

Session Description:

One of the key pillars of Microsoft Fabric is that it’s lake centric and open. At the core of this pillar there’s OneLake - a single, unified storage system for all your data. Come to this session to learn how different teams working with Fabric can easily explore and discover data, re-use existing data products and work in a Data Mesh operating model without the need to maintain separate data lakes and keeping many copies of the same data.

Come to this session to learn how Microsoft Fabric users can benefit from OneLake - the storage foundation of the platform.

To attend the session, we kindly request you to mark yourself as "Going" on the event page and add it to your calendar. The link to join the session will be shared on the event page a few days before the session. Make sure to check the event page at the time of attending for the link.

Looking forward to your active participation and insightful discussions during this enriching session!

Please join us on 20th July 2023 to learn about OneLake in Microsoft Fabric.

What \~ Toronto Data Professionals Community (Virtual)

When \~ Thursday 20th July 2023

Agenda:

12:00 PM Networking and Introduction

12:15 PM Topic: OneLake - The Foundation of Microsoft Fabric, OneDrive for Data and the Data Silo Killer by Pawel Potasinski who is a Senior Program Manager at Microsoft, Member of the Microsoft Fabric Community team focused on community management and engagements.

1:30 PM End

Where: Online via Microsoft team

Session Details: The world is awash with data and it is up to data professionals to make sense of it all. Just a few weeks ago Microsoft announced the next generation of analytics at Build: Microsoft Fabric!

Fabric is a complete end-to-end analytics platform bringing together Azure Data Factory, Azure Synapse, and Power BI in a single location and more deeply integrated than ever. It’s built on a SaaS platform which helps deliver innovation more quickly and allowing users to get up and running in seconds. At the core Fabric provides a lake-centric and open data hub using the popular Delta format with built-in security, governance, and compliance throughout. At the edge, Fabric delivers flexibility for data scientists, data warehouse developers, and Power BI users to build and analyze using their unique skill sets.

One of the key pillars of Microsoft Fabric is that it’s lake centric and open. At the core of this pillar there’s OneLake - a single, unified storage system for all your data. Come to this session to learn how different teams working with Fabric can easily explore and discover data, re-use existing data products and work in a Data Mesh operating model without the need to maintain separate data lakes and keeping many copies of the same data.

Speaker Bio: Pawel Potasinski, Senior Program Manager at Microsoft, Member of the Microsoft Fabric Community team, focused on community management and engagements. In his professional career Pawel has always been associated with data engineering and analytics (SQL, BI, Big Data). Founder of the Polish SQL Server Users Group (PLSSUG), today known as Data Community Poland. Regular speaker at conferences, community events and user groups. Former Microsoft Most Valuable Professional (MVP).