Let’s look at how to use DAX user defined functions, defined in the semantic model and used in reports. And what about the other UDF? Let’s also look at Fabric user data functions used in reports. Know the difference! Use them all to make your reports awesome.

Zoe Douglas, Principal Program Manager at Microsoft, will share how to “take the gloves off” and access the full analytics power of DAX with DAX queries—demystifying the process and showing just how easy it can be to get started with real-world examples you can apply to your own data. Whether you're new to Power BI or looking to deepen your analytical skills, this session is designed to empower you with practical strategies for turning raw data into compelling stories.

📅 Tuesday\, December 9th \| 🕢 19:30 CET \| 🌐 Online Event 🎤 Speaker: Brian Bønk (Microsoft MVP)

Our DP-600 preparation journey continues with a deep dive into the Microsoft Fabric Data Warehouse — one of the most crucial components for the exam and for building scalable analytics solutions. In this session, Brian Bønk, Fabric expert and MVP, will guide you through the complete lifecycle of working with a Data Warehouse in Fabric, from loading data to securing and monitoring it effectively.

📚 What we’ll cover in Session 2

1️⃣ Get Started with Data Warehouses

Understand how the Fabric Data Warehouse works, its architecture, and when to use it.

2️⃣ Load Data into a Data Warehouse

Learn the core ingestion paths, loading strategies, and performance considerations.

3️⃣ Query a Data Warehouse

Best practices for querying, performance optimization, and using SQL endpoints efficiently.

4️⃣ Monitor a Data Warehouse

Explore the tools and metrics needed to ensure reliability, performance, and cost control.

5️⃣ Secure a Data Warehouse

Learn essential security concepts: roles, permissions, row-level security, and workspace governance.

🎯 Who is this session for?

• Candidates preparing for DP-600
• Data engineers, analytics engineers & BI professionals
• Power BI users transitioning into Fabric engineering
• Anyone wanting a structured, expert-led understanding of Fabric DWH

🔔 Call to Action

👉 RSVP now to reserve your virtual seat! 👉 Share this event with colleagues or community members who are preparing for DP-600 or working with Fabric in production. Let’s grow our communities in Athens and Berlin with high-quality, expert-led learning sessions!

📅 Monday\, December 8th \| 🕒 Online Event \| 🎤 Speaker: Ioannis Philippides Join us for the first session of our 4-part DP-600 preparation series, hosted as part of Fabric Data Days! This session lays the foundation for understanding Microsoft Fabric and the engineering components that are essential for both real-world workloads and the DP-600 exam. Whether you're just starting your Fabric journey or preparing seriously for certification, this session will give you a clear, practical, and hands-on starting point.

📚 What we’ll cover in Session 1

1️⃣ Intro to Microsoft Fabric

A guided overview of the Fabric ecosystem, key experiences, and how everything fits together — in a way that finally makes sense.

2️⃣ Get Started with Lakehouses

How Lakehouses work in Fabric, why they matter, and how they serve as the backbone for analytics workloads.

3️⃣ Ingest Data with Dataflows Gen2

Hands-on concepts around building ingestion processes, managing transformations, and preparing data for further engineering.

4️⃣ Orchestrate Processes with Data Factory Pipelines

Build reliable, automated movement and transformation processes using Fabric Data Factory.

5️⃣ Use Spark in Fabric

Understand when, why, and how Spark fits into your end-to-end architecture — with practical examples relevant for the exam.

🎯 Who is this session for?

• Anyone preparing for DP-600
• Power BI professionals moving into Fabric
• Data engineers, analytics engineers, and advanced business analysts
• Anyone who wants a strong, structured entry into Fabric

🔔 Call to Action

👉 RSVP now to secure your spot! 👉 Share this event with colleagues, friends, and anyone looking to grow their Microsoft Fabric skills. Let’s build the strongest, most collaborative Fabric study community together!

🎓 Get your exam discount voucher Microsoft is offering limited DP-600 exam vouchers. Request yours here: 👉 https://community.fabric.microsoft.com/t5/custom/page/page-id/campaign_form?campaignID=Q0FNUEFJR05fMTc2MTE4OTgxNTQwMg==

When: Thursday 18th September 2025 Time: arrive for 5:45pm with talks starting from 6pm start prompt. Location: Ecosurety, 2nd Floor, 4 Colston Ave, Bristol, BS1 4ST

Complimentary drinks & pizza provided by our sponsor Method Resourcing Solutions.

Sit tight for venue details, we'll have them for you ASAP!

Session 1 - Fabricating Your Move: A Tailored Path from Power BI Premium to Fabric - David Mitchell As Microsoft Power BI evolves into Fabric, organizations gain powerful new capabilities—but also face the challenge of transitioning from Premium capacities. In this session, discover a streamlined, automated approach to migrating from Power BI Premium to Fabric capacities using tools developed within Semantic Link Labs. We’ll cover:

• Why Fabric capacities matter and the benefits they unlock
• How to plan and execute your migration with minimal disruption
• A live demonstration of the migration process in action

Learn how to future-proof your analytics environment with confidence, efficiency, and continuity for your users.

About David David Mitchell is a Cloud Solutions Architect at Microsoft, and is a key contributor to the migration solution. He brings over 10 years’ experience working in Information Analysis, Data Warehousing and Data Engineering.

Session 2 - Getting Started with Terraform and CI/CD for your Fabric Projects - Anna-Maria Wykes Microsoft Fabric is transforming how organizations build unified data platforms for analytics, data science, and business intelligence. Until recently, deploying and managing Fabric resources required manual effort or adhoc automation. That changed with the release of the Terraform provider for Microsoft Fabric last year, enabling teams to manage Fabric infrastructure as code.

In this session, you'll learn how to get started using Terraform to provision and manage Microsoft Fabric components — including workspaces, pipelines, dataflows, and more — in a repeatable and scalable way. We'll cover core Terraform concepts, walk through practical examples, and share best practices for integrating with Azure and CI/CD workflows. By the end of the session, you'll be equipped to bring automation, consistency, and governance to your Microsoft Fabric environments using Terraform.

About Anna Anna is a veteran software and data engineer as well as a Microsoft AI MVP, boasting over 18 years of experience. She has undertaken various projects, including real-time analytics with Scala and Kafka, constructing Data Lakes with Spark, and applying engineering to Data Science. Anna currently serves as a RSA (Resident Solution Architect) and consultant. With a genuine passion for data, she endeavours to bridge the gap between Software Development and Data Science. Anna's other areas of interest include DevOps (DataOps, MLOps, LLMOps), Agile methodologies, and organizing or participating in local Code Clubs.

We all look forward to seeing you there!!

Microsoft Fabric is transforming how organizations build unified data platforms for analytics, data science, and business intelligence. Until recently, deploying and managing Fabric resources required manual effort or ad hoc automation. That changed with the release of the Terraform provider for Microsoft Fabric last year, enabling teams to manage Fabric infrastructure as code. In this session, you'll learn how to get started using Terraform to provision and manage Microsoft Fabric components — including workspaces, pipelines, dataflows, and more — in a repeatable and scalable way. Aimed at data engineers, cloud architects, and DevOps professionals, we'll cover core Terraform concepts, walk through practical examples, and share best practices for integrating with Azure and CI/CD workflows. By the end of the session, you'll be equipped to bring automation, consistency, and governance to your Microsoft Fabric environments using Terraform.

_________________

MeetUp Agenda

18:00 - Online Chat & Networking 18:30 - Guest Speaker Talk 19:30 - Q&A 19:45 - Event Close

_________________

Data Governance in a Nutshell - From Strategy to Hands-On with Microsoft Purview and Fabric

Data is one of the most valuable assets in any organization - but understanding your data estate, its systems, dependencies, responsible roles, and supporting processes is what truly drives data literacy and trust. This is the realm of Data Governance, which bridges organizational frameworks and technical capabilities.

In this session, we’ll demystify what Data Governance really means. We’ll begin with the foundational concepts: Who are the key stakeholders? What processes and policies should be in place? Why is a governance strategy essential before diving into tools?

Once we’ve established the strategic and organizational groundwork, we’ll move to the technical side with a hands-on look at Microsoft Purview and Microsoft Fabric. You’ll learn how to:

• Discover and scan your data sources • Classify and catalog data for better understanding • Build a business glossary for a shared language • Enable secure sharing through data domains and data products • Trace data lineage and apply observability principles • Ensure data quality in a governed environment

By the end of this session, you'll have a clear picture of how to get started with Data Governance in your organization—from defining the vision and roles to implementing it with Microsoft's modern data tools.

_________________

MeetUp Member Resources In GitHub Here:

https://github.com/EastMidlandsData

Including introduction slides and our Code of Conduct.

_________________

--------------------------------------------------------------------------------- 📢

We’re excited to announce Global Fabric Day 2025, our first-ever annual, free, in-person Global Fabric Community conference!

This global conference is coordinated globally and hosted locally across multiple cities around the world during the weekend of May 31 – Jun 1, 2025.

In Romania, Timisoara is the city we're meeting in, with sessions both in person and also online, gathering the local Fabric Community for the first in-person event.

Check out all the participating cities this year: https://globalfabric.community and click on your city's user group link on the map to find the event page where you can register.

A special thank-you to Kim Manis for championing this community initiative and delivering the opening welcome message at our conference.

We’re also deeply grateful to all our community members, organizers, sponsors, and volunteers - this wouldn’t be possible without your passion, support and dedication.

GlobalFabricDay #GFD25 #MicrosoftFabric #PowerBI

===

--------------------------------------------------------------------------------- Agenda: ---------------------------------------------------------------------------------

• 10:00 AM - 10:30 AM

Welcome and Registration

• 10:30 AM - 10:50 AM

Session: Global Fabric Day 2025 Keynote Speaker: Kim Manis, Corporate VP of Product, Microsoft Fabric & Power BI

• 11:00 AM - 11:50 AM

Session: Accelerarea adopției în Power BI Fabric: de la documentație și testare automată la tooluri externe și UX/UI Speaker (remote): Alexandru Badiu, BI Consultant Teams Link: https://bit.ly/GFDRomania Details: În această sesiune vei descoperi cum să accelerezi adopția Power BI Fabric printr-un flux practic: de la crearea și organizarea documentației, la implementarea testelor automate cu DAX Query View, gestionarea modelului cu Tabular Editor/TMDL și integrarea tehnicilor de ghidare direct în raport (filtre de rapel, butoane Info, ghiduri de suport etc.). Vei vedea exemple concrete și vei primi un checklist cu bunele practici recomandate. La final, vei pleca cu un plan clar de acțiune pentru a oferi utilizatorilor o experiență UX/UI intuitivă și un suport complet, reducând solicitările de asistență și scalând eficient soluțiile BI.

• 12:00 PM - 12:50 PM

Session: Getting Started with the TMDL View Speaker: Cristian Prifti, Power BI Solution Artchitect Details: Power BI modelling has traditionally relied on graphical interfaces, but with the introduction of Tabular Model Definition Language (TMDL), developers now have a powerful scripting alternative. This session will introduce TMDL View in Power BI Desktop, explaining how it enables semantic modelling through code instead of the traditional UI without eternal tools support. We will cover:

• How to enable and use TMDL View in Power BI Desktop
• Scripting and modifying semantic model objects such as tables, measures, and relationships

• Enhancing efficiency with code-based modelling, bulk edits, and reusability This session is perfect for those new to TMDL or looking for a streamlined way to manage Power BI models.

• 13:00 PM - 14:00 PM Lunch Break

• 14:00 PM - 14:50 PM Session: Aggregate at light speed Speaker: Liviu Ieran, Microsoft Romania Details: We will discover together the benefits of using Columnstore Indexes in your Data Warehouse and how they can help you achieve better performance than regular or no indexes.

• 15:00 PM - 16:00 PM Session: Adaptive Cloud & Fabric Speaker: Catalin Gheorghiu, Solution Artchitect, Microsoft MVP Details: So, first what is this Adaptive Cloud and how do we get there? The first step is deploying Azure IoT Operations. Which Microsoft presents as being “Enabled by Azure Arc, Azure IoT Operations expands on our IoT portfolio with a composable set of services that help organizations onboard IT and OT assets, capture, evolve, and unify insights, and take actions to drive transformation at scale. OK, now what we get something database related running there, no? At the end of this session, you will know what if is that “database related”, how you get started, and what can do for you, on the edge (maybe). This is besides trying to get up to speed on the Adaptive Cloud vision. Of course it will be a sweet session.

• Location: ETA2U (Enterprise Meeting Room), Str. Gh. Dima, Nr. 1, Timisoara, Romania

_________________

MeetUp Agenda

18:00 - Online Chat & Networking 18:30 - Guest Speaker Talk 19:30 - Q&A 19:45 - Event Close

_________________

Data Governance in a Nutshell - From Strategy to Hands-On with Microsoft Purview and Fabric

Data is one of the most valuable assets in any organization - but understanding your data estate, its systems, dependencies, responsible roles, and supporting processes is what truly drives data literacy and trust. This is the realm of Data Governance, which bridges organizational frameworks and technical capabilities.

In this session, we’ll demystify what Data Governance really means. We’ll begin with the foundational concepts: Who are the key stakeholders? What processes and policies should be in place? Why is a governance strategy essential before diving into tools?

Once we’ve established the strategic and organizational groundwork, we’ll move to the technical side with a hands-on look at Microsoft Purview and Microsoft Fabric. You’ll learn how to:

• Discover and scan your data sources • Classify and catalog data for better understanding • Build a business glossary for a shared language • Enable secure sharing through data domains and data products • Trace data lineage and apply observability principles • Ensure data quality in a governed environment

By the end of this session, you'll have a clear picture of how to get started with Data Governance in your organization—from defining the vision and roles to implementing it with Microsoft's modern data tools.

_________________

MeetUp Member Resources In GitHub Here:

https://github.com/EastMidlandsData

Including introduction slides and our Code of Conduct.

_________________

The world of data is transforming to more and more real-time need and organizations demanding insights in faster and faster paces.

With the release of Real-Time Hub the Real-Time analytics service in Fabric takes its next step to gather all the streams and events in one service with a catalogue of data and ease of use.

In this introductory session I will walk you through the bits and bytes of the service and show you how easy it has become to ingest, categorize and work with your streaming data.

From all the newly announced sources, to categorizing the streams, from ingestion to discovery and from discovery to usage - this session will give you the needed overview to get started with the Real-Time Hub service in Fabric.

I will also touch base with how to implement the medallion architecture in real-time with sub-second processing from ingestion to gold layer and to use in analytics.

Join for a thrilling ride into the new Real-Time Hub service and when we end the session together with and round of QnA you will be ready to implement this in your own organization.

About the speaker

Brian Bønk

Brian has worked with SQL Server for more than two decades - varying projects on both size and complexity.

Now he's working to support the Data & Analytics team at Intellishore. MVP on Data Platform and Microsoft Recognized Fasttrack Solution Architect.

Brian loves data and is always trying to glue the business and tech together using his knowledge and experience. He is always open to meet new people and help them to get better tomorrow.

Welcome to the first episode of Get Certified: DP700 - Become a Fabric Data Engineer. Dive into the fundamentals of Microsoft Fabric and explore its core components, including OneLake, Lakehouses, and the Delta format. This session sets the stage for the series, covering how Fabric’s unified architecture streamlines data engineering workflows and how it compares to Synapse.

Welcome to the first episode of Get Certified: DP700 - Become a Fabric Data Engineer. Dive into the fundamentals of Microsoft Fabric and explore its core components, including OneLake, Lakehouses, and the Delta format. This session sets the stage for the series, covering how Fabric’s unified architecture streamlines data engineering workflows and how it compares to Synapse.

Welcome to the first episode of Get Certified: DP700 - Become a Fabric Data Engineer. Dive into the fundamentals of Microsoft Fabric and explore its core components, including OneLake, Lakehouses, and the Delta format. This session sets the stage for the series, covering how Fabric’s unified architecture streamlines data engineering workflows and how it compares to Synapse.

This event is a part of a series. To learn more, click here

Pre-requisites: Live sessions will be available in two time zones. If you are looking for a later session, visit this series**

Para sesiones en español, visita esta serie**

Para sessões em português, visite esta série**

Welcome to the first episode of Get Certified: DP700 - Become a Fabric Data Engineer. Dive into the fundamentals of Microsoft Fabric and explore its core components, including OneLake, Lakehouses, and the Delta format. This session sets the stage for the series, covering how Fabric’s unified architecture streamlines data engineering workflows and how it compares to Synapse.

This event is a part of a series. To learn more, click here

Pre-requisites: Live sessions will be available in two time zones. If you are looking for a later session, visit this series**

Para sesiones en español, visita esta serie**

Para sessões em português, visite esta série**

Welcome to the first episode of Get Certified: DP700 - Become a Fabric Data Engineer. Dive into the fundamentals of Microsoft Fabric and explore its core components, including OneLake, Lakehouses, and the Delta format. This session sets the stage for the series, covering how Fabric’s unified architecture streamlines data engineering workflows and how it compares to Synapse.

This event is a part of a series. To learn more, click here

Pre-requisites: Live sessions will be available in two time zones. If you are looking for a later session, visit this series**

Para sesiones en español, visita esta serie**

Para sessões em português, visite esta série**

Join us on January 29 for a chance to connect, learn, and get inspired by the latest and greatest in AI. Whether you're a pro at AI or just getting started, this event is all about building our community and shaping the future of AI development. Get ready for an evening packed with awesome sessions, discussions, and networking opportunities!

Let's dive into the exciting world of AI together!

Agenda

• 6:00 PM ET: Welcome and introduction
• 6:05 PM ET to 8:00 PM ET: Program
• 8:00 PM ET to 9:00 PM ET: Networking

Program Sessions

| Session Title | Session Description | Speaker | | :------------- |:-------------|:-------------| | Empowering Enterprises with LLM Agents: Assistants for Data-Driven Insights | Large Language Model (LLM) agents are emerging as powerful tools that can revolutionize how organizations interact with their data. This session will delve into the concept of LLM agents and their potential to transform internal operations by creating intelligent virtual assistants on top of your datawarehouse. We will explore how these agents can be leveraged to generate SQL query statements dynamically, enabling seamless access to data stored in various data warehouses such as Databricks, Microsoft Fabric. Attendees will gain insights into the architecture and implementation of these virtual assistants, and discover how they can enhance productivity, streamline decision-making processes, and unlock valuable insights from their data. | Sammy Deprez | | Simplifying AI in your .NET Application | Discover how to effortlessly integrate AI into your .NET applications with the Microsoft.Extensions.AI library. Learn to add chat features, embedding generation, and tool calling seamlessly. Plus, explore innovative techniques for real-time prompt management, allowing you to test and refine AI prompts live without stopping your application. Join us for a fast-paced session packed with practical insights and cutting-edge strategies to optimize your development workflow. | Maria Naggaga Nakanwagi | | AutoGen 0.4: A Programming Framework for Agentic AI reimagined | AutoGen is an open-source framework for building AI agent systems. It simplifies the creation of event-driven, distributed, scalable, and resilient agentic applications using multi-agent architectures. In this talk, Jack will provide an overview of the AutoGen framework and dig into 0.4, a reimagining of the framework. | Jack Gerrits | | AI in Action: Lessons and Success Stories | Join us for an engaging panel discussion featuring practitioners of artificial intelligence. Our panelists will share their firsthand experiences, challenges, and successes in implementing solutions across various industries. Learn about innovative use cases and discover best practices from experts who are at the forefront of AI technology.| Lisa Qu, Vivian Lei, Samir Mahmoud, Justin Trugman |

Python - what's all the fuss about and how do I get started

We've had many sessions recently about Python and Fabric highlighting some of the amazing possibilities that are available with the use of these two tools.

In this session, I want to strip it back and take you through step by step on how to get started so that you can play and build in confidence. We'll be utilising Chat GPT, we'll create notebooks with Power BI reports in them, we'll explore Sempy, we'll explore Pandas, we'll look at apis and we'll replicate Pagniated Reporting. By the end of the session, you'll feel confident enought to have a go and create your own notebooks.

Blog: Most of you probably know who I am as I'm the founder of the User Group but if you don't, see Chat GPT's beautiful summary of me:

Sue Bayes is a Microsoft Data Platform MVP with over five years of independent consulting experience as a data analyst and Power BI developer, helping businesses harness the full potential of their data. Certified as a Fabric Analytics Engineer, an Azure Enterprise Data Analyst and Power BI Data Analyst Associate, Sue is passionate about leveraging the latest tools in Microsoft Fabric to provide end-to-end data engineering solutions, with a strong emphasis on integrating Python into workflows.

Her expertise spans a wide range of reporting solutions, from project management and financial reporting to bespoke data cleansing, sentiment analysis, and advanced data pipelines within the Fabric environment. Sue's technical skills include Python, SQL, R, and C#, but her real passion lies in Power BI's M language and DAX, which she uses to create high-performance, dynamic reports.

Before becoming an independent consultant, Sue spent 15 years lecturing in Business and Computing, helping to shape the next generation of data professionals. Her deep understanding of data engineering, paired with a love for creating streamlined, actionable insights, has made her a sought-after expert in both the public and private sectors.

Outside of her professional life, Sue enjoys running, walking her dog, and embracing the outdoors whenever she can.

Detailed information and registration can be found here : https://dataminds.be/local-fabric-development-fabric-adoption-roadmap-ae-leuven/

Local Development of your Microsoft FAbric Lakehouse with Spark jobs

Speaker: Thibault Croonenborghs

Session: While Microsoft has significantly enhanced the developer experience for notebooks in Fabric, encouraging their use for daily workloads, Spark job definitions offer a compelling alternative for ETL code development. This session explores some advantages of using Spark job definitions over notebooks, with a focus on local development practices that optimize both efficiency and resource utilization. We will cover the setup of a local development environment, techniques for developing and testing Spark job definitions locally on your machine and how to integrate your jobs into robust CI/CD pipelines. Whether you’re looking to optimize resource usage, improve code quality, or streamline your deployment process, this presentation will provide valuable insights for data engineers and developers working with Microsoft Fabric.

Speaker: Thibault Croonenborghs Thibauld Croonenborghs started his studies at the Belgian Royal Military Academy and worked as an army officer for several years while simultaneously studying computer science.

He is now working as a data engineer for AE, but before he was working as a software engineer at TomTom improving the map that is being used for navigation. He has been a Python enthousiast and avid user for over 6 years.

Organizing Fabric Adoption

Speaker: Eduard Koekkoek

Session: A data or analytics Center of Excellence (COE) is an internal team of technical and business experts. The team actively assists others within the organization who are working with data. A federated COE comprises a shared services team (the core COE members) plus satellite members from each functional area or major business unit (by definition of Microsoft Learn). We will draft an approach of the roadmap for Fabric based on Eduard's 5 years of experience with Power BI Adoption at Royal Heijmans of The Netherlands.

As a data analytics engineer or as a team manager you can get confronted with the price of success of your data platform and self-service users creating dashboards from it. Get practical approaches to actively involve your management, key users, and other stakeholders. At Heijmans the Power BI community consists of more than 350 dashboard creators for about 200 daily readers. Learn from our lessons for cultivating data culture by organizing from mandate through support, training, and user services for Fabric.

Discover in this session the considerations for the different COE-approaches, a data platform architecture for enterprise- and streaming data, and organize means of communication and services with and for internal audiences.

Speaker: Eduard Koekkoek Chronically data-enthusiast, engineer and consultant on the Microsoft-stack (Azure, Fabric & Power Platform) and all things SQL or Spark, with some expertise on other data-solutions as well like TimeXtender, and Qlikview. I like sharing my stories on adoption strategies, and ideas for implementation on Fabric-Adoption on the road to data maturity. With the aim of improving organisations, processes and people I am data-experienced in the Benelux-geography since 2004 with Amsterdam Airport Schiphol/Royal Schiphol Group, Royal Heijmans (building contractor), Nabuurs Supply Chain Solutions, Micromedia Benelux (wholesale), Provelu (data-consulting), Danone Nutrition-products and Heart for Arts (agency and ICT-consultancy for NGO's). I am the proud holder of degrees in Logistics, and Business Economics, and Microsoft-certificates. My motto's are: Be yourself, and give the best you can. I like moving around in the Microsoft data- and performing arts- communities. I am an event organizer, speaker, and volunteer in the Benelux data-usergroups.

Summary

Data lakehouse architectures have been gaining significant adoption. To accelerate adoption in the enterprise Microsoft has created the Fabric platform, based on their OneLake architecture. In this episode Dipti Borkar shares her experiences working on the product team at Fabric and explains the various use cases for the Fabric service.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst is an end-to-end data lakehouse platform built on Trino, the query engine Apache Iceberg was designed for, with complete support for all table formats including Apache Iceberg, Hive, and Delta Lake. Trusted by teams of all sizes, including Comcast and Doordash. Want to see Starburst in action? Go to dataengineeringpodcast.com/starburst and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino. Your host is Tobias Macey and today I'm interviewing Dipti Borkar about her work on Microsoft Fabric and performing analytics on data withou

Interview

Introduction How did you get involved in the area of data management? Can you describe what Microsoft Fabric is and the story behind it? Data lakes in various forms have been gaining significant popularity as a unified interface to an organization's analytics. What are the motivating factors that you see for that trend? Microsoft has been investing heavily in open source in recent years, and the Fabric platform relies on several open components. What are the benefits of layering on top of existing technologies rather than building a fully custom solution?

What are the elements of Fabric that were engineered specifically for the service? What are the most interesting/complicated integration challenges?

How has your prior experience with Ahana and Presto informed your current work at Microsoft? AI plays a substantial role in the product. What are the benefits of embedding Copilot into the data engine?

What are the challenges in terms of safety and reliability?

What are the most interesting, innovative, or unexpected ways that you have seen the Fabric platform used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on data lakes generally, and Fabric specifically? When is Fabric the wrong choice? What do you have planned for the future of data lake analytics?

Contact Info

LinkedIn

Parting Question

From your perspective, what is the biggest gap in the tooling or technology for data management today?

Closing Announcements

Thank you for listening! Don't forget to check out our other shows. Podcast.init covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast helps you go from idea to production with machine learning. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.

Links

Microsoft Fabric Ahana episode DB2 Distributed Spark Presto Azure Data MAD Landscape

Podcast Episode ML Podcast Episode

Tableau dbt Medallion Architecture Microsoft Onelake ORC Parquet Avro Delta Lake Iceberg

Podcast Episode

Hudi

Podcast Episode

Hadoop PowerBI

Podcast Episode

Velox Gluten Apache XTable GraphQL Formula 1 McLaren

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA Sponsored By: Starburst:

This episode is brought to you by Starburst - an end-to-end data lakehouse platform for data engineers who are battling to build and scale high quality data pipelines on the data lake. Powered by T

* This a Data Celi event and you MUST register and pay 250 Euro on https://www.dataceili.ie/precons to attend

Data Celi is a community run event for the Irish Microsoft Data Platform. *

Microsoft Fabric is a new service that was announced during Microsoft Build 2023. Which has caused a lot of excitement in the Microsoft Data Platform community. ​ During this training day we will cover what Microsoft Fabric is and its place in the Microsoft Intelligent Data Platform. It is co-presented by two MVP’s who have looked into solutions for clients from different perspectives. One from a Power Bi background and one from a Data Engineering services background, so that you get a holistic view about the entire Microsoft Fabric environment. During the training day we cover:

• Confidently administer and govern the Microsoft Fabric environment (including licensing)
• Work with One Lake
• Realistically implement Lakehouses
• Work with Data warehouses
• Implement Data Factory for various scenarios
• Start with real-time analytics
• Get started with Data Science
• Start with real-time analytics
• Become Data Driven++ with Data Activator
• Use Power BI with some of the above experiences

A lot of these elements can be worked on in your own Microsoft Fabric environment. However, we will also be showing demos throughout the day. In addition, we will also explain its role in the Microsoft Data Platform with some practical demos. We will send out information on how you can look to create your own Fabric environment beforehand. Just be aware that some functionality might be limited. ​ In addition, it helps to have various applications installed locally. We will send out a list of applications before the training day. At the end of this training day, you leave with the knowledge you need to get started Microsoft Fabric.

Global Azure 2024 is here! Communities around the world are organizing localized hybrid events and live streams for everyone around the world to join and learn about Azure from the best-in-class community leaders.

Agenda for London

10:15 AM – Azure AI and Terraform - Exploring our Options by Jake Walsh & Nicholas Chang A look into the options for using Azure Terraform to work with Azure AI Services and Models. Covering an overview of Azure Terraform, Deployment Options, and the types of Azure AI Resources we can manage.

10:45 AM – Unlocking the Potential of Azure Blob Storage: A Guide to the lesser-known Features by John Kilmister In this talk, we will delve into the world of Azure Blob Storage and discover some of its lesser-known features and capabilities. We will explore the ways in which these features can enhance your use of Azure storage solutions. Whether you are a seasoned Azure user or new to the platform, this talk will provide valuable insights and actionable strategies for optimizing your use of Azure Blob Storage.

11:30 AM – Azure Verified Modules by Marcel Lupo In this session we take a closer look at (AVM) Azure Verified Modules. With AVM the aim is to provide a trusted platform for organisations to find and deploy solutions that can be easily integrate with their Azure-based services and infrastructure. They can span a wide range of solutions, using popular infrastructure as code (IaC) tools such as Bicep and Terraform.

12:00 PM – Unlocking the Power of Azure OpenAI: A Comprehensive Exploration and Demonstration by Alpa Buddhabhatti During this concise yet comprehensive session, we'll immerse ourselves in the realm of Azure OpenAI. We will delve into the fundamental principles, capabilities, and practical use cases of this cutting-edge AI technology. Whether you're new to Azure OpenAI or seeking a quick refresher, this session will serve as your gateway to unlock its full potential. In this demonstration, we will focus on the following technologies: 1. Azure OpenAI 2. Azure Data Factory 3. Microsoft Fabric 4. Azure SQL/ LakeHouse 5. Visual studio Code You'll gain valuable insights to effectively grasp and employ Azure OpenAI, paving the way for further exploration and integration into your projects.

12:30 PM – Microsoft Azure Service Bus (MaaS) Integration Simplified by Dias Manjaly The purpose of this foundation session is to demonstrate the usage of Azure Service Bus (MaaS). This session walkthrough the story of a bad customer experience and how it was resolved using Azure Service Bus. It has a short demo based on the story as well. It also covers cool features in Azure Service Bus, High Level Solution Design diagrams, How messages are sent and received using Azure Service Bus and an Azure Function, How to use Azure Service Bus Explorer, Service Bus Quotas and Dynamics 365 CRM Integration with Azure Service Bus, Comparison with other Azure messaging services. No prior knowledge is required to attend this foundation session. Audience will get a clear understanding of Azure Service Bus and it is a very useful skill to have as it a common use case.

1:15 PM – Code, Serverless Containers, and Cloud: Azure Container Apps for Developers by Jonah Andersson Do you want to build and develop serverless containers on a cloud platform such as Microsoft Azure? In this session, you will learn about the important technical concepts you need to know about building and developing Azure containers. The session explores serverlesss containers such Azure Container Apps and its key useful features for cloud development of microservices. Regardless of whether you are new, you will learn how to get started developing with and building your code with ACA and leverage the power of Azure with it.

1:45 PM – .NET on tiny IOT Meadow Boards by Clifford Agius Description It's a fact that DotNET has been around for 20 years and was once just the preserve of Windows, but has in recent years moved to Mobile using Mono and Xamarin, but still big powerful systems and processors. However, thanks to the work of WildernessLabs there is now the Meadow F7 board, a small form factor IOT board based around the Adafruit Feather. This means you can now write your Dot NET C#/F# code and truly run anywhere. The idea of this talk is to show that your existing DotNET skills can be used on IOT platforms without the scary Arduino version of C, you really do now have the skills to write code that will run anywhere. I am just a DotNET dev like you, I don’t work for WildernessLabs and this is not going to be a sales talk about the Meadow system, I backed the Kickstarter and I just enjoy playing with IOT. I want to show you that it’s scary and you too have the skills to diving in and get that LED blinking after that you can automate the home. We'll discuss: - The process of setting up the Meadow board and getting that first Hello World Blinky light going. - Brief explanation of the board and tooling - Demo a more complex system where the board is battery powered measuring sensor values and reporting this to an Azure function for processing.

2:15 PM – Would YOU Survive the Titanic?", with ML and .NET by Simon Painter Have you ever wondered whether YOU would survive the sinking of RMS Titanic or not? In this talk, we'll be using Visual Studio, C# and ML.NET to find out. Machine Learning is a hot topic these days. Business around the world are keen to utilise it; but doesn't it mean learning python?? In fact - no! These days it's not only possible to do ML in Visual Studio, but it's actually easy to produce high quality results with very little effort. Be warned though, there are ice bergs ahead...

3:00 PM – Umbraco Cloud's Journey with Kubernetes, Terraform & Azure DevOps by Dan Lister Dive into the inner workings of Umbraco Cloud's Platform transformation in this talk. See our hands-on approach to deploying and managing cloud infrastructure using Terraform. See how we use of rapid release cycles with Azure DevOps, minimising deployment times while maintaining quality and security. Explore our journey leveraging Azure Kubernetes Services for global scalability and high availability. Witness the integration of Azure API Management in streamlining interactions between our services, enforcing policies, and ensuring a secure, unified API gateway. Gain firsthand insights into how Umbraco Cloud's Platform Team innovatively wields these tools, offering practical knowledge for your own cloud environments.

3:30 PM – Azure Storage Has Never Been So Cool! by Anthony Mashford This session is about the Azure NetApp Files service. The Azure NetApp Files service delivers high-performance, low-latency file storage on-demand. We will cover the features of the service, such as snapshots, backups, replication and the new addition of the cool-access tiering feature. This new tiering feature is a great option to help drive down costs for Azure file storage whilst maintaining the great functional capabilities of Azure NetApp Files. The session will also include a live demonstration of the service and its features and Infrastructure-as-Code deployments.