Search – talk-data.com

Title & Speakers	Event
Microsoft Fabric and its place in the Microsoft Intelligent Data Platform 2024-06-13 · 08:00 * This a Data Celi event and you MUST register and pay 250 Euro on https://www.dataceili.ie/precons to attend Data Celi is a community run event for the Irish Microsoft Data Platform. * Microsoft Fabric is a new service that was announced during Microsoft Build 2023. Which has caused a lot of excitement in the Microsoft Data Platform community. During this training day we will cover what Microsoft Fabric is and its place in the Microsoft Intelligent Data Platform. It is co-presented by two MVP’s who have looked into solutions for clients from different perspectives. One from a Power Bi background and one from a Data Engineering services background, so that you get a holistic view about the entire Microsoft Fabric environment. During the training day we cover: Confidently administer and govern the Microsoft Fabric environment (including licensing) Work with One Lake Realistically implement Lakehouses Work with Data warehouses Implement Data Factory for various scenarios Start with real-time analytics Get started with Data Science Start with real-time analytics Become Data Driven++ with Data Activator Use Power BI with some of the above experiences A lot of these elements can be worked on in your own Microsoft Fabric environment. However, we will also be showing demos throughout the day. In addition, we will also explain its role in the Microsoft Data Platform with some practical demos. We will send out information on how you can look to create your own Fabric environment beforehand. Just be aware that some functionality might be limited. In addition, it helps to have various applications installed locally. We will send out a list of applications before the training day. At the end of this training day, you leave with the knowledge you need to get started Microsoft Fabric.	Microsoft Fabric and its place in the Microsoft Intelligent Data Platform
Delta Lake Tables and ADF Managed Airflow 2024-01-18 · 18:00 Hosted by Origin Workspace and Bud, sponsored by Solar Battery Scheduler Location: 40 Berkeley Square Bristol, BS8 1HP AGENDA 18.00 – 18:30 Meet & Greet Grab a soft drink, chat and meet people. Feel free to bring your own alcoholic drink. -------------- 18:30 - 19:15 Delta Lake Tables 101 by Kamil Nowinski There are more and more file formats nowadays: Parquet format is not the best shiny star any longer. Now, the Delta Lake takes the prim. Why people do confuse it with Parquet and always talk about files in this case? In this session, we'll take a look at the evolution of ETL into ELT and its storage aspect, which explain why it is "a must" for modern data warehouse solutions and how is it related Delta Lake technology in cloud environments like Databricks or Synapse Analytics. Finally, we'll check what's Delta-Parquet creature presented in Microsoft Fabric OneLake recently. We will see also what data layers (stages) are commonly set up and why they make sense. -------------- 19:15 - 19:45 Pizza and Networking -------------- 19:45 - 20:30 Better ETL with Managed Airflow in ADF by Niall Langley Building complex data workflows using Azure Data Factory can get a little clunky - as you orchestration needs get more complex you hit limitations like not being able to nest loops or conditionals, running simple Python, bash or PowerShell scripts is difficult, and costs can grow quickly as you are charged per task execution. Recently another option become available, Managed Airflow in ADF. Apace Airflow is a code-centric open-source platform for developing, scheduling and monitoring batch-based data workflows, built using the python language Data Engineers know and love. But until Managed Airflow, getting it working in Azure was a complex task for customers more used to PaaS services such as ADF, Databricks and Fabric. It is also an important ETL orchestrator on AWS and GCP, so cross cloud compatibility becomes simpler to achieve. In this session we’ll look at what Airflow is, how it’s different from ADF, and what advantages Managed Airflow in ADF gives us. We talk about the idea of a DAG for building the workflow, and then work through some demos to show just how easy it is to use Python to write an Airflow DAG’s and import them into the Managed Airflow Environment as pipelines. We then dive into the excellent monitoring UI and find out just how easy is it to trigger a pipeline, view it to see the dependencies between tasks, and monitor runs. By the end of the session attendees will have a good understanding of what Airflow is, when to use it, and how it fits into the Azure Data Platform. -------------- 20:30 - Pub -------------- About Origin Workspace - https://originworkspace.co.uk/ Origin Workspace is designed to meet the changing needs of today's entrepreneurs, small and growing businesses, ambitious consultants, freelancers and remote workers. We know that your workplace needs to be more than a desk with internet connection. You need networking opportunities as well as IT support, a place to hold meetings. And all in a supportive environment that inspires you to connect and grow, and in a convenient location next to Bristol's Clifton Triangle. About Bud - https://bud.co.uk/ Bud is a training management platform, designed to streamline the processes involved in delivering apprenticeships and skills training. Built around workflows that require key items of data to be captured only once, Bud’s customers benefit from reduced administrative time and improved data accuracy. About Solar Battery Scheduler - https://www.solarbatteryscheduler.com/ Make the most of your Solar Power Generation with Eco Intelligent Charging of your Home Battery. ---- Photos We ask that you do NOT take photos at this meetup. We will invite people to be included in a group photo/s during the event. Speakers will let you know if it's okay to photograph their presentation (excluding other attendees). You may see organisers taking photos during the talks. These will be of speakers, if they have agreed to this, and will not include faces of attendees.	Delta Lake Tables and ADF Managed Airflow

* This a Data Celi event and you MUST register and pay 250 Euro on https://www.dataceili.ie/precons to attend

Data Celi is a community run event for the Irish Microsoft Data Platform. *

Microsoft Fabric is a new service that was announced during Microsoft Build 2023. Which has caused a lot of excitement in the Microsoft Data Platform community. During this training day we will cover what Microsoft Fabric is and its place in the Microsoft Intelligent Data Platform. It is co-presented by two MVP’s who have looked into solutions for clients from different perspectives. One from a Power Bi background and one from a Data Engineering services background, so that you get a holistic view about the entire Microsoft Fabric environment. During the training day we cover:

Confidently administer and govern the Microsoft Fabric environment (including licensing)
Work with One Lake
Realistically implement Lakehouses
Work with Data warehouses
Implement Data Factory for various scenarios
Start with real-time analytics
Get started with Data Science
Start with real-time analytics
Become Data Driven++ with Data Activator
Use Power BI with some of the above experiences

A lot of these elements can be worked on in your own Microsoft Fabric environment. However, we will also be showing demos throughout the day. In addition, we will also explain its role in the Microsoft Data Platform with some practical demos. We will send out information on how you can look to create your own Fabric environment beforehand. Just be aware that some functionality might be limited. In addition, it helps to have various applications installed locally. We will send out a list of applications before the training day. At the end of this training day, you leave with the knowledge you need to get started Microsoft Fabric.

Hosted by Origin Workspace and Bud, sponsored by Solar Battery Scheduler

Location: 40 Berkeley Square Bristol, BS8 1HP

AGENDA

18.00 – 18:30 Meet & Greet Grab a soft drink, chat and meet people. Feel free to bring your own alcoholic drink.

--------------

18:30 - 19:15 Delta Lake Tables 101 by Kamil Nowinski

There are more and more file formats nowadays: Parquet format is not the best shiny star any longer. Now, the Delta Lake takes the prim. Why people do confuse it with Parquet and always talk about files in this case?

In this session, we'll take a look at the evolution of ETL into ELT and its storage aspect, which explain why it is "a must" for modern data warehouse solutions and how is it related Delta Lake technology in cloud environments like Databricks or Synapse Analytics. Finally, we'll check what's Delta-Parquet creature presented in Microsoft Fabric OneLake recently. We will see also what data layers (stages) are commonly set up and why they make sense.

--------------

19:15 - 19:45 Pizza and Networking

--------------

19:45 - 20:30 Better ETL with Managed Airflow in ADF by Niall Langley

Building complex data workflows using Azure Data Factory can get a little clunky - as you orchestration needs get more complex you hit limitations like not being able to nest loops or conditionals, running simple Python, bash or PowerShell scripts is difficult, and costs can grow quickly as you are charged per task execution. Recently another option become available, Managed Airflow in ADF.

Apace Airflow is a code-centric open-source platform for developing, scheduling and monitoring batch-based data workflows, built using the python language Data Engineers know and love. But until Managed Airflow, getting it working in Azure was a complex task for customers more used to PaaS services such as ADF, Databricks and Fabric. It is also an important ETL orchestrator on AWS and GCP, so cross cloud compatibility becomes simpler to achieve.

In this session we’ll look at what Airflow is, how it’s different from ADF, and what advantages Managed Airflow in ADF gives us. We talk about the idea of a DAG for building the workflow, and then work through some demos to show just how easy it is to use Python to write an Airflow DAG’s and import them into the Managed Airflow Environment as pipelines. We then dive into the excellent monitoring UI and find out just how easy is it to trigger a pipeline, view it to see the dependencies between tasks, and monitor runs.

By the end of the session attendees will have a good understanding of what Airflow is, when to use it, and how it fits into the Azure Data Platform.

--------------

20:30 - Pub

--------------

About Origin Workspace - https://originworkspace.co.uk/

Origin Workspace is designed to meet the changing needs of today's entrepreneurs, small and growing businesses, ambitious consultants, freelancers and remote workers. We know that your workplace needs to be more than a desk with internet connection. You need networking opportunities as well as IT support, a place to hold meetings. And all in a supportive environment that inspires you to connect and grow, and in a convenient location next to Bristol's Clifton Triangle.

About Bud - https://bud.co.uk/

Bud is a training management platform, designed to streamline the processes involved in delivering apprenticeships and skills training. Built around workflows that require key items of data to be captured only once, Bud’s customers benefit from reduced administrative time and improved data accuracy.

About Solar Battery Scheduler - https://www.solarbatteryscheduler.com/

Make the most of your Solar Power Generation with Eco Intelligent Charging of your Home Battery.

---- Photos We ask that you do NOT take photos at this meetup.

We will invite people to be included in a group photo/s during the event. Speakers will let you know if it's okay to photograph their presentation (excluding other attendees).

You may see organisers taking photos during the talks. These will be of speakers, if they have agreed to this, and will not include faces of attendees.

talk-data.com

People (1 result)

Activities & events