The rise of AI has sparked excitement, disruption and a fair bit of existential dread among data engineers. With automation encroaching on traditional workflows and generative models promising to write code, build pipelines and even architect solutions, where does that leave the humble data engineer?

In this engaging and thought provoking session, Simon Whiteley - Databricks Champion, YouTube creator, and CTO of Advancing Analytics cuts through the hype to explore what survival really looks like in this new AI driven landscape. Drawing on real-world experience delivering production grade AI solutions with Databricks, Simon will unpack:

• The evolving role of the data engineer in AI-centric projects
• Which skills are becoming obsolete and which are more vital than ever
• How tools like Databricks are reshaping the engineering workflow
• Practical strategies to stay relevant, valuable, and ahead of the curve

Whether you're a seasoned engineer or just starting out, this session will leave you with a clearer view of the future, a few laughs, and a toolkit for thriving not just surviving in the age of AI.

We’re excited to be back at Big Data LDN this year—huge thanks to the organisers for hosting Databricks London once more!

Join us for an evening of insights, networking, and community with the Databricks Team and Advancing Analytics!

🎤 Agenda:

6:00 PM – 6:10 PM | Kickoff & Warm Welcome

Grab a drink, say hi, and get the lowdown on what’s coming up. We’ll set the scene for an evening of learning and laughs.

6:10 PM – 6:50 PM | The Metadata Marathon: How three projects are racing forward – Holly Smith (Staff Developer Advocate, Databricks)

With the enormous amount of discussion about open storage formats between nerds and even not-nerds, it can be hard to keep track of who’s doing what and how this actually makes any impact on day to day data projects.

Holly will take a closer look at the three big projects in this space; Delta, Hudi and Iceberg. They’re all trying to solve for similar data problems and have tackled the various challenges in different ways. Her talk will start with the very basics of how we got here, what the history is before diving deep into the underlying tech, their roadmaps, and their impacts on the data landscape as a whole.

6:50 PM – 7:10 PM | What’s New in Databricks & Databricks AI – Simon Whiteley & Gavi Regunath

Hot off the press! Simon and Gavi will walk you through the latest and greatest from Databricks, including shiny new AI features and platform updates you’ll want to try ASAP.

7:10 PM onwards | Q&A Panel + Networking

Your chance to ask the experts anything—then stick around for drinks, snacks, and some good old-fashioned data geekery.

Details

Welcome to the Databricks Scotland Meetup in Glasgow! Whether you're a seasoned Databricks user, just dipping your toes, or simply intrigued about what the platform can offer, this meetup promises to deliver insights, best practices, and opportunities for networking with like-minded individuals. We hope to see you there!

Come join us for a special Databricks Scotland meetup, 11th September at 5.00pm for an evening of Databricks to get you warmed up for DATA:Scotland on Friday 12th!

17:00 - 17:15 Welcome & Networking 17:15 - 18:00 Simon Whiteley - A Lightning Data & AI Summit Recap 18:00 - 18:30 Grab some food & drink! 18:30 - 19:00 Simon Whiteley - Harnessing Conversational BI with AI/BI Genie and Agent Bricks

A Lightning Data + AI Summit Recap We saw a whirlwind of announcements at the Databricks Data + AI Summit this year. If you've not yet had a chance to dig through the noise, this session distils the biggest reveals, hottest innovations, and most eyebrow-raising features into a fast-paced, insight-packed recap. Harnessing Conversational BI with AI/BI Genie and Agent Bricks We’ll then roll up our sleeves and dive into a live technical demo of Genie and Agent Bricks, showing how these next-generation capabilities can supercharge end-user productivity, accelerate insight, and make conversational BI a more natural part of your data workflows. Expect some live building, practical use-cases, and a dash of AI-powered mischief.

About Us If you need to contact us for any reason you can email us at: [email protected] We are sponsored by:

• Advancing Analytics

Thank you to our hosts CGI for their kind support.

Welcome to the London Databricks Meetup! Join us for an exciting evening of insights, expert sessions, and networking.

In July's edition, we have CTO & Co-founder of Advancing Analytics Simon Whiteley breaking down the announcements from Data + AI Summit. This is an event you would not want to miss!

Event Schedule: 18:00 - 18:15: Arrival & Networking Mingle with fellow data enthusiasts and professionals. 18:15 - 18:30: Opening Remarks & Introductions Get to know your fellow attendees and the organizers. 18:30 - 19:15: Data + AI Summit Announcements Breakdown -- Simon Whiteley 19:15 - 19:45: AMA & General Discussion 19:45 onwards: Pizza, Drinks & Networking Enjoy some delicious pizza and beverages while networking with peers. Date and Location: Date: 18th July Venue: IDEA London Join us for a fantastic evening of learning and networking at the London Databricks meetup!

_________________

MeetUp Agenda

18:00 - Arrive. Networking 18:30 - Guest Speaker Talk 19:30 - Pizza. Networking 20:30 - Event Close

_________________

An Introduction to Azure Databricks

Azure Databricks is a cloud-based platform that provides a unified environment for data engineering, data science, and analytics. It is based on Apache Spark, an open-source framework for distributed computing, and offers various features and benefits that make it a powerful and scalable solution for data-driven applications. In this talk, we will introduce the main concepts and components of Azure Databricks and show how it can help you accelerate your data projects and deliver business value.

Some of the topics that we will cover in this talk are:

• What is Azure Databricks and how does it differ from other cloud services?

• How to create and manage Azure Databricks clusters and workspaces?

• How to use Azure Databricks notebooks and tools for interactive data exploration and collaboration?

• What is Delta Lake and how does it enable reliable and performant data storage and processing?

• What is Unity Catalog and how does it provide data governance and security across your data assets?

• What are Delta Live Tables and how do they enable real-time data ingestion and transformation?

• How to use Azure Databricks workflows and pipelines to orchestrate and automate your data workflows?

• How to use Azure Databricks to run Spark streaming jobs and handle streaming data sources?

By the end of this talk, you will have a better understanding of Azure Databricks and its capabilities, and you will be able to start using it for your own data projects. You will also learn some best practices and tips for working with Azure Databricks and optimizing its performance and cost. Whether you are new to Azure Databricks or already have some experience with it, this talk will provide you with valuable insights and knowledge that you can apply to your data scenarios.

_________________

Venue Website

https://www.castlerockbrewery.co.uk/pubs/vat-and-fiddle

_________________

MeetUp Member Resources In GitHub Here:

https://github.com/EastMidlandsData

Including introduction slides and our Code of Conduct.

_________________

Welcome to the Inaugural London Databricks Meetup! Join us for an exciting evening of insights, expert sessions, and networking. Event Schedule:

18:00 - 18:15: Arrival & Networking Mingle with fellow data enthusiasts and professionals.

18:15 - 18:30: Opening Remarks & Introductions Get to know your fellow attendees and the organizers.

18:30 - 19:00: Exploring Key Updates on the Databricks Platform Discover the latest advancements in the Databricks technology -- Simon Whiteley

19:00 - 19:30: Photon for dummies -- Holly Smith

19:30 onwards: Pizza, Drinks & Networking Enjoy some delicious pizza and beverages while networking with peers.

Date and Location: Date: 7th December Venue: IDEALondon

Join us for a fantastic evening of learning and networking at the launch of the London Databricks meetup!

We are excited to welcome Simon Whiteley and Terry McCann for our November meetup.

Fabric Notebooks for the Power BI Professional

Microsoft Fabric combines the worlds of Power BI and Data Platforms, and there is a huge array of powerful tools at your now at your fingertips - but if you've never written python, how do you even begin to use the Notebook experience? What is a DataFrame and how do you use them? How can you avoid Python and use SQL instead?

In this session, we'll take a look at how the Spark engine is surprisingly similar to Power Query. We'll load some files into our OneLake, write some PySpark to read the data, transform it a little, and save it as a Lakehouse table that we can then use datasets, warehouses and more. We’ll look at some of the tips and tricks for using a little bit of Spark to speed up your data curation, as well as some of the neat new tools that come in the Fabric experience.

Finally, we'll discuss using notebooks as part of your end-to-end data process, and answer your Fabric design questions!

Simon Whiteley

Data Platform MVP. Databricks Beacon. Cloud Architect, Nerd

Director of Engineering for Advancing Analytics Ltd, Microsoft Data Platform MVP and Databricks Beacon. Simon is a seasoned solution architect & technical lead with well over a decade of Microsoft Analytics experience, who spends an inordinate amount of time running the Advancing Spark YouTube series. A deep techie with a focus on emerging cloud technologies and applying "big data" thinking to traditional analytics problems, Simon also has a passion for bringing it back to the high level and making sense of the bigger picture. When not tinkering with tech, Simon is a death-dodging London cyclist, a sampler of craft beers, an avid chef and a generally nerdy person.

Terry McCann

Microsoft MVP & CEO

Microsoft MVP. CEO & Owner of Advancing Analytics. A consultancy who helps businesses advance their analytics and better understand their data. Frequent speaker at events across the globe. Organiser of the Global AI Bootcamp London, Data Science user group in Exeter, UK and previous co-organiser of SQL Saturday Exeter, UK.

OPEN DATA CHALLENGE: Before our main session we are bringing back the Open Data Challenge. Your opportunity to showcase your talent to the audience. This month's challenge is Women's Football World Cup 2023 data from StatsBomb.

https://statsbomb.com/news/statsbomb-release-free-2023-womens-world-cup-data/

You'll have the opportunity to present a short (max 5 minute) presentation to show road report and and other aspects of Microsoft Fabric used in creating your solution.

Please bring a laptop to present, or contact us in advance.

VENUE: We will meet at the Nigel Frank Offices at 60 Great Tower Street. If you've not been there before, it's just a little further up the street from Brewdog. Google Maps link to help you find the building: https://goo.gl/maps/GRZWr5LfScZhApu6A