Summary In this episode of the Data Engineering Podcast Professor Paul Groth, from the University of Amsterdam, talks about his research on knowledge graphs and data engineering. Paul shares his background in AI and data management, discussing the evolution of data provenance and lineage, as well as the challenges of data integration. He explores the impact of large language models (LLMs) on data engineering, highlighting their potential to simplify knowledge graph construction and enhance data integration. The conversation covers the evolving landscape of data architectures, managing semantics and access control, and the interplay between industry and academia in advancing data engineering practices, with Paul also sharing insights into his work with the intelligent data engineering lab and the importance of human-AI collaboration in data engineering pipelines.

Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data managementData migrations are brutal. They drag on for months—sometimes years—burning through resources and crushing team morale. Datafold's AI-powered Migration Agent changes all that. Their unique combination of AI code translation and automated data validation has helped companies complete migrations up to 10 times faster than manual approaches. And they're so confident in their solution, they'll actually guarantee your timeline in writing. Ready to turn your year-long migration into weeks? Visit dataengineeringpodcast.com/datafold today for the details.Your host is Tobias Macey and today I'm interviewing Paul Groth about his research on knowledge graphs and data engineeringInterview IntroductionHow did you get involved in the area of data management?Can you start by describing the focus and scope of your academic efforts?Given your focus on data management for machine learning as part of the INDELab, what are some of the developing trends that practitioners should be aware of?ML architectures / systems changing (matteo interlandi) GPUs for data mangementYou have spent a large portion of your career working with knowledge graphs, which have largely been a niche area until recently. What are some of the notable changes in the knowledge graph ecosystem that have resulted from the introduction of LLMs?What are some of the other ways that you are seeing LLMs change the methods of data engineering?There are numerous vague and anecdotal references to the power of LLMs to unlock value from unstructured data. What are some of the realitites that you are seeing in your research?A majority of the conversations in this podcast are focused on data engineering in the context of a business organization. What are some of the ways that management of research data is disjoint from the methods and constraints that are present in business contexts?What are the most interesting, innovative, or unexpected ways that you have seen LLM used in data management?What are the most interesting, unexpected, or challenging lessons that you have learned while working on data engineering research?What do you have planned for the future of your research in the context of data engineering, knowledge graphs, and AI?Contact Info WebsiteemailParting Question From your perspective, what is the biggest gap in the tooling or technology for data management today?Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.init covers the Python language, its community, and the innovative ways it is being used. The AI Engineering Podcast is your guide to the fast-moving world of building AI systems.Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.Links INDELabData ProvenanceElsevierSIGMOD 2025Digital TwinKnowledge GraphWikiDataKuzuDBPodcast Episodedata.worldPodcast EpisodeGraphRAGSPARQLSemantic WebGQL == Graph Query LanguageCypherAmazon NeptuneRDF == Resource Description FrameworkSwellDBFlockMTLDuckDBPodcast EpisodeMatteo InterlandiPaolo PapottiNeuromorphic ComputingPoint CloudsLongform.aiBASIL DBThe intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

Data is your business. Have you unlocked its full potential? If you read nothing else on data strategy, read this book. We've combed through hundreds of Harvard Business Review articles and selected the most important ones to help you maximize your analytics capabilities; harness the power of data, algorithms, and AI; and gain competitive advantage in our hyperconnected world. This book will inspire you to: Reap the rewards of digital transformation Make better data-driven decisions Design breakout products that generate profitable insights Address vulnerabilities to cyberattacks and data breaches Reskill your workforce and build a culture of continuous learning Win with personalized customer experiences at scale This collection of articles includes "What's Your Data Strategy?," by Leandro DalleMule and Thomas H. Davenport; "Democratizing Transformation," by Marco Iansiti and Satya Nadella; "Why Companies Should Consolidate Tech Roles in the C-Suite," by Thomas H. Davenport, John Spens, and Saurabh Gupta; "Developing a Digital Mindset," by Tsedal Neeley and Paul Leonardi; "What Does It Actually Take to Build a Data-Driven Culture?," by Mai B. AlOwaish and Thomas C. Redman; "When Data Creates Competitive Advantage," by Andrei Hagiu and Julian Wright; "Building an Insights Engine," by Frank van den Driest, Stan Sthanunathan, and Keith Weed; "Personalization Done Right," by Mark Abraham and David C. Edelman; "Ensure High-Quality Data Powers Your AI," by Thomas C. Redman; "The Ethics of Managing People's Data," by Michael Segalla and Dominique Rouzies; "Where Data-Driven Decision-Making Can Go Wrong," by Michael Luca and Amy C. Edmondson; "Sizing Up Your Cyberrisks," by Thomas J. Parenty and Jack J. Domet; "A Better Way to Put Your Data to Work," Veeral Desai, Tim Fountaine, and Kayvaun Rowshankish; and "Heavy Machinery Meets AI," by Vijay Govindarajan and Venkat Venkatraman. HBR's 10 Must Reads are definitive collections of classic ideas, practical advice, and essential thinking from the pages of Harvard Business Review. Exploring topics like disruptive innovation, emotional intelligence, and new technology in our ever-evolving world, these books empower any leader to make bold decisions and inspire others.

The modern data stack has improved the lives of data teams everywhere. But has it helped the rest of the business? In this talk, we’ll discuss the business teams’ perspective. Are they actually getting value from the modern data stack? How does help it them do their jobs better? And why do data teams keep questioning if we’re “adding value” with our powerful new tools? Attendees will gain perspective on their data ‘customers’ and learn ideas on how to deliver tangible business value.

Speaker: Paul Blankley CTO Zenlytic

Read the blog to learn about the latest dbt Cloud features announced at Coalesce, designed to help organizations embrace analytics best practices at scale https://www.getdbt.com/blog/coalesce-2024-product-announcements

Please note that this event takes place in person on 5 June 2024 at 20:00 BST (London time).

Paul Bilokon, Oleksandr Bilokon, and Nataliya Bilokon have recently published, with general public audience in mind,

A Brief History of Artificial Intelligence: https://www.amazon.co.uk/Brief-History-Artificial-Intelligence-Thalesians/dp/B0CV898K4M/ (February 2024)

In addition, Oleksandr Bilokon has published

Artificial Intelligence in Shipping and Logistics: https://www.amazon.co.uk/Artificial-Intelligence-Logistics-Thalesians-Technology/dp/B0CX8JYY54/ (March 2024)

We'll be organizing a public lecture followed by a book signing.

Artificial Intelligence (AI) is transforming business, science, technology, and medicine. Achievements that one could only dream of in the 20th century have materialized in the 21st: AlphaGo has beaten Lee Sedol at the game of Go, probably the hardest game that humanity has invented; intelligent machines created by an army of quants (mathematicians and computer scientists) have been gradually displacing human traders on Wall Street and in the City of London; protein folding, a 50-year-old grand challenge in biology, has essentially been solved by AlphaFold, with unprecedented implications for areas like drug design and environmental sustainability.

Even before the advent of AI, technological advances such as social media, Deliveroo, and Uber have transformed the lives of numerous people, sometimes creating new, sometimes eliminating existing jobs. The emergence of large language models (LLMs), such as GPT behind the AI system ChatGPT, means that AI is now almost universally available and will transform your life – for worse or better. It is therefore essential to be informed about the AI well before its universal adoption.

The purpose of this book is to make AI accessible to literally everyone: whether you are currently at primary or secondary school, preparing to join or already at college or university, working in the academe at masters or PhD level or beyond, doing an AI-related – or a totally unrelated – job, running a business, serving a religious organisation, a charity, a country, and/or a government. Irrespective of whether you are not in the work pool, unemployed, employed, self-employed, retired, if you are interested in what dangers and opportunities are presented to you personally, and other people around you, by AI, this book is for you. If you are a busy executive or, more generally, decision maker, this short book (less than 100 pages long) will serve to update you on the current state-of-the-art in AI.

The primary purpose of this book is to inform. The secondary, to entertain and inspire. This is not an academic treatise, so there are (almost) no formulae and no citations. There are some anecdotes that, we hope, you will find intriguing. Granted, you may not have heard of all the references (just as we would not have heard of all the references that you have heard of!), but you have an unfair advantage: access to search engines, such as Google.

Read this book, and you will learn why AI has been created, what it is, how it works and how to use it, as well as what will happen (and what won't happen) if you start adopting AI today.

Dear all, We are happy to announce a joint session by two of the greatest and bestest Meetup groups in Berlin - the Berlin DevOps and the Berlin AWS User Group. Registering on both is not necessary 😉

The State of Infrastructure as Code

The idea is to have insightful discussions with the audience about the current Infrastructure as Code landscape and allow practitioners to share their experiences and discuss with others.

If you are keen to tell your story, your successes and failures to others, please fill in this form to register with your tooling of choice - https://forms.gle/T5zUQRtfFgnwCVey8

In the second part we will have (remote) guests from various projects, giving food for thought and visions for the IaC landscape in the months ahead.This event will happen in-person, kindly sponsored by HeyJobs! as well as remotely.Please only RSVP if you intend to participate in person in Kreuzberg**. The event is published in both Meetup groups - Berlin DevOps and AWS User Group - please RSVP in only one of them if you happen to be a member of both groups.

Details for remote access will be shared in time with the community.Please note, we will take pictures during the event as well as record the sessions.** ------------------------------------------------------------------------------------------------ The evening 18:30 - Warming up and networking

18:45 - Welcome talk by HeyJobs!

19:00 - 20:00 - Fishbowl discussion

Practitioners share their opinionated experiences with landscapes and toolings while the audience is invited to join the discussions. An introduction to the format happens at the evening, pre-read here: https://en.wikipedia.org/wiki/Fishbowl_(conversation)

20:00 - 20:30 - Break with snacks and drinks

20:30 - 21:15 - Food for thought

We invited Adam Jacob from System Initiative (https://www.systeminit.com/), as well as people from Hashicorp, Winglang.io and OpenTF Foundation to address the crowd and broaden our perspectives on IaC.

21:15 - 21:30 Closing and networking ------------------------------------------------------------------------------------------------ How to get to HeyJobs Address: Paul-Lincke-Ufer 39, 10999 Berlin, First courtyard, 4th floor Closest station is Kottbusser Tor (U8, U1 and U3). Follow the direction of Paul-Linke-Ufer, then walk on Paul-Lincke-Ufer street for about 3 minutes. Then turn left into the courtyard where Concierge coffee is, just before Zola pizzeria. Enter the courtyard and to your right you will see a sign for HeyJobs. Take the elevator or stairs to the 4th floor and follow the signs inside to get to the room (called Warehouse).

Dear all, We are happy to announce a joint session by two of the greatest and bestest Meetup groups in Berlin - the Berlin DevOps and the Berlin AWS User Group. Registering on both is not necessary 😉

The State of Infrastructure as Code

The idea is to have insightful discussions with the audience about the current Infrastructure as Code landscape and allow practitioners to share their experiences and discuss with others.

If you are keen to tell your story, your successes and failures to others, please fill in this form to register with your tooling of choice - https://forms.gle/T5zUQRtfFgnwCVey8

In the second part we will have (remote) guests from various projects, giving food for thought and visions for the IaC landscape in the months ahead.

This event will happen in-person, kindly sponsored by HeyJobs! as well as remotely. Please only RSVP if you intend to participate in person in Kreuzberg. The event is published in both Meetup groups - Berlin DevOps and AWS User Group - please RSVP in only one of them if you happen to be a member of both groups.

Details for remote access will be shared in time with the community.

Please note, we will take pictures during the event as well as record the sessions.

===================================================================== The evening

18:30 - Warming up and networking 18:45 - Welcome talk by HeyJobs! 19:00 - 20:00 - Fishbowl discussion Practitioners share their opinionated experiences with landscapes and toolings while the audience is invited to join the discussions. An introduction to the format happens at the evening, pre-read here: https://en.wikipedia.org/wiki/Fishbowl_(conversation)

20:00 - 20:30 - Break with snacks and drinks

20:30 - 21:15 - Food for thought We invited Adam Jacob from System Initiative (https://www.systeminit.com/), as well as people from Hashicorp, Winglang.io and OpenTF Foundation to address the crowd and broaden our perspectives on IaC. 21:15 - 21:30 Closing and networking

=====================================================================

How to get to HeyJobs Address: Paul-Lincke-Ufer 39, 10999 Berlin, First courtyard, 4th floor

Closest station is Kottbusser Tor (U8, U1 and U3). Follow the direction of Paul-Linke-Ufer, then walk on Paul-Lincke-Ufer street for about 3 minutes. Then turn left into the courtyard where Concierge coffee is, just before Zola pizzeria. Enter the courtyard and to your right you will see a sign for HeyJobs. Take the elevator or stairs to the 4th floor and follow the signs inside to get to the room (called Warehouse).

Summary

Business intellingence has been chasing the promise of self-serve data for decades. As the capabilities of these systems has improved and become more accessible, the target of what self-serve means changes. With the availability of AI powered by large language models combined with the evolution of semantic layers, the team at Zenlytic have taken aim at this problem again. In this episode Paul Blankley and Ryan Janssen explore the power of natural language driven data exploration combined with semantic modeling that enables an intuitive way for everyone in the business to access the data that they need to succeed in their work.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control. Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Sign up free at dataengineeringpodcast.com/rudderstack Your host is Tobias Macey and today I'm interviewing Paul Blankley and Ryan Janssen about Zenlytic, a no-code business intelligence tool focused on emerging commerce brands

Interview

Introduction How did you get involved in the area of data management? Can you describe what Zenlytic is and the story behind it? Business intelligence is a crowded market. What was your process for defining the problem you are focused on solving and the method to achieve that outcome? Self-serve data exploration has been attempted in myriad ways over successive generations of BI and data platforms. What are the barriers that have been the most challenging to overcome in that effort?

What are the elements that are coming together now that give you confidence in being able to deliver on that?

Can you describe how Zenlytic is implemented?

What are the evolutions in the understanding and implementation of semantic layers that provide a sufficient substrate for operating on? How have the recent breakthroughs in large language models (LLMs) improved your ability to build features in Zenlytic? What is your process for adding domain semantics to the operational aspect of your LLM?

For someone using Zenlytic, what is the process for getting it set up and integrated with their data? Once it is operational, can you describe some typical workflows for using Zenlytic in a business context?

Who are the target users? What are the collaboration options available?

What are the most complex engineering/data challenges that you have had to address in building Zenlytic? What are the most interesting, innovative, or unexpected ways that you have seen Zenlytic used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Zenlytic? When is Zenlytic the wrong choice? What do you have planned for the future of Zenlytic?

Contact Info

Paul Blankley (LinkedIn)

Parting Question

From your perspective, what is the biggest gap in the tooling or technology for data management today?

Closing Announcements

Thank you for listening! Don't forget to check out our other shows. Podcast.init covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast helps you go from idea to production with machine learning. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email [email protected]) with your story. To help other people find the show please leave a review on Apple Podcasts and tell your friends and co-workers

Links

Zenlytic OLAP Cube Large Language Model Starburst Pr

Globally available resources have become the status quo. They're accessible, distributed, and resilient. Our traditional SQL database options haven't kept up. Centralized SQL databases, even those with read replicas in the cloud, put all the transactional load on a central system. The further away that a transaction happens from the user, the more the user experience suffers. If the transactional data powering the application is greatly slowed down, fast-loading web pages mean nothing. In this report, Paul Modderman, Jim Walker, and Charles Custer explain how distributed SQL fits all applications and eliminates complex challenges like sharding from traditional RDBMS systems. You'll learn how distributed SQL databases can reach global scale without introducing the consistency trade-offs found in NoSQL solutions. These databases come to life through cloud computing, while legacy databases simply can't rise to meet the elastic and ubiquitous new paradigm. You'll learn: Key concepts driving this new technology, including the CAP theorem, the Raft consensus algorithm, multiversion concurrency control, and Google Spanner How distributed SQL databases meet enterprise requirements, including management, security, integration, and Everything as a Service (XaaS) The impact that distributed SQL has already made in the telecom, retail, and gaming industries Why serverless computing is an ideal fit for distributed SQL How distributed SQL can help you expand your company's strategic plan

Printed in full color! Unlock the groundbreaking advances of deep learning with this extensively revised new edition of the bestselling original. Learn directly from the creator of Keras and master practical Python deep learning techniques that are easy to apply in the real world. In Deep Learning with Python, Second Edition you will learn: Deep learning from first principles Image classification and image segmentation Timeseries forecasting Text classification and machine translation Text generation, neural style transfer, and image generation Printed in full color throughout Deep Learning with Python has taught thousands of readers how to put the full capabilities of deep learning into action. This extensively revised full color second edition introduces deep learning using Python and Keras, and is loaded with insights for both novice and experienced ML practitioners. You’ll learn practical techniques that are easy to apply in the real world, and important theory for perfecting neural networks. About the Technology Recent innovations in deep learning unlock exciting new software capabilities like automated language translation, image recognition, and more. Deep learning is quickly becoming essential knowledge for every software developer, and modern tools like Keras and TensorFlow put it within your reach—even if you have no background in mathematics or data science. This book shows you how to get started. About the Book Deep Learning with Python, Second Edition introduces the field of deep learning using Python and the powerful Keras library. In this revised and expanded new edition, Keras creator François Chollet offers insights for both novice and experienced machine learning practitioners. As you move through this book, you’ll build your understanding through intuitive explanations, crisp color illustrations, and clear examples. You’ll quickly pick up the skills you need to start developing deep-learning applications. What's Inside Deep learning from first principles Image classification and image segmentation Time series forecasting Text classification and machine translation Text generation, neural style transfer, and image generation Printed in full color throughout About the Reader For readers with intermediate Python skills. No previous experience with Keras, TensorFlow, or machine learning is required. About the Author François Chollet is a software engineer at Google and creator of the Keras deep-learning library. Quotes Chollet is a master of pedagogy and explains complex concepts with minimal fuss, cutting through the math with practical Python code. He is also an experienced ML researcher and his insights on various model architectures or training tips are a joy to read. - Martin Görner, Google Immerse yourself into this exciting introduction to the topic with lots of real-world examples. A must-read for every deep learning practitioner. - Sayak Paul, Carted The modern classic just got better. - Edmon Begoli, Oak Ridge National Laboratory Truly the bible of deep learning. - Yiannis Paraskevopoulos, University of West Attica

Summary Anomaly detection is a capability that is useful in a variety of problem domains, including finance, internet of things, and systems monitoring. Scaling the volume of events that can be processed in real-time can be challenging, so Paul Brebner from Instaclustr set out to see how far he could push Kafka and Cassandra for this use case. In this interview he explains the system design that he tested, his findings for how these tools were able to work together, and how they behaved at different orders of scale. It was an interesting conversation about how he stress tested the Instaclustr managed service for benchmarking an application that has real-world utility.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With 200Gbit private networking, scalable shared block storage, and a 40Gbit public network, you’ve got everything you need to run a fast, reliable, and bullet-proof data platform. If you need global distribution, they’ve got that covered too with world-wide datacenters including new ones in Toronto and Mumbai. And for your machine learning workloads, they just announced dedicated CPU instances. Go to dataengineeringpodcast.com/linode today to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show! Integrating data across the enterprise has been around for decades – so have the techniques to do it. But, a new way of integrating data and improving streams has evolved. By integrating each silo independently – data is able to integrate without any direct relation. At CluedIn they call it “eventual connectivity”. If you want to learn more on how to deliver fast access to your data across the enterprise leveraging this new method, and the technologies that make it possible, get a demo or presentation of the CluedIn Data Hub by visiting dataengineeringpodcast.com/cluedin. And don’t forget to thank them for supporting the show! You listen to this show to learn and stay up to date with what’s happening in databases, streaming platforms, big data, and everything else you need to know about modern data management.For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Dataversity, and the Open Data Science Conference. Coming up this fall is the combined events of Graphorum and the Data Architecture Summit. The agendas have been announced and super early bird registration for up to $300 off is available until July 26th, with early bird pricing for up to $200 off through August 30th. Use the code BNLLC to get an additional 10% off any pass when you register. Go to dataengineeringpodcast.com/conferences to learn more and take advantage of our partner discounts when you register. Go to dataengineeringpodcast.com to subscribe to the show, sign up for the mailing list, read the show notes, and get in touch. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Join the community in the new Zulip chat workspace at dataengineeringpodcast.com/chat Your host is Tobias Macey and today I’m interviewing Paul Brebner about his experience designing and building a scalable, real-time anomaly detection system using Kafka and Cassandra

Interview

Introduction How did you get involved in the area of data management? Can you start by describing the problem that you were trying to solve and the requirements that you were aiming for?

What are some example cases where anomaly detection is useful or necessary?

Once you had established the requirements in terms of functionality and data volume, what was your approach for dete

Easy to read and comprehensive, Survival Analysis Using SAS: A Practical Guide, Second Edition, by Paul D. Allison, is an accessible, data-based introduction to methods of survival analysis. Researchers who want to analyze survival data with SAS will find just what they need with this fully updated new edition that incorporates the many enhancements in SAS procedures for survival analysis in SAS 9. Although the book assumes only a minimal knowledge of SAS, more experienced users will learn new techniques of data input and manipulation. Numerous examples of SAS code and output make this an eminently practical book, ensuring that even the uninitiated become sophisticated users of survival analysis. The main topics presented include censoring, survival curves, Kaplan-Meier estimation, accelerated failure time models, Cox regression models, and discrete-time analysis. Also included are topics not usually covered in survival analysis books, such as time-dependent covariates, competing risks, and repeated events.

Survival Analysis Using SAS: A Practical Guide, Second Edition, has been thoroughly updated for SAS 9, and all figures are presented using ODS Graphics. This new edition also documents major enhancements to the STRATA statement in the LIFETEST procedure; includes a section on the PROBPLOT command, which offers graphical methods to evaluate the fit of each parametric regression model; introduces the new BAYES statement for both parametric and Cox models, which allows the user to do a Bayesian analysis using MCMC methods; demonstrates the use of the counting process syntax as an alternative method for handling time-dependent covariates; contains a section on cumulative incidence functions; and describes the use of the new GLIMMIX procedure to estimate random-effects models for discrete-time data.

This book is part of the SAS Press program.