talk-data.com

Build high-performance and enterprise-ready AI applications with AlloyDB for PostgreSQL

PostgreSQL is the world’s most popular database, and Google’s AlloyDB for PostgreSQL takes it further with enterprise-grade performance and built-in AI capabilities. Available both as a fully managed service and as AlloyDB Omni to run anywhere, AlloyDB is helping organizations build the next generation of intelligent applications.

In this session, Sanjay Mishra (former Head of Product for Azure SQL and SQL Server at Microsoft, now at Google) will showcase real-world examples and demos of how AlloyDB is powering AI-driven solutions across industries.

Learning Objective: 🔹 Understand AlloyDB Omni and AlloyDB AI 🔹 Explore key differentiators of AlloyDB vs. other PostgreSQL offerings 🔹 See how real-world applications are using AlloyDB to solve customer challenges

𝐌𝐨𝐝𝐞𝐫𝐚𝐭𝐨𝐫: Jean Joseph - Data & AI Expert \| MCT \| Ex-Microsoft \| Ex-Microsoft MVP \| IT event planner \| Speaker \| Blogger \| Data Driven Community Builder \| Founder & main organizer of Cloud Data Driven User Group & Future Data Driven Summit

Vector Search with Azure SQL Database

Description

This session Vector Search with Azure SQL Database is a deep dive into the cutting-edge integration of vector search capabilities with Azure SQL Database, revolutionizing the landscape of search functionalities. Attendees will gain insights into the principles of vector search and how it surpasses traditional search methods. Participants will gain insights into the principles of vector search, surpassing conventional methods, and delve into topics such as Integrated Vectorization, Retrieval Augmented Generation (RAG), Chunking, OpenAI, and more. The session will detail the seamless integration of vector search with Azure SQL Database, allowing for efficient similarity searches across vast datasets. Participants will explore real-world use cases, demonstrating how vector search transforms information retrieval, enhances recommendation systems, and facilitates content matching. This session is tailored for Data and AI developers, architects, and data enthusiasts keen on harnessing the power of vector search within the Azure SQL ecosystem for more robust and intelligent search experiences.

Register on the event website to receive joining link: https://www.aicamp.ai/event/eventdetails/W2024012609

Description: Welcome to the weekly AI virtual seminars. Join us for deep dive tech talks on AI/ML/Data, hands-on experiences on code labs, workshops, and networking with speakers & fellow developers from all over the world.

Topic: Chat with your Data Speaker: Muazma Zahid (Microsoft); Sanjay Mishra (Microsoft) Abstract: Immerse yourself in the world of "Data Dialogues," where developers transcend traditional database interactions to engage in rich conversations with their data. This session unveils the power of integrating natural language processing into SQL databases, enabling developers to build intelligent applications that not only query data but hold dynamic dialogues. Discover how developers can seamlessly chat with their data, opening up new avenues for intuitive and interactive data exploration. This session is your gateway to a new era of intelligent and conversational applications with SQL Databases.

Community on Slack/Discord - Event chat: chat and connect with speakers and attendees - Sharing blogs\, events\, job openings\, projects collaborations Join Slack (search and join the #virtualevents channel) \| Join Discord

Community Managers: we are looking for passionate community managers who are enthusiastic about nurturing tech communities. Join us in planning, hosting meetup events, and build and grow the local AI/ML developer community. In addition to broadening your personal network and acquiring valuable tech skills, we'll make sure to reward your dedication and time.

Speakers: If you have a keen interest in speaking to our community, we invite you to submit topics for consideration: https://forms.gle/JkMt91CZRtoJBSFUA

Sponsors: We are actively seeking sponsors to support our community. Whether it is by offering venue spaces, providing food/drink, or cash sponsor. Sponsors will have the chance to speak at the meetups, receive prominent recognition, and gain exposure to our extensive membership base of 300K+ developers worldwide.

Register on the event website to receive joining link: https://www.aicamp.ai/event/eventdetails/W2024012609

Description: Welcome to the weekly AI virtual seminars. Join us for deep dive tech talks on AI/ML/Data, hands-on experiences on code labs, workshops, and networking with speakers & fellow developers from all over the world.

Topic: Chat with your Data Speaker: Muazma Zahid (Microsoft); Sanjay Mishra (Microsoft) Abstract: Immerse yourself in the world of "Data Dialogues," where developers transcend traditional database interactions to engage in rich conversations with their data. This session unveils the power of integrating natural language processing into SQL databases, enabling developers to build intelligent applications that not only query data but hold dynamic dialogues. Discover how developers can seamlessly chat with their data, opening up new avenues for intuitive and interactive data exploration. This session is your gateway to a new era of intelligent and conversational applications with SQL Databases.

Community on Slack/Discord - Event chat: chat and connect with speakers and attendees - Sharing blogs\, events\, job openings\, projects collaborations Join Slack (search and join the #virtualevents channel) \| Join Discord

Community Managers: we are looking for passionate community managers who are enthusiastic about nurturing tech communities. Join us in planning, hosting meetup events, and build and grow the local AI/ML developer community. In addition to broadening your personal network and acquiring valuable tech skills, we'll make sure to reward your dedication and time.

Speakers: If you have a keen interest in speaking to our community, we invite you to submit topics for consideration: https://forms.gle/JkMt91CZRtoJBSFUA

Sponsors: We are actively seeking sponsors to support our community. Whether it is by offering venue spaces, providing food/drink, or cash sponsor. Sponsors will have the chance to speak at the meetups, receive prominent recognition, and gain exposure to our extensive membership base of 300K+ developers worldwide.

Register on the event website to receive joining link: https://www.aicamp.ai/event/eventdetails/W2024012609

Description: Welcome to the weekly AI virtual seminars. Join us for deep dive tech talks on AI/ML/Data, hands-on experiences on code labs, workshops, and networking with speakers & fellow developers from all over the world.

Topic: Chat with your Data Speaker: Muazma Zahid (Microsoft); Sanjay Mishra (Microsoft) Abstract: Immerse yourself in the world of "Data Dialogues," where developers transcend traditional database interactions to engage in rich conversations with their data. This session unveils the power of integrating natural language processing into SQL databases, enabling developers to build intelligent applications that not only query data but hold dynamic dialogues. Discover how developers can seamlessly chat with their data, opening up new avenues for intuitive and interactive data exploration. This session is your gateway to a new era of intelligent and conversational applications with SQL Databases.

Community on Slack/Discord - Event chat: chat and connect with speakers and attendees - Sharing blogs\, events\, job openings\, projects collaborations Join Slack (search and join the #virtualevents channel) \| Join Discord

Community Managers: we are looking for passionate community managers who are enthusiastic about nurturing tech communities. Join us in planning, hosting meetup events, and build and grow the local AI/ML developer community. In addition to broadening your personal network and acquiring valuable tech skills, we'll make sure to reward your dedication and time.

Speakers: If you have a keen interest in speaking to our community, we invite you to submit topics for consideration: https://forms.gle/JkMt91CZRtoJBSFUA

Sponsors: We are actively seeking sponsors to support our community. Whether it is by offering venue spaces, providing food/drink, or cash sponsor. Sponsors will have the chance to speak at the meetups, receive prominent recognition, and gain exposure to our extensive membership base of 300K+ developers worldwide.

Register on the event website to receive joining link: https://www.aicamp.ai/event/eventdetails/W2024012609

Description: Welcome to the weekly AI virtual seminars. Join us for deep dive tech talks on AI/ML/Data, hands-on experiences on code labs, workshops, and networking with speakers & fellow developers from all over the world.

Topic: Chat with your Data Speaker: Muazma Zahid (Microsoft); Sanjay Mishra (Microsoft) Abstract: Immerse yourself in the world of "Data Dialogues," where developers transcend traditional database interactions to engage in rich conversations with their data. This session unveils the power of integrating natural language processing into SQL databases, enabling developers to build intelligent applications that not only query data but hold dynamic dialogues. Discover how developers can seamlessly chat with their data, opening up new avenues for intuitive and interactive data exploration. This session is your gateway to a new era of intelligent and conversational applications with SQL Databases.

Community on Slack/Discord - Event chat: chat and connect with speakers and attendees - Sharing blogs\, events\, job openings\, projects collaborations Join Slack (search and join the #virtualevents channel) \| Join Discord

Community Managers: we are looking for passionate community managers who are enthusiastic about nurturing tech communities. Join us in planning, hosting meetup events, and build and grow the local AI/ML developer community. In addition to broadening your personal network and acquiring valuable tech skills, we'll make sure to reward your dedication and time.

Speakers: If you have a keen interest in speaking to our community, we invite you to submit topics for consideration: https://forms.gle/JkMt91CZRtoJBSFUA

Sponsors: We are actively seeking sponsors to support our community. Whether it is by offering venue spaces, providing food/drink, or cash sponsor. Sponsors will have the chance to speak at the meetups, receive prominent recognition, and gain exposure to our extensive membership base of 300K+ developers worldwide.

Register on the event website to receive joining link: https://www.aicamp.ai/event/eventdetails/W2024012609

Description: Welcome to the weekly AI virtual seminars. Join us for deep dive tech talks on AI/ML/Data, hands-on experiences on code labs, workshops, and networking with speakers & fellow developers from all over the world.

Topic: Chat with your Data Speaker: Muazma Zahid (Microsoft); Sanjay Mishra (Microsoft) Abstract: Immerse yourself in the world of "Data Dialogues," where developers transcend traditional database interactions to engage in rich conversations with their data. This session unveils the power of integrating natural language processing into SQL databases, enabling developers to build intelligent applications that not only query data but hold dynamic dialogues. Discover how developers can seamlessly chat with their data, opening up new avenues for intuitive and interactive data exploration. This session is your gateway to a new era of intelligent and conversational applications with SQL Databases.

Community on Slack/Discord - Event chat: chat and connect with speakers and attendees - Sharing blogs\, events\, job openings\, projects collaborations Join Slack (search and join the #virtualevents channel) \| Join Discord

Community Managers: we are looking for passionate community managers who are enthusiastic about nurturing tech communities. Join us in planning, hosting meetup events, and build and grow the local AI/ML developer community. In addition to broadening your personal network and acquiring valuable tech skills, we'll make sure to reward your dedication and time.

Speakers: If you have a keen interest in speaking to our community, we invite you to submit topics for consideration: https://forms.gle/JkMt91CZRtoJBSFUA

Sponsors: We are actively seeking sponsors to support our community. Whether it is by offering venue spaces, providing food/drink, or cash sponsor. Sponsors will have the chance to speak at the meetups, receive prominent recognition, and gain exposure to our extensive membership base of 300K+ developers worldwide.

Register on the event website to receive joining link: https://www.aicamp.ai/event/eventdetails/W2024012609

Description: Welcome to the weekly AI virtual seminars. Join us for deep dive tech talks on AI/ML/Data, hands-on experiences on code labs, workshops, and networking with speakers & fellow developers from all over the world.

Topic: Chat with your Data Speaker: Muazma Zahid (Microsoft); Sanjay Mishra (Microsoft) Abstract: Immerse yourself in the world of "Data Dialogues," where developers transcend traditional database interactions to engage in rich conversations with their data. This session unveils the power of integrating natural language processing into SQL databases, enabling developers to build intelligent applications that not only query data but hold dynamic dialogues. Discover how developers can seamlessly chat with their data, opening up new avenues for intuitive and interactive data exploration. This session is your gateway to a new era of intelligent and conversational applications with SQL Databases.

Community on Slack/Discord - Event chat: chat and connect with speakers and attendees - Sharing blogs\, events\, job openings\, projects collaborations Join Slack (search and join the #virtualevents channel) \| Join Discord

Community Managers: we are looking for passionate community managers who are enthusiastic about nurturing tech communities. Join us in planning, hosting meetup events, and build and grow the local AI/ML developer community. In addition to broadening your personal network and acquiring valuable tech skills, we'll make sure to reward your dedication and time.

Speakers: If you have a keen interest in speaking to our community, we invite you to submit topics for consideration: https://forms.gle/JkMt91CZRtoJBSFUA

Sponsors: We are actively seeking sponsors to support our community. Whether it is by offering venue spaces, providing food/drink, or cash sponsor. Sponsors will have the chance to speak at the meetups, receive prominent recognition, and gain exposure to our extensive membership base of 300K+ developers worldwide.

Summary

Artificial intelligence applications require substantial high quality data, which is provided through ETL pipelines. Now that AI has reached the level of sophistication seen in the various generative models it is being used to build new ETL workflows. In this episode Jay Mishra shares his experiences and insights building ETL pipelines with the help of generative AI.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. You specify the customer traits, then Profiles runs the joins and computations for you to create complete customer profiles. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Datafold leverages data-diffing to compare production and development environments and column-level lineage to show you the exact impact of every code change on data, metrics, and BI tools, keeping your team productive and stakeholders happy. Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold You shouldn't have to throw away the database to build with fast-changing data. You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products. Whether it’s real-time dashboarding and analytics, personalization and segmentation or automation and alerting, Materialize gives you the ability to work with fresh, correct, and scalable results — all in a familiar SQL interface. Go to dataengineeringpodcast.com/materialize today to get 2 weeks free! As more people start using AI for projects, two things are clear: It’s a rapidly advancing field, but it’s tough to navigate. How can you get the best results for your use case? Instead of being subjected to a bunch of buzzword bingo, hear directly from pioneers in the developer and data science space on how they use graph tech to build AI-powered apps. . Attend the dev and ML talks at NODES 2023, a free online conference on October 26 featuring some of the brightest minds in tech. Check out the agenda and register at Neo4j.com/NODES. Your host is Tobias Macey and today I'm interviewing Jay Mishra about the applications for generative AI in the ETL process

Interview

Introduction How did you get involved in the area of data management? What are the different aspects/types of ETL that you are seeing generative AI applied to?

What kind of impact are you seeing in terms of time spent/quality of output/etc.?

What kinds of projects are most likely to benefit from the application of generative AI? Can you describe what a typical workflow of using AI to build ETL workflows looks like?

What are some of the types of errors that you are likely to experience from the AI? Once the pipeline is defined, what does the ongoing maintenance look like? Is the AI required to operate within the pipeline in perpetuity?

For individuals/teams/organizations who are experimenting with AI in their data engineering workflows, what are the concerns/questions that they are trying to address? What are the most interesting, innovative, or unexpected w