Dive into Azure AI Studio - a cutting-edge 'code first' experience to build generative AI applications including enterprise chat and custom copilots. We'll share how to ingest real-time data from Microsoft Fabric's OneLake, test Large Language Models (LLMs), and deploy Proof of Concepts (PoCs) at scale. Join us for a journey of innovation!

To learn more, please check out these resources: * https://aka.ms/Ignite23CollectionsBRK201H * https://info.microsoft.com/ww-landing-contact-me-for-events-m365-in-person-events.html?LCID=en-us&ls=407628-contactme-formfill * https://aka.ms/azure-ignite2023-dataaiblog

𝗦𝗽𝗲𝗮𝗸𝗲𝗿𝘀: * Nabila Babar * Courtney Brewer * Li-Juan Qin * Richard L. Tso * John Montgomery

𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗜𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻: This video is one of many sessions delivered for the Microsoft Ignite 2023 event. View sessions on-demand and learn more about Microsoft Ignite at https://ignite.microsoft.com

BRK201HG | AI & Apps

MSIgnite

Generative AI is enabling businesses across verticals to disrupt industries. With Microsoft, you can accelerate AI innovation and drive business growth at each stage of your company’s journey. In this session we will cover how you can build AI apps on Azure while also leveraging programs that enable you to easily get started and grow your business. Hear from MakeMyTrip, India's leading online travel platform, on how they leveraged Azure AI to expand their customer reach.

𝗦𝗽𝗲𝗮𝗸𝗲𝗿𝘀: * Neelay Thaker * Ravi Sudhakar Kambhampati

𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗜𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻: This video is one of many sessions delivered for the Microsoft Ignite 2023 event. View sessions on-demand and learn more about Microsoft Ignite at https://ignite.microsoft.com

OD26 | English (US) | AI & Apps

MSIgnite

Send us a text He's BACK!  Roger Premo, General Manager, Corporate Strategy and Ventures Development at IBM.  How the world has changed in a short year.  Generative AI and more!   02:29 Meet Roger Premo Take 205:52 A Changing World08:18 Generative AI12:48 Both Sides of the Story14:22 Hybrid Cloud and AI20:50 IBM's watsonx25:53 What Have We Learned?27:46 Enterprise Models29:59 Hugging Face31:03 IBM's Differentiation32:23 The 2 min Bar Pitch35:57 Three Questions42:21 An Intentional Hybrid Cloud Architecture 46:40 Responsible AILinkedin: https://www.linkedin.com/in/ropremo/ Website: https://www.ibm.com/watsonx Want to be featured as a guest on Making Data Simple?  Reach out to us at [email protected] and tell us why you should be next.  The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.  Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

Our industry’s breathless hype about generative AI tends to overlook the stubborn challenge of data governance. Data catalogs address this challenge by evaluating and controlling the accuracy, explainability, privacy, IP friendliness, and fairness of GenAI inputs. Published at: https://www.eckerson.com/articles/generative-ai-needs-vigilant-data-cataloging-and-governance

Summary

Software development involves an interesting balance of creativity and repetition of patterns. Generative AI has accelerated the ability of developer tools to provide useful suggestions that speed up the work of engineers. Tabnine is one of the main platforms offering an AI powered assistant for software engineers. In this episode Eran Yahav shares the journey that he has taken in building this product and the ways that it enhances the ability of humans to get their work done, and when the humans have to adapt to the tool.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. You specify the customer traits, then Profiles runs the joins and computations for you to create complete customer profiles. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Datafold leverages data-diffing to compare production and development environments and column-level lineage to show you the exact impact of every code change on data, metrics, and BI tools, keeping your team productive and stakeholders happy. Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics. Trusted by teams of all sizes, including Comcast and Doordash, Starburst is a data lake analytics platform that delivers the adaptability and flexibility a lakehouse ecosystem promises. And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Want to see Starburst in action? Go to dataengineeringpodcast.com/starburst and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino. You shouldn't have to throw away the database to build with fast-changing data. You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products. Whether it’s real-time dashboarding and analytics, personalization and segmentation or automation and alerting, Materialize gives you the ability to work with fresh, correct, and scalable results — all in a familiar SQL interface. Go to dataengineeringpodcast.com/materialize today to get 2 weeks free! Your host is Tobias Macey and today I'm interviewing Eran Yahav about building an AI powered developer assistant at Tabnine

Interview

Introduction How did you get involved in machine learning? Can you describe what Tabnine is and the story behind it? What are the individual and organizational motivations for using AI to generate code?

What are the real-world limitations of generative AI for creating software? (e.g. size/complexity of the outputs, naming conventions, etc.) What are the elements of skepticism/overs

Companies today are moving rapidly to integrate generative AI into their products and services. But there's a great deal of hype (and misunderstanding) about the impact and promise of this technology. With this book, Chris Fregly, Antje Barth, and Shelbee Eigenbrode from AWS help CTOs, ML practitioners, application developers, business analysts, data engineers, and data scientists find practical ways to use this exciting new technology. You'll learn the generative AI project life cycle including use case definition, model selection, model fine-tuning, retrieval-augmented generation, reinforcement learning from human feedback, and model quantization, optimization, and deployment. And you'll explore different types of models including large language models (LLMs) and multimodal models such as Stable Diffusion for generating images and Flamingo/IDEFICS for answering questions about images. Apply generative AI to your business use cases Determine which generative AI models are best suited to your task Perform prompt engineering and in-context learning Fine-tune generative AI models on your datasets with low-rank adaptation (LoRA) Align generative AI models to human values with reinforcement learning from human feedback (RLHF) Augment your model with retrieval-augmented generation (RAG) Explore libraries such as LangChain and ReAct to develop agents and actions Build generative AI applications with Amazon Bedrock

Jonathan Frankle is the Chief Scientist at MosaicML, which was recently bought by Databricks for $1.3 billion.  MosaicML helps customers train generative AI models on their data. Lots of companies are excited about gen AI, and the hope is that their company data and information will be what sets them apart from the competition.  In this conversation with Tristan and Julia, Jonathan discusses a potential future where you can train specialized, purpose-built models, the future of MosaicML inside of Databricks, and the importance of responsible AI practices. For full show notes and to read 6+ years of back issues of the podcast's companion newsletter, head to https://roundup.getdbt.com. The Analytics Engineering Podcast is sponsored by dbt Labs.

I think it's safe to say that we are in the peak of the hype cycle with generative AI. Almost every week now, we see new startups with exciting new GenAI use-cases and products. However, exciting doesn't necessarily translate to useful. And now more than ever, it's important for leaders, whether at incumbents or startups, to adapt and drive value with generative AI and focus on useful use-cases. So how can they adapt well to these tectonic changes? Jason Feifer is the editor in chief of Entrepreneur magazine and host of the podcast Problem Solvers. Outside of Entrepreneur, he is the author of the book Build For Tomorrow, which helps readers find new opportunities in times of change, and co-hosts the podcast Help Wanted, where he helps solve listeners' work problems. He also writes a newsletter called One Thing Better, which each week gives you one better way to build a career or company you love. In the episode, Jason and Adel explore AI’s role in entrepreneurship, use cases and applications of AI, the effectiveness of certain AI tools, AI’s impact on established business models, frameworks for navigating change, advice for leaders and individuals on using AI in their work and much more.  Links Mentioned in the Show: Build for Tomorrow by Jason FeiferOne Thing Better NewsletterHeyGenBurger King Accepting Credit Cards in the 90s[COURSE] Implementing AI Solutions in Business

Send us a text Sam Torres is the Chief Digital Officer and co-founder of The Gray Dot Company, Gray Dot Company is a consulting firm that specializes in search engine optimization. Sam outlines expertise in complex digital analytics and consumer insights data. 03:40 Meet Sam Torres05:57 Marketing Platforms07:49 Digital Consumer Intelligence14:55 Defining Success17:55 AIs Impact on Google22:07 Should I Trust Sponsored Adds?23:58 GenAI Positives LinkedIn: linkedin.com/in/samantha-torres-seo Website: https://thegray.company,  https://legendarypodcasts.com/sam-torres/

Want to be featured as a guest on Making Data Simple? Reach out to us [email protected] and tell us why you should be next. The MakingData Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, wherewe explore trending technologies, business innovation, and leadership ... whilekeeping it simple & fun. Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

Summary

Artificial intelligence applications require substantial high quality data, which is provided through ETL pipelines. Now that AI has reached the level of sophistication seen in the various generative models it is being used to build new ETL workflows. In this episode Jay Mishra shares his experiences and insights building ETL pipelines with the help of generative AI.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. You specify the customer traits, then Profiles runs the joins and computations for you to create complete customer profiles. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Datafold leverages data-diffing to compare production and development environments and column-level lineage to show you the exact impact of every code change on data, metrics, and BI tools, keeping your team productive and stakeholders happy. Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold You shouldn't have to throw away the database to build with fast-changing data. You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! It’s the only true SQL streaming database built from the ground up to meet the needs of modern data products. Whether it’s real-time dashboarding and analytics, personalization and segmentation or automation and alerting, Materialize gives you the ability to work with fresh, correct, and scalable results — all in a familiar SQL interface. Go to dataengineeringpodcast.com/materialize today to get 2 weeks free! As more people start using AI for projects, two things are clear: It’s a rapidly advancing field, but it’s tough to navigate. How can you get the best results for your use case? Instead of being subjected to a bunch of buzzword bingo, hear directly from pioneers in the developer and data science space on how they use graph tech to build AI-powered apps. . Attend the dev and ML talks at NODES 2023, a free online conference on October 26 featuring some of the brightest minds in tech. Check out the agenda and register at Neo4j.com/NODES. Your host is Tobias Macey and today I'm interviewing Jay Mishra about the applications for generative AI in the ETL process

Interview

Introduction How did you get involved in the area of data management? What are the different aspects/types of ETL that you are seeing generative AI applied to?

What kind of impact are you seeing in terms of time spent/quality of output/etc.?

What kinds of projects are most likely to benefit from the application of generative AI? Can you describe what a typical workflow of using AI to build ETL workflows looks like?

What are some of the types of errors that you are likely to experience from the AI? Once the pipeline is defined, what does the ongoing maintenance look like? Is the AI required to operate within the pipeline in perpetuity?

For individuals/teams/organizations who are experimenting with AI in their data engineering workflows, what are the concerns/questions that they are trying to address? What are the most interesting, innovative, or unexpected w

From the dawn of humanity, decisions, both big and small, have shaped our trajectory. Decisions have built civilizations, forged alliances, and even charted the course of our very evolution. And now, as data & AI become more widespread, the potential upside for better decision making is massive. Yet, like any technology, the true value of data & AI is realized by how we wield it.  We're often drawn to the allure of the latest tools and techniques, but it's crucial to remember that these tools are only as effective as the decisions we make with them. ChatGPT is only as good as the prompt you decide to feed it and what you decide to do with the output. A dashboard is only as good as the decisions that it influences. Even a data science team is only as effective as the value they deliver to the organization.  So in this vast landscape of data and AI, how can we master the art of better decision making? How can we bridge data & AI with better decision intelligence? ​​Cassie Kozyrkov founded the field of Decision Intelligence at Google where, until recently, she served as Chief Decision Scientist, advising leadership on decision process, AI strategy, and building data-driven organizations. Upon leaving Google, Cassie started her own company of which she is the CEO, Data Scientific. In almost 10 years at the company, Cassie personally trained over 20,000 Googlers in data-driven decision-making and AI and has helped over 500 projects implement decision intelligence best practices. Cassie also previously served in Google's Office of the CTO as Chief Data Scientist, and the rest of her 20 years of experience was split between consulting, data science, lecturing, and academia.  Cassie is a top keynote speaker and a beloved personality in the data leadership community, followed by over half a million tech professionals. If you've ever went on a reading spree about AI, statistics, or decision-making, chances are you've encountered her writing, which has reached millions of readers.  In the episode Cassie and Richie explore misconceptions around data science, stereotypes associated with being a data scientist, what the reality of working in data science is, advice for those starting their career in data science, and the challenges of being a data ‘jack-of-all-trades’.  Cassie also shares what decision-science and decision intelligence are, what questions to ask future employers in any data science interview, the importance of collaboration between decision-makers and domain experts, the differences between data science models and their real-world implementations, the pros and cons of generative AI in data science, and much more.  Links mentioned in the Show: Data scientist: The sexiest job of the 22nd centuryThe Netflix PrizeAI Products: Kitchen AnalogyType one, Two & Three Errors in StatisticsCourse: Data-Driven Decision Making for BusinessRadar: Data & AI Literacy...

Generative AI initiatives require new data pipelines that prepare text files for querying by language models. Data engineers, scientists, and other stakeholders collaborate to design and implement these pipelines, which span text sources, tokens, vectors, vector databases, and LMs. Published at: https://www.eckerson.com/articles/the-new-data-pipeline-for-generative-ai-where-and-how-it-works

It's been a year since ChatGPT burst onto the scene. It has given many of us a sense of the power and potential that LLMs hold in revolutionizing the global economy. But the power that generative AI brings also comes with inherent risks that need to be mitigated.  For those working in AI, the task at hand is monumental: to chart a safe and ethical course for the deployment and use of artificial intelligence. This isn't just a challenge; it's potentially one of the most important collective efforts of this decade. The stakes are high, involving not just technical and business considerations, but ethical and societal ones as well. How do we ensure that AI systems are designed responsibly? How do we mitigate risks such as bias, privacy violations, and the potential for misuse? How do we assemble the right multidisciplinary mindset and expertise for addressing AI safety?  Reid Blackman, Ph.D., is the author of “Ethical Machines” (Harvard Business Review Press), creator and host of the podcast “Ethical Machines,” and Founder and CEO of Virtue, a digital ethical risk consultancy. He is also an advisor to the Canadian government on their federal AI regulations, was a founding member of EY’s AI Advisory Board, and a Senior Advisor to the Deloitte AI Institute. His work, which includes advising and speaking to organizations including AWS, US Bank, the FBI, NASA, and the World Economic Forum, has been profiled by The Wall Street Journal, the BBC, and Forbes. His written work appears in The Harvard Business Review and The New York Times. Prior to founding Virtue, Reid was a professor of philosophy at Colgate University and UNC-Chapel Hill. In the episode, Reid and Richie discuss the dominant concerns in AI ethics, from biased AI and privacy violations to the challenges introduced by generative AI, such as manipulative agents and IP issues. They delve into the existential threats posed by AI, including shifts in the job market and disinformation. Reid also shares examples where unethical AI has led to AI projects being scrapped, the difficulty in mitigating bias, preemptive measures for ethical AI and much more.  Links mentioned in the show: Ethical Machines by Reid BlackmanVirtue Ethics ConsultancyAmazon’s Scrapped AI Recruiting ToolNIST AI Risk Management FrameworkCourse: AI EthicsDataCamp Radar: Data & AI Literacy

For the past few years, we've seen the importance of data literacy and why organizations must invest in a data-driven culture, mindset, and skillset. However, as generative AI tools like ChatGPT have risen to prominence in the past year, AI literacy has never been more important. But how do we begin to approach AI literacy? Is it an extension of data literacy, a complement, or a new paradigm altogether? How should you get started on your AI literacy ambitions?  Cindi Howson is the Chief Data Strategy Officer at ThoughtSpot and host of The Data Chief podcast. Cindi is a data analytics, AI, and BI thought leader and an expert with a flair for bridging business needs with technology. As Chief Data Strategy Officer at ThoughtSpot, she advises top clients on data strategy and best practices to become data-driven, speaks internationally on top trends such as AI ethics, and influences ThoughtSpot’s product strategy.

Cindi was previously a Gartner Research Vice President, the lead author for the data and analytics maturity model and analytics and BI Magic Quadrant, and a popular keynote speaker. She introduced new research in data and AI for good, NLP/BI Search, and augmented analytics, bringing both BI bake-offs and innovation panels to Gartner globally. She’s frequently quoted in MIT, Harvard Business Review, and Information Week. She is rated a top 12 influencer in big data and analytics by Analytics Insight, Onalytca, Solutions Review, and Humans of Data.

In the episode, Cindi and Adel discuss how generative AI accelerates an organization’s data literacy, how leaders can think beyond data literacy and start to think about AI literacy, the importance of responsible use of AI, how to best communicate the value of AI within your organization, what generative AI means for data teams, AI use-cases in the data space, the psychological barriers blocking AI adoption, and much more. 

Links Mentioned in the Show: The Data Chief Podcast  ThoughtSpot Sage  BloombergGPT  Radar: Data & AI Literacy Course: AI Ethics  Course: Generative AI Concepts Course: Implementing AI Solutions in Business 

Responsible AI ethical principles provide a clear, unifying purpose for the technological, business, and social goals of AI initiatives. Published at: https://www.eckerson.com/articles/the-opportunity-and-risk-of-generative-ai-part-iii-responsible-ai-ethics

Gen AI, LLMs, AI assistants and intelligent agents are powering next-generation customer experiences, transforming every business. But there is no AI without Data. And only the right data delivers accurate, relevant results, with the context, scale and security you need.

"I Love AI" will unlock the power of Generative AI for you, with unique insights into the data platform and AI solutions you need, delivered by experts with real-world experience making AI a reality. This virtual event will help application architects, software developers, practitioners and CTOs learn how to:

• Deliver AI outcomes with extreme accuracy and relevance
• Build Generative AI apps with scale, governance and data security
• Overcome the biggest obstacles keeping Gen AI from being enterprise ready
• Deploy powerful vector search capabilities at a fraction of the cost
• Use cutting edge innovations in the biggest, most powerful vector database

Gen AI, LLMs, AI assistants and intelligent agents are powering next-generation customer experiences, transforming every business. But there is no AI without Data. And only the right data delivers accurate, relevant results, with the context, scale and security you need.

"I Love AI" will unlock the power of Generative AI for you, with unique insights into the data platform and AI solutions you need, delivered by experts with real-world experience making AI a reality. This virtual event will help application architects, software developers, practitioners and CTOs learn how to:

• Deliver AI outcomes with extreme accuracy and relevance
• Build Generative AI apps with scale, governance and data security
• Overcome the biggest obstacles keeping Gen AI from being enterprise ready
• Deploy powerful vector search capabilities at a fraction of the cost
• Use cutting edge innovations in the biggest, most powerful vector database

Session 1 (Americas/EMEA): August 23, 2023, 10AM PDT / 1PM EDT. Gen AI, LLMs, AI assistants and intelligent agents powering next-generation customer experiences. This session will cover data platforms, AI solutions, vector search capabilities, governance and data security.

Session 2 (APAC/EMEA): August 24, 2023, 10AM CEST / 1:30PM IST / 4PM SGT / 6PM AEST. Gen AI, LLMs, AI assistants and intelligent agents powering next-generation customer experiences. This session will cover data platforms, AI solutions, vector search capabilities, governance and data security.