This session provides a comprehensive guide to building a secure and unified AI lakehouse on BigQuery with the power of open source software (OSS). We’ll explore essential components, including data ingestion, storage, and management; AI and machine learning workflows; pipeline orchestration; data governance; and operational efficiency. Learn about the newest features that support both Apache Spark and Apache Iceberg.

In a world where speed and innovation are paramount, bolting on security after deployment is no longer an option. In this Spotlight, experts from Palo Alto Networks and Google Cloud will dive into the future of AI security through practical experiences faced by customers to address today and tomorrow's most pressing challenges in AI adoption—Shadow AI, data governance complexities, and the evolving threat landscape to AI applications. Discover how a Secure by Design approach, with security embedded across AI initiatives from inception, ensures resilience, scalability, and trust. Don't miss this opportunity to gain actionable insights for accelerating your AI journey securely.

This Session is hosted by a Google Cloud Next Sponsor.
Visit your registration profile at g.co/cloudnext to opt out of sharing your contact information with the sponsor hosting this session.

This blog defines the governance requirements that streaming data pipelines must meet to make artificial intelligence/machine learning (AI/ML) initiatives successful. Published at: https://www.eckerson.com/articles/streaming-data-governance-three-must-have-requirements-to-support-ai-ml-innovation

Data sharing is essential for driving innovation in the enterprise, but ensuring security and compliance can be challenging. Join us learn from Google, LiveRamp, and Levi's about how the unified and tightly integrated data governance capabilities of BigQuery can simplify data discovery, governance, and secure sharing. Discover how to build, share, and monetize data products with robust security and compliance, while fostering a data-driven culture across your organization.

McDonald’s is advancing its AI capabilities and achieving transformative business outcomes with Collibra and Google Cloud Gemini. By integrating a multi-cloud, multi-data platform with unified data governance for AI, McDonald’s is reducing governance fragmentation, activating unused data, and addressing AI deployment risks. This session examines how McDonald's utilizes Collibra and Google Cloud's data and AI technologies to quickly implement innovative solutions and improve customer, restaurant team, and employee experiences.

Ensuring data usability is paramount to unlocking a company’s full potential and driving informed decision-making. Part of author Saurav Bhattacharya’s trilogy that covers the essential pillars of digital ecosystems—security, reliability, and usability—this book offers a comprehensive exploration of the fundamental concepts, principles, and practices essential for enhancing data accessibility and effectiveness. You’ll study the core aspects of data design, standardization, and interoperability, gaining the knowledge needed to create and maintain high-quality data environments. By examining the tools and technologies that improve data usability, along with best practices for data visualization and user-centric strategies, this book serves as an invaluable resource for professionals seeking to leverage data more effectively. The book also addresses crucial governance issues, ensuring data quality, integrity, and security are maintained. Through a detailed analysis of data governance frameworks and privacy concerns, you’ll see how to manage data responsibly. Additionally, the book includes compelling case studies that highlight successful data usability implementations, future trends, and the challenges faced in achieving optimal data usability. By fostering a culture of data literacy and usability, this book will help you and your organization navigate the evolving data landscape and harness the power of data for innovation and growth. What You Will Learn Understand the fundamental concepts and importance of data usability, including effective data design, enhancing data accessibility, and ensuring data standardization and interoperability. Review the latest tools and technologies that enhance data usability, best practices for data visualization, and strategies for implementing user-centric data approaches. Ensure data quality and integrity, while navigating data privacy and security concerns. Implement robust data governance frameworks to manage data responsibly and effectively. Who This Book Is For Cybersecurity and IT professionals

Data engineers proficient in Databricks are currently in high demand. As organizations gather more data than ever before, skilled data engineers on platforms like Databricks become critical to business success. The Databricks Data Engineer Associate certification is proof that you have a complete understanding of the Databricks platform and its capabilities, as well as the essential skills to effectively execute various data engineering tasks on the platform. In this comprehensive study guide, you will build a strong foundation in all topics covered on the certification exam, including the Databricks Lakehouse and its tools and benefits. You'll also learn to develop ETL pipelines in both batch and streaming modes. Moreover, you'll discover how to orchestrate data workflows and design dashboards while maintaining data governance. Finally, you'll dive into the finer points of exactly what's on the exam and learn to prepare for it with mock tests. Author Derar Alhussein teaches you not only the fundamental concepts but also provides hands-on exercises to reinforce your understanding. From setting up your Databricks workspace to deploying production pipelines, each chapter is carefully crafted to equip you with the skills needed to master the Databricks Platform. By the end of this book, you'll know everything you need to ace the Databricks Data Engineer Associate certification exam with flying colors, and start your career as a certified data engineer from Databricks! You'll learn how to: Use the Databricks Platform and Delta Lake effectively Perform advanced ETL tasks using Apache Spark SQL Design multi-hop architecture to process data incrementally Build production pipelines using Delta Live Tables and Databricks Jobs Implement data governance using Databricks SQL and Unity Catalog Derar Alhussein is a senior data engineer with a master's degree in data mining. He has over a decade of hands-on experience in software and data projects, including large-scale projects on Databricks. He currently holds eight certifications from Databricks, showcasing his proficiency in the field. Derar is also an experienced instructor, with a proven track record of success in training thousands of data engineers, helping them to develop their skills and obtain professional certifications.

Jason Touleyrou, Data Engineering Manager at Corewell Health joined Yuliia to discuss why most organizations struggle with data governance. He argues that data teams should focus on building trust through flexible systems rather than rigid controls. Challenging traditional data quality approaches, Jason suggests starting with basic freshness checks and evolving governance gradually. Drawing from his experience across healthcare and marketing analytics, he shares practical strategies for implementing governance during migrations and measuring data team value beyond conventional metrics. Jason's linkedin page - https://www.linkedin.com/in/jasontouleyrou/

Sarah Levy (CEO of Euno) joins me to chat about modern data governance for analytics.

Euno - https://euno.ai/

2024 was another huge year for data and AI. Generative AI continued to shape the way we work and interact with technology, with companies of all sizes racing to integrate AI into their products. We saw strides in tools like AI-enhanced data science notebooks, rapid adoption of generative image AI, and a steady march toward video generation AI. At the same time, foundational skills like AI literacy and data governance gained traction as critical areas for individuals and organizations to master. This time last year, DataCamp Co-Founders Jonathan and Martijn made a series of predictions and data and AI for 2024, today, they join Richie to reflect on their 2024 predictions and share their vision for data and AI in 2025. In the episode, Richie, Jonathan, and Martijn review the mainstream adoption of generative AI and its journey toward daily use, the rise of AI literacy as a critical skill, the growing overlap between data science and software engineering with the emergence of AI engineers, evolving trends in programming languages, how generative AI has moved from prototype to production, the near-mainstreaming of video generation AI, why AI hype continues to thrive and much more. Links Mentioned in the Show: Data & AI Trends & Predictions 2025Skill Track: AI Business FundamentalsRelated Episode: Data Trends & Predictions 2024 with DataCamp's CEO & COO, Jonathan Cornelissen & Martijn TheuwissenRewatch sessions from RADAR: Forward Edition New to DataCamp? Learn on the go using the DataCamp mobile appEmpower your business with world-class data and AI skills with DataCamp for business

AI features and products are the hottest area of software development. Creating high quality AI software is both essential and challenging for many businesses. In this episode, we look at retrieval augmented generation, an important technique for improving text generation quality in AI applications. Beyond technical measures, we look at the broader quality problem for AI applications. How do you ensure your AI applications are effective and secure? What steps should you take to integrate AI into your existing data governance frameworks? And how do you measure the success of these AI-driven solutions? Theresa Parker is the Director of Product Management at Rocket Software. She has 25 years of experience as a technology executive with a focus on software development processes, consultancy, and business development. Her recent work in content management focuses on the use of AI and RAG to improve content discoverability. Sudhi Balan is the Chief Technology Officer for AI & Cloud. He leads the AI and data teams for data modernization, driving AI adoption of Rocket's structured and unstructured data products. He also shapes AI strategy for Rocket’s infrastructure and app portfolio. He has earned patents for safe and scalable applications of transformational technology. Previously, he led digital transformation and hybrid cloud strategy for Rocket’s unstructured data business and was Senior Director of Product Development at ASG. In the episode, Richie, Theresa, and Sudhi explore retrieval-augmented generation, its applications in customer support and loan processing, the importance of data governance and privacy, the role of testing and guardrails in AI, cost management strategies, and the potential of AI to transform customer experiences, and much more. Links Mentioned in the Show: Rocket SoftwareConnect with Theresa and SudhiCourse: Retrieval Augmented Generation (RAG) with LangChainRelated Episode: Getting Generative AI Into Production with Lin Qiao, CEO and Co-Founder of Fireworks AIRewatch sessions from RADAR: Forward Edition New to DataCamp? Learn on the go using the DataCamp mobile appEmpower your business with world-class data and AI skills with DataCamp for business

Explore Snowflake’s core concepts and unique features that differentiates it from industry competitors, such as, Azure Synapse and Google BigQuery. This book provides recipes for architecting and developing modern data pipelines on the Snowflake data platform by employing progressive techniques, agile practices, and repeatable strategies. You’ll walk through step-by-step instructions on ready-to-use recipes covering a wide range of the latest development topics. Then build scalable development pipelines and solve specific scenarios common to all modern data platforms, such as, data masking, object tagging, data monetization, and security best practices. Throughout the book you’ll work with code samples for Amazon Web Services, Microsoft Azure, and Google Cloud Platform. There’s also a chapter devoted to solving machine learning problems with Snowflake. Authors Dillon Dayton and John Eipe are both Snowflake SnowPro Core certified, specializing in data and digital services, and understand the challenges of finding the right solution to complex problems. The recipes in this book are based on real world use cases and examples designed to help you provide quality, performant, and secured data to solve business initiatives. What You’ll Learn Handle structured and un- structured data in Snowflake. Apply best practices and different options for data transformation. Understand data application development. Implement data sharing, data governance and security. Who This book Is For Data engineers, scientists and analysts moving into Snowflake, looking to build data apps. This book expects basic knowledge in Cloud (AWS or Azure or GCP), SQL and Python

We’re improving DataFramed, and we need your help! We want to hear what you have to say about the show, and how we can make it more enjoyable for you—find out more here. Imagine spending millions on data tools only to find you can’t trust the answers they provide. What if different teams define key metrics in different ways? Without a clear, unified approach, chaos reigns, and confidence erodes. What role do data governance and semantic layers play in helping you trust the AI tools you build and the insights you get from your data? Sarah Levy is a seasoned executive with extensive experience in data science, artificial intelligence, and technology leadership. Currently serving as Co-Founder and CEO of Euno since January 2023, Sarah has previously held significant positions, including VP of Data Science and Data Analytics for Real Estate at Pagaya and CTO at Sight Diagnostics, where innovative advancements in blood testing were achieved. With a strong foundation in research and development from roles at Sight Diagnostics and Natural Intelligence, as well as a robust background in cyber security gained from tenure at the IDF, Sarah has consistently driven impactful decision-making and technological advancements throughout their career. Academic credentials include a Master's degree in Condensed Matter Physics from the Weizmann Institute of Science and a Bachelor's degree in Mathematics and Physics from The Hebrew University of Jerusalem. In the episode, Richie and Sarah explore the challenges of data governance, the role of semantic layers in ensuring data trust, the emergence of analytics engineers, the integration of AI in data processes, and much more. Links Mentioned in the Show: EunoConnect with SarahCourse: Responsible AI Data ManagementRelated Episode: How Data Leaders Can Make Data Governance a Priority with Saurabh Gupta, Chief Strategy & Revenue Officer at The Modern Data CompanyRewatch sessions from RADAR: Forward Edition New to DataCamp? Learn on the go using the DataCamp mobile appEmpower your business with world-class data and AI skills with DataCamp for business

The quality of data powers business decisions that drive outcomes. Successful businesses run on trusted data that is reliable and accurate. Join this session to learn how to apply Amazon DataZone and AWS Glue to deliver data integrity and consistency through precise data transformation, data cataloging, data governance, and data lineage, as well as to set up data quality checks, automate validation processes, and manage metadata.

Learn more: AWS re:Invent: https://go.aws/reinvent. More AWS events: https://go.aws/3kss9CP

Subscribe: More AWS videos: http://bit.ly/2O3zS75 More AWS events videos: http://bit.ly/316g9t4

About AWS: Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

AWSreInvent #AWSreInvent2024

Discover cutting-edge data governance strategies with AWS in this exciting customer panel. Learn how leading organizations transform their data management practices to accelerate data-driven decisions while ensuring security and compliance. Hear firsthand from AWS customers about their innovative approaches to automating data integration and quality, curating data to prevent proliferation, and using centralized catalogs to boost data literacy. Explore how these pioneers are tackling emerging trends like generative AI and applying precise permissions for confident data sharing. Don’t miss this chance to gain actionable insights and see real-world examples of successful data governance in action.

Learn more: AWS re:Invent: https://go.aws/reinvent. More AWS events: https://go.aws/3kss9CP

Subscribe: More AWS videos: http://bit.ly/2O3zS75 More AWS events videos: http://bit.ly/316g9t4

About AWS: Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

AWSreInvent #AWSreInvent2024

Data-driven organizations need to ensure that the right data is accessed by the right user for the right purpose—in accordance with the organization’s security regulations—without relying on individual credentials. Join this session to explore strategies for efficiently sharing data across teams and platforms while maintaining security and compliance with AWS analytics services. Learn best practices for managing permissions, encryption, and data governance to ensure secure and efficient data sharing across your organization.

Learn more: AWS re:Invent: https://go.aws/reinvent. More AWS events: https://go.aws/3kss9CP

Subscribe: More AWS videos: http://bit.ly/2O3zS75 More AWS events videos: http://bit.ly/316g9t4

About AWS: Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

AWSreInvent #AWSreInvent2024

Join this session to explore the latest data governance innovations and features in AWS analytics. Our experts guide you through the latest innovations in Amazon DataZone, AWS Lake Formation, and AWS Glue that are helping organizations establish robust data governance frameworks and maintain compliance standards.

Learn more: AWS re:Invent: https://go.aws/reinvent. More AWS events: https://go.aws/3kss9CP

Subscribe: More AWS videos: http://bit.ly/2O3zS75 More AWS events videos: http://bit.ly/316g9t4

About AWS: Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world's most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

AWSreInvent #AWSreInvent2024

🌟 Session Overview 🌟

Session Name: Roadmap to Responsible AI Speaker: Amardeep Lidder Session Description: In this session, Amardeep will share actionable, tried-and-tested approaches to establish effective governance for AI.

There has naturally been a lot of excitement around the potential of AI. However, many conversations are limited to concepts and buzzwords. AI governance is critical in ensuring the responsible and ethical use of AI, as well as driving discovery and innovation.

Speaker will discuss practical examples and learnings based on their successful data governance programs.

Key Takeaways:

Establishing Data Trust: Our framework to measure and improve data quality and governance, ensuring that model inputs and outputs can be trusted. Navigating Regulatory Compliance: Overview of current regulations and considerations for the future. Mitigating Risks: Establishing guardrails to ensure AI tools and systems remain safe and ethical; safeguarding against bias, privacy, and misuse. Balancing Safety and Innovation: How to enable transparent decision-making and explainability to ensure responsible AI use.

🚀 About Big Data and RPA 2024 🚀

Unlock the future of innovation and automation at Big Data & RPA Conference Europe 2024! 🌟 This unique event brings together the brightest minds in big data, machine learning, AI, and robotic process automation to explore cutting-edge solutions and trends shaping the tech landscape. Perfect for data engineers, analysts, RPA developers, and business leaders, the conference offers dual insights into the power of data-driven strategies and intelligent automation. 🚀 Gain practical knowledge on topics like hyperautomation, AI integration, advanced analytics, and workflow optimization while networking with global experts. Don’t miss this exclusive opportunity to expand your expertise and revolutionize your processes—all from the comfort of your home! 📊🤖✨

📅 Yearly Conferences: Curious about the evolution of QA? Check out our archive of past Big Data & RPA sessions. Watch the strategies and technologies evolve in our videos! 🚀 🔗 Find Other Years' Videos: 2023 Big Data Conference Europe https://www.youtube.com/playlist?list=PLqYhGsQ9iSEpb_oyAsg67PhpbrkCC59_g 2022 Big Data Conference Europe Online https://www.youtube.com/playlist?list=PLqYhGsQ9iSEryAOjmvdiaXTfjCg5j3HhT 2021 Big Data Conference Europe Online https://www.youtube.com/playlist?list=PLqYhGsQ9iSEqHwbQoWEXEJALFLKVDRXiP

💡 Stay Connected & Updated 💡

Don’t miss out on any updates or upcoming event information from Big Data & RPA Conference Europe. Follow us on our social media channels and visit our website to stay in the loop!

🌐 Website: https://bigdataconference.eu/, https://rpaconference.eu/ 👤 Facebook: https://www.facebook.com/bigdataconf, https://www.facebook.com/rpaeurope/ 🐦 Twitter: @BigDataConfEU, @europe_rpa 🔗 LinkedIn: https://www.linkedin.com/company/73234449/admin/dashboard/, https://www.linkedin.com/company/75464753/admin/dashboard/ 🎥 YouTube: http://www.youtube.com/@DATAMINERLT

🌟 Session Overview 🌟

Session Name: Artificial Intelligence is Nothing without Data Governance Speaker: Philippe Nieuwbourg Session Description: Since the US administration was ordered to appoint Chief Artificial Intelligence Officers by the end of May 2024, major companies around the world understand that AI cannot be deployed without oversight. In Europe, the AI Act provides a framework, and several ISO standards also exist. However, the priority of this framework work is to prepare its data. Yet many companies still believe that AI is magic. Without good quality, compliant, and referenced data, it's impossible to implement effective AI tools. Worse still, this leads to errors, discrimination, and hallucinations, which can be costly for the company. We'll find out what's involved in the governance of artificial intelligence, and how Chief Data Officers can add this skill to their roles today.

🚀 About Big Data and RPA 2024 🚀

Unlock the future of innovation and automation at Big Data & RPA Conference Europe 2024! 🌟 This unique event brings together the brightest minds in big data, machine learning, AI, and robotic process automation to explore cutting-edge solutions and trends shaping the tech landscape. Perfect for data engineers, analysts, RPA developers, and business leaders, the conference offers dual insights into the power of data-driven strategies and intelligent automation. 🚀 Gain practical knowledge on topics like hyperautomation, AI integration, advanced analytics, and workflow optimization while networking with global experts. Don’t miss this exclusive opportunity to expand your expertise and revolutionize your processes—all from the comfort of your home! 📊🤖✨

📅 Yearly Conferences: Curious about the evolution of QA? Check out our archive of past Big Data & RPA sessions. Watch the strategies and technologies evolve in our videos! 🚀 🔗 Find Other Years' Videos: 2023 Big Data Conference Europe https://www.youtube.com/playlist?list=PLqYhGsQ9iSEpb_oyAsg67PhpbrkCC59_g 2022 Big Data Conference Europe Online https://www.youtube.com/playlist?list=PLqYhGsQ9iSEryAOjmvdiaXTfjCg5j3HhT 2021 Big Data Conference Europe Online https://www.youtube.com/playlist?list=PLqYhGsQ9iSEqHwbQoWEXEJALFLKVDRXiP

💡 Stay Connected & Updated 💡

Don’t miss out on any updates or upcoming event information from Big Data & RPA Conference Europe. Follow us on our social media channels and visit our website to stay in the loop!

🌐 Website: https://bigdataconference.eu/, https://rpaconference.eu/ 👤 Facebook: https://www.facebook.com/bigdataconf, https://www.facebook.com/rpaeurope/ 🐦 Twitter: @BigDataConfEU, @europe_rpa 🔗 LinkedIn: https://www.linkedin.com/company/73234449/admin/dashboard/, https://www.linkedin.com/company/75464753/admin/dashboard/ 🎥 YouTube: http://www.youtube.com/@DATAMINERLT

🌟 Session Overview 🌟

Session Name: From Zero to Hero: How Decentralized Data Products are Changing GEMA from Inside Out Speaker: Martin Zuern, Markus Zachai Session Description: In a data-driven era, organizations are challenged to efficiently refine this valuable resource. At GEMA, using self-service data platforms, lakehouse architecture, and data mesh principles, we embarked on a transformational journey. Our approach has transformed the organization from the inside out, rooted in lean governance and decentralized ownership. In the first ten months, more than 100 data products have been created, with over 40% of the workforce actively using the platform.

Join us as we explore the challenges and solutions encountered while implementing data mesh and governance. We'll delve into the intricacies of our data journey, from technical hurdles to organizational mindset shifts. We'll also look at growth hacking strategies and the critical role of the data governance manager, a position that is often misunderstood in the governance setup.

Discover how GEMA's data journey has led to exciting use cases and valuable insights, and what you can take away for your own organization. 🚀 About Big Data and RPA 2024 🚀

Unlock the future of innovation and automation at Big Data & RPA Conference Europe 2024! 🌟 This unique event brings together the brightest minds in big data, machine learning, AI, and robotic process automation to explore cutting-edge solutions and trends shaping the tech landscape. Perfect for data engineers, analysts, RPA developers, and business leaders, the conference offers dual insights into the power of data-driven strategies and intelligent automation. 🚀 Gain practical knowledge on topics like hyperautomation, AI integration, advanced analytics, and workflow optimization while networking with global experts. Don’t miss this exclusive opportunity to expand your expertise and revolutionize your processes—all from the comfort of your home! 📊🤖✨

📅 Yearly Conferences: Curious about the evolution of QA? Check out our archive of past Big Data & RPA sessions. Watch the strategies and technologies evolve in our videos! 🚀 🔗 Find Other Years' Videos: 2023 Big Data Conference Europe https://www.youtube.com/playlist?list=PLqYhGsQ9iSEpb_oyAsg67PhpbrkCC59_g 2022 Big Data Conference Europe Online https://www.youtube.com/playlist?list=PLqYhGsQ9iSEryAOjmvdiaXTfjCg5j3HhT 2021 Big Data Conference Europe Online https://www.youtube.com/playlist?list=PLqYhGsQ9iSEqHwbQoWEXEJALFLKVDRXiP

💡 Stay Connected & Updated 💡

Don’t miss out on any updates or upcoming event information from Big Data & RPA Conference Europe. Follow us on our social media channels and visit our website to stay in the loop!

🌐 Website: https://bigdataconference.eu/, https://rpaconference.eu/ 👤 Facebook: https://www.facebook.com/bigdataconf, https://www.facebook.com/rpaeurope/ 🐦 Twitter: @BigDataConfEU, @europe_rpa 🔗 LinkedIn: https://www.linkedin.com/company/73234449/admin/dashboard/, https://www.linkedin.com/company/75464753/admin/dashboard/ 🎥 YouTube: http://www.youtube.com/@DATAMINERLT