We have a similar pattern of DAGs running for different data quality dimensions like accuracy, timeliness, & completeness. To do this again and again, we would be duplicating and potentially introducing human error while doing copy paste of code or making people write same code again. To solve for this, we are doing few things: Run DAGs via DagFactory to dynamically generate DAGs using just some YAML code for all the steps we want to run in our DQ checks. Hide this behind a UI which is hooked to github PR open step, now the user just provides some inputs or selects from dropdown in UI and a YAML DAG is generated for them. This highlights the potential for DAGFactory to hide Airflow Python code from users and make it more accessible to Data Analysts and Business Intelligence along with normal Software Engg, along with reducing human error. YAML is the perfect format to be able to generate code, create a PR and DagFactory is the perfect fir for that. All of this is running in GCP Cloud Composer.

There have been lots of social media posts declaring things to be dead - SQL, R, data engineering, BI, etc.

I give my thoughts on these proclamations, why it's a wrong way to think about our space, and more.

Discover best practices to scale Power BI effectively—improving performance, governance, and self-service across enterprise environments. Published at: https://datalere.com/articles/power-bi-best-practices-from-the-field-lessons-from-enterprise-deployments

Welcome to the Data & AI NXT Conference! 🎉 This year, we explore the next frontier in analytics: Agentic AI.

🔍 Next-Generation Agentic Analytics Artificial Intelligence is pushing analytics beyond static dashboards and reports. At this event, discover how next-gen AI Agents transform fragmented, siloed data, both historical and real-time, into optimized, actionable intelligence.

Learn how businesses are evolving from reactive analytics to self-improving decision systems that span the entire enterprise.

🗓️ Agenda & Chapters

0:00 Start 7:37 Opening 23:15 The Unseen Sportian’s Playbook: Redefining Sports through Data and AI | Leandro Mora 1:06:16 Ethical implications of self-driving intelligence | Avijeet Dutta, Dr Shivani Rai Gupta, Jyothish Jayaraman, and Andres Tenorio 1:59:54 Governance in the age of AI Agents | Roberto Contreras 2:48:11 Synthetic data, digital twins & the future of testing | Carla Molgora, Ana Lía Villarreal and Cristina Garita 3:44:34 Future of BI: from dashboards to autonomous intelligence | Nacho Vuotto, Esteban Bertuccio, Carlos Alarcón, and Sergio Soliz 4:43:02 Real-Time vs. historical: balancing speed and context | Daniel Esteban Vesga, Oscar Narvaez, Martin Sciarrillo, and Abraham Jacob Montoya 5:38:16 AI Agents and the future of Human-Tech | Almudena Claudio

🙌 Thanks for joining us! Don't forget to like, comment, and subscribe for more tech insights from Globant.

💚

This comprehensive guide is designed to address the most frequent and challenging issues faced by users of Power Query, a powerful data transformation tool integrated into Excel, Power BI, and Microsoft Azure. By tackling 96 real-world problems with practical, step-by-step solutions, this book is an essential resource for data analysts, Excel enthusiasts, and Power BI professionals. It aims to enhance your data transformation skills and improve efficiency in handling complex data sets. Structured into 12 chapters, the book covers specific areas of Power Query such as data extraction, referencing, column splitting and merging, sorting and filtering, and pivoting and unpivoting tables. You will learn to combine data from Excel files with varying column names, handle multi-row headers, perform advanced filtering, and manage missing values using techniques such as linear interpolation and K-nearest neighbors (K-NN) imputation. The book also dives into advanced Power Query functions such as Table.Group, List.Accumulate, and List.Generate, explored through practical examples such as calculating running totals and implementing complex grouping and iterative processes. Additionally, it covers crucial topics such as error-handling strategies, custom function creation, and the integration of Python and R with Power Query. In addition to providing explanations on the use of functions and the M language for solving real-world challenges, this book discusses optimization techniques for data cleaning processes and improving computational speed. It also compares the execution time of functions across different patterns and proposes the optimal approach based on these comparisons. In today’s data-driven world, mastering Power Query is crucial for accurate and efficient data processing. But as data complexity grows, so do the challenges and pitfalls that users face. This book serves as your guide through the noise and your key to unlocking the full potential of Power Query. You’ll quickly learn to navigate and resolve common issues, enabling you to transform raw data into actionable insights with confidence and precision. What You Will Learn Master data extraction and transformation techniques for various Excel file structures Apply advanced filtering, sorting, and grouping methods to organize and analyze data Leverage powerful functions such as Table.Group, List.Accumulate, and List.Generate for complex transformations Optimize queries to execute faster Create and utilize custom functions to handle iterative processes and advanced list transformation Implement effective error-handling strategies, including removing erroneous rows and extracting error reasons Customize Power Query solutions to meet specific business needs and share custom functions across files Who This Book Is For Aspiring and developing data professionals using Power Query in Excel or Power BI who seek practical solutions to enhance their skills and streamline complex data transformation workflows

Metadata, data quality and data observability tools provide significant capabilities to ensure good data for your BI and AI initiatives. Metadata tools help discover, and inventory your data assets. Data quality tools help business users manage their data at sources by setting rules and policies. Data observability tools give organizations integrated visibility over the health of data, data pipeline and data landscape. Together the tools help organizations lay good foundation in data management for BI and AI initiatives.

This comprehensive guide book equips you with the knowledge and confidence needed to prep for the exam and thrive as a Power Platform Solution Architect. The book starts with a foundation for successful solution architecture, emphasizing essential skills such as requirements gathering, governance, and security. You will learn to navigate customer discovery, translate business needs into technical requirements, and design solutions that address both functional and non-functional needs. The second part of the book delves into the Microsoft Power Platform ecosystem, offering an in-depth look at its core components—Power Apps, Power Automate, Power BI, Microsoft Copilot, and Robotic Process Automation (RPA). Detailed insights into data modeling, security strategies, and AI integration will guide you in building scalable, secure solutions. Coverage of application life cycle management, which empowers solution architects to design, implement, and deploy Power Platform solutions effectively, is discussed next. You will then go through real-world scenarios, giving you a practical understanding of the challenges and considerations in managing Power Platform projects within a business context. The book concludes with strategies for continuous learning and resources for professional development, including practice questions to assess knowledge and readiness for the PL-600 exam. After reading the book, you will be ready to take the exam and become a successful Power Platform Solution Architect. What You Will Learn Understand the Solution Architect's role, responsibilities, and strategic approaches to successfully navigate projects Master the basics of Power Platform Solution Architecture Understand governance, security, and integration concepts in real-world scenarios Design and deploy effective business solutions using Power Platform components Gain the skills necessary to prep for the PL-600 certification exam Who This Book Is For Professionals pursuing Microsoft PL-600 Solution Architect certification and IT consultants and developers transitioning to solution architect roles

Generative AI is taking the world by storm. What happens when it meets its structured, fact-driven soulmate, the relational knowledge graph? We explore how combining GenAI with knowledge graphs creates a dynamic duo: AI that's not just creative, but also accurate, explainable, and deeply informed. We'll dive into real-world use cases, from chatbots to BI. Expect insights, practical applications, and a few AI-generated jokes (with fact-checking by our knowledge graph, of course!)

Self-service analytics is now as intuitive as a conversation with your data. With Snowflake Cortex AI, users across industries can extract insights effortlessly using generative AI — no advanced expertise required. Learn how Northeast Georgia Health System boosts efficiency with AI-driven insights, and see how Cortex AI powers real-world applications, automates workflows and enhances decision-making. Join us to explore the future of automated analytics.

D&A value is not possible without data storytelling that offers a better way to engage communication findings than just BI reporting or data science notebooks. Join this session to know about the fundamentals of data storytelling and how to fill the gap between data science speakers and decision makers. It further discusses how to tell the best data storytelling and how to upscale data storytelling for future in landscape of GenAI.

D&A value is not possible without data storytelling that offers a better way to engage communication findings than just BI reporting or data science notebooks. Join this session to know about the fundamentals of data storytelling and how to fill the gap between data science speakers and decision makers. It further discusses how to tell the best data storytelling and how to upscale data storytelling for future in landscape of GenAI.

Analytics is experiencing another monumental change. Just as visual drag and drop BI tools and augmented insights led to changes in analytics delivery, we now experience conversational interfaces, automated workflows and AI agents that cause us to rethink how analytics will be done. Join this session to learn the new technologies that are making an impact and how this will affect plans for future investment in analytics tools, platforms and solutions.

The modern data stack has transformed how organizations work with data, but are our BI tools keeping pace with these changes? As data schemas become increasingly fluid and analysis needs range from quick explorations to production-grade reporting, traditional approaches are being challenged. How can we create analytics experiences that accommodate both casual spreadsheet users and technical data modelers? With semantic layers becoming crucial for AI integration and data governance growing in importance, what skills do today's BI professionals need to master? Finding the balance between flexibility and governance is perhaps the greatest challenge facing data teams today. Colin Zima is the Co-Founder and CEO of Omni, a business intelligence platform focused on making data more accessible and useful for teams of all sizes. Prior to Omni, he was Chief Analytics Officer and VP of Product at Looker, where he helped shape the product and data strategy leading up to its acquisition by Google for $2.6 billion. Colin’s background spans roles in data science, analytics, and product leadership, including positions at Google, HotelTonight, and as founder of the restaurant analytics startup PrimaTable. He holds a degree in Operations Research and Financial Engineering from Princeton University and began his career as a Structured Credit Analyst at UBS. In the episode, Richie and Colin explore the evolution of BI tools, the challenges of integrating casual and rigorous data analysis, the role of semantic layers, and the impact of AI on business intelligence. They discuss the importance of understanding business needs, creating user-focused dashboards, and the future of data products, and much more. Links Mentioned in the Show: OmniConnect with ColinSkill Track: Design in Power BIRelated Episode: Self-Service Business Intelligence with Sameer Al-Sakran, CEO at MetabaseRegister for RADAR AI - June 26 New to DataCamp? Learn on the go using the DataCamp mobile appEmpower your business with world-class data and AI skills with DataCamp for business

The insurance industry is at the crossroads of digital transformation, facing challenges from market competition and customer expectations. While conventional ML applications have historically provided capabilities in this domain, the emergence of Agentic AI frameworks presents a revolutionary opportunity to build truly autonomous insurance applications. We will address issues related to data governance and quality while discussing how to monitor/evaluate fine-tune models. We'll demonstrate the application of the agentic framework in the insurance context and how these autonomous agents can work collaboratively to handle complex insurance workflows — from submission intake and risk evaluation to expedited quote generation. This session demonstrates how to architect intelligent insurance solutions using Databricks Mosaic AI agentic core components including Unity Catalog, Playground, model evaluation/guardrails, privacy filters, AI functions and AI/BI Genie.

I have metrics, you have metrics — we all have metrics. But the real problem isn’t having metrics, it’s that the numbers never line up, leading to endless cycles of reconciliation and confusion. Join us as we share how our Data Team at Databricks tackled this fundamental challenge in Business Intelligence by building an internal Metric Store — creating a single source of truth for all business metrics using the newly-launched UC Metric Views. Imagine a world where numbers always align, metric definitions are consistently applied across the organization and every metric comes with built-in ML-based forecasting, AI-powered anomaly detection and automatic explainability. That’s the future we’ve built — and we’ll show you how you can get started today.

Discover how the latest innovations in Databricks AI/BI Dashboards and Genie are transforming self-service analytics. This session offers a high-level tour of new capabilities that empower business users to ask questions in natural language, generate insights faster and make smarter decisions. Whether you're a long-time Databricks user or just exploring what's possible with AI/BI, you'll walk away with a clear understanding of how these tools are evolving — and how to leverage them for greater business impact.

Bringing AI/BI to every business user starts with getting security, access and governance right. In this session, we’ll walk through the latest best practices for configuring Databricks accounts, setting up workspaces, and managing authentication protocols to enable secure and scalable onboarding. Whether you're supporting a small team or an entire enterprise, you'll gain practical insights to protect your data while ensuring seamless and governed access to AI/BI tools.

Databricks SQL has added significant features in the last year at a fast pace. This session will share the most impactful features and the customer use cases that inspired them. We will highlight the new SQL editor, SQL coding features, streaming tables and materialized views, BI integrations, cost management features, system tables and observability features, and more. We will also share AI-powered performance optimizations.

AI/BI Genie lets anyone simply talk with their own data, using natural language, fully secured through UC to provide accurate answers within the context for your organization. AI/BI Dashboards goes beyond traditional BI tools, democratizing everyone to self-serve immediate interactive visuals on your own secured data. Hear from a customer and Databricks experts on the latest developments.

Every analytics, BI and AI project relies on high-quality data. This is why data engineering, the practice of building reliable data pipelines that ingest and transform data, is consequential to the success of these projects. In this session, we'll show how you can use Lakeflow to accelerate innovation in multiple parts of the organization. We'll review real-world examples of Databricks customers using Lakeflow in different industries such as automotive, healthcare and retail. We'll touch on how the foundational data engineering capabilities Lakeflow provides help power initiatives that improve customer experiences, make real-time decisions and drive business results.