The industry has witnessed some tectonic changes over the last few years: on prem to cloud to multi-cloud, BI to AI to GenAI, and data warehouses to data lakes to data lakehouses, to name a few. This constant evolution coupled with the ever-increasing demands of the business makes platform thinking crucial in order to ensure a future-proof infrastructure. As companies race to advance their AI strategies, Dell has seen a gravitational pull towards a modern data architecture that can create high quality data to feed AI and generate high quality outcomes. Join this session to learn about how the Dell Data Lakehouse, powered by Starburst, is the modern paradigm for this new era. You’ll learn about the investments Dell is making in data, analytics, and AI, why Dell and Starburst partnered up on this solution, and how it enables a tremendously powerful yet open and flexible data architecture.

A good BI tool should integrate with a larger ecosystem, not be a monolith. At Hashboard we’ve built Hashquery, a new Python SDK which we believe connects the dots. Import models from your semantic layer of choice, express complex queries with ease, and push the result wherever you need it. In this talk we’ll explore how.

In this game you will learn to build a BI dashboard with Looker Studio as the front end, powered by BigQuery on the back end, learn to use BigQuery to find data, build a time series model to forecast demand of multiple products using BigQuery ML, and create a basic report in Google Data Studio.

Click the blue “Learn more” button above to tap into special offers designed to help you implement what you are learning at Google Cloud Next 25.

Discover how to build a customized, AI-powered Business Intelligence solution.In this hands-on workshop, you'll learn to integrate Looker, Gemini and Vertex AI, to: - Generate queries and visualizations effortlessly using natural language with Looker's AI Explore Assistant. - Extract instant insights from dashboards with AI-generated summaries and actions. - Understand the powered by Looker platform to build new data experiences with leveraging the entire Google Cloud ecosystem Join us and learn to architect innovative, AI-powered analytics with Looker and Vertex AI!

Click the blue “Learn more” button above to tap into special offers designed to help you implement what you are learning at Google Cloud Next 25.

Gemini in Looker has made the most complex data actions simple, letting you chat with your business data. The latest advancements in Google's business intelligence (BI) suite bring insights to your users and customers, and form the basis of your own data-driven applications. This session will show you how we are building the future of BI with AI at the center, and keeping our focus on an open ecosystem that enables you to bring all your important data and share it with your teams – all driven by generative AI.

Click the blue “Learn more” button above to tap into special offers designed to help you implement what you are learning at Google Cloud Next 25.

Uncover how Accenture leads the charge in harnessing Google's cutting-edge Generative AI technologies to develop innovative client solutions. This session features a captivating live demo that highlights the game-changing potential of Natural Language Search and Q&A functionalities within a client portal. Explore how these tools navigate through technical documentation, incidents, and threat intelligence, providing unparalleled insights into business operations and critical incident management. Witness how Accenture is not just keeping pace with the AI revolution but actively shaping its trajectory, offering a glimpse into the future of business intelligence and operational efficiency.

By attending this session, your contact information may be shared with the sponsor for relevant follow up for this event only.

Click the blue “Learn more” button above to tap into special offers designed to help you implement what you are learning at Google Cloud Next 25.

With the surge of new generative AI capabilities, companies and their customers can now interact with systems and data in new ways. To activate AI organizations require a data foundation with the scale and efficiency to bring business data together with AI models and ground them in customer reality. Join this session to learn the latest innovations for data analytics and BI, and why tens of thousands of organizations are fueling their journey with BigQuery and Looker.

Click the blue “Learn more” button above to tap into special offers designed to help you implement what you are learning at Google Cloud Next 25.

Install data analytics into your brain with this comprehensive introduction Data Analytics & Visualization All-in-One For Dummies collects the essential information on mining, organizing, and communicating data, all in one place. Clocking in at around 850 pages, this tome of a reference delivers eight books in one, so you can build a solid foundation of knowledge in data wrangling. Data analytics professionals are highly sought after these days, and this book will put you on the path to becoming one. You’ll learn all about sources of data like data lakes, and you’ll discover how to extract data using tools like Microsoft Power BI, organize the data in Microsoft Excel, and visually present the data in a way that makes sense using a Tableau. You’ll even get an intro to the Python, R, and SQL coding needed to take your data skills to a new level. With this Dummies guide, you’ll be well on your way to becoming a priceless data jockey. Mine data from data sources Organize and analyze data Use data to tell a story with Tableau Expand your know-how with Python and R New and novice data analysts will love this All-in-One reference on how to make sense of data. Get ready to watch as your career in data takes off.

Summary

Maintaining a single source of truth for your data is the biggest challenge in data engineering. Different roles and tasks in the business need their own ways to access and analyze the data in the organization. In order to enable this use case, while maintaining a single point of access, the semantic layer has evolved as a technological solution to the problem. In this episode Artyom Keydunov, creator of Cube, discusses the evolution and applications of the semantic layer as a component of your data platform, and how Cube provides speed and cost optimization for your data consumers.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management This episode is brought to you by Datafold – a testing automation platform for data engineers that prevents data quality issues from entering every part of your data workflow, from migration to dbt deployment. Datafold has recently launched data replication testing, providing ongoing validation for source-to-target replication. Leverage Datafold's fast cross-database data diffing and Monitoring to test your replication pipelines automatically and continuously. Validate consistency between source and target at any scale, and receive alerts about any discrepancies. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold. Dagster offers a new approach to building and running data platforms and data pipelines. It is an open-source, cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability, a declarative programming model, and best-in-class testability. Your team can get up and running in minutes thanks to Dagster Cloud, an enterprise-class hosted solution that offers serverless and hybrid deployments, enhanced security, and on-demand ephemeral test deployments. Go to dataengineeringpodcast.com/dagster today to get started. Your first 30 days are free! Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics. Trusted by teams of all sizes, including Comcast and Doordash, Starburst is a data lake analytics platform that delivers the adaptability and flexibility a lakehouse ecosystem promises. And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Want to see Starburst in action? Go to dataengineeringpodcast.com/starburst and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino. Your host is Tobias Macey and today I'm interviewing Artyom Keydunov about the role of the semantic layer in your data platform

Interview

Introduction How did you get involved in the area of data management? Can you start by outlining the technical elements of what it means to have a "semantic layer"? In the past couple of years there was a rapid hype cycle around the "metrics layer" and "headless BI", which has largely faded. Can you give your assessment of the current state of the industry around the adoption/implementation of these concepts? What are the benefits of having a discrete service that offers the business metrics/semantic mappings as opposed to implementing those concepts as part of a more general system? (e.g. dbt, BI, warehouse marts, etc.)

At what point does it become necessary/beneficial for a team to adopt such a service? What are the challenges involved in retrofitting a semantic layer into a production data system?

evolution of requirements/usage patterns technical complexities/performance and cost optimization What are the most interesting, innovative, or unexpected ways that you have seen Cube used? What are the most interesting, unexpec

Matt Turck has been publishing his ecosystem map since 2012. It was first called the Big Data Landscape. Now it's the Machine Learning, AI & Data (MAD) Landscape.  The 2024 MAD Landscape includes 2,011(!) logos, which Matt attributes first a data infrastructure cycle and now an ML/AI cycle. As Matt writes, "Those two waves are intimately related. A core idea of the MAD Landscape every year has been to show the symbiotic relationship between data infrastructure, analytics/BI,  ML/AI, and applications." Matt and Tristan discuss themes in Matt's post: generative AI's impact on data analytics, the modern AI stack compared to the modern data stack, and Databricks vs. Snowflake (plus Microsoft Fabric). For full show notes and to read 7+ years of back issues of the podcast's companion newsletter, head to https://roundup.getdbt.com. The Analytics Engineering Podcast is sponsored by dbt Labs.

The Complete Power BI Interview Guide is your companion to mastering Power BI roles and acing data analyst interviews. With hands-on skills, expert tips, and targeted preparation strategies, this resource equips you to excel in interviews and certifications while navigating the competitive job market. What this Book will help me do Create a powerful professional brand to optimize your resume and online presence. Master essential Power BI skills including data modeling, DAX programming, and visualization. Prepare effectively for interviews with industry-relevant questions, answers, and insights. Gain an edge in the market by understanding hiring procedures and negotiation tactics. Develop comprehensive analytics solutions exemplified with real-world case studies. Author(s) Sandielly Ortega Polanco, Gogula Aryalingam, and Abu Bakar Nisar Alvi bring years of collective experience in data analytics, Power BI, and career mentorship. Their insights are drawn from extensive professional practice and their passion for empowering future data analysts. Together, they provide an approachable and practical guide to securing roles in the competitive landscape of data analytics. Who is it for? This book is ideal for aspiring data analysts, business intelligence developers, or those shifting into Power BI roles who wish to enhance their knowledge and refine their strategies for interview success. It speaks to both newcomers to the field and seasoned professionals aiming to elevate their expertise.

Advance your Power BI skills by adding AI to your repertoire at a practice level. With this practical book, business-oriented software engineers and developers will learn the terminologies, practices, and strategy necessary to successfully incorporate AI into your business intelligence estate. Jen Stirrup, CEO of AI and BI leadership consultancy Data Relish, and Thomas Weinandy, research economist at Upside, show you how to use data already available to your organization. Springboarding from the skills that you already possess, this book adds AI to your organization's technical capability and expertise with Microsoft Power BI. By using your conceptual knowledge of BI, you'll learn how to choose the right model for your AI work and identify its value and validity. Use Power BI to build a good data model for AI Demystify the AI terminology that you need to know Identify AI project roles, responsibilities, and teams for AI Use AI models, including supervised machine learning techniques Develop and train models in Azure ML for consumption in Power BI Improve your business AI maturity level with Power BI Use the AI feedback loop to help you get started with the next project

Creating a culture of continuous learning within analytics functions isn't just beneficial; it's essential. In the session, Russell Johnson, Chief Data Scientist at Marks & Spencer, Denisse Groenendaal-Lopez, Learning & Development Business Partner at Booking Group, and Mark Stern, VP of Business Intelligence & Analytics at BetMGM will address the importance of fostering a learning environment for driving success with analytics. They will provide insights on developing a culture where continuous learning, experimentation, and curiosity are the norms—and strategies leaders can adopt today to drive up excitement around analytics adoption & upskilling. 

In "Extending Power BI with Python and R," you'll learn how to enhance your Power BI reports and analyses by leveraging the advanced analytical capabilities of Python and R. From working with large datasets to creating sophisticated visuals, this book provides practical instructions on powerful techniques that unlock new possibilities in Power BI. What this Book will help me do Configure and optimize Python and R integration in Power BI for enhanced performance. Implement advanced data transformation techniques to overcome Power BI limitations. Develop advanced visualizations using the Grammar of Graphics in Python and R. Analyze data leveraging powerful Python and R algorithms, including machine learning models. Secure your Power BI data with anonymization and pseudonymization techniques. Author(s) None Zavarella is a data analytics expert with years of practical experience in business intelligence and data analytics. With a passion for enhancing data tools with programming languages like Python and R, they bring practical knowledge and technical acumen to this comprehensive resource. They aim to make complex concepts approachable to their readers. Who is it for? This book is aimed at professionals such as business analysts, business intelligence specialists, and data scientists who leverage Power BI for their data solutions. Readers should have a working knowledge of Power BI basics and a desire to extend its capabilities. A familiarity with Python and R programming basics is also beneficial for following the advanced techniques presented.

Dive into the comprehensive world of data transformation with "The Definitive Guide to Power Query (M)". This book empowers you with the knowledge and skills necessary to effectively utilize Power Query for complex data transformation tasks. You will develop expertise in practical techniques, advanced M language concepts, and optimization strategies. What this Book will help me do Understand the fundamentals of Power Query and its functionalities. Learn to perform complex data transformations using various Power Query functions. Gain insight into advanced M language structures such as custom functions and nested expressions. Develop skills in error handling and debugging to streamline your data processes. Master performance optimization techniques for efficient data handling with Power Query. Author(s) Gregory Deckler, Rick de Groot, and Melissa de Korte are seasoned professionals in business intelligence and data analytics. With years of experience using Power Query, they bring a wealth of knowledge and practical insight into tackling real-world data problems. Their combined expertise ensures a clear and immersive learning experience for readers, guiding them through fundamental to advanced topics. Who is it for? This book is ideal for business analysts, data professionals, and power users who wish to advance their data transformation capabilities. If you're someone with foundational experience in Power Query looking to become proficient or an industry professional aiming to optimize workflows, this book is tailored to suit your goals.

Send us a text Welcome to the cozy corner of the tech world where ones and zeros mingle with casual chit-chat. Datatopics Unplugged is your go-to spot for relaxed discussions around tech, news, data, and society. Dive into conversations that should flow as smoothly as your morning coffee (but don't), where industry insights meet laid-back banter. Whether you're a data aficionado or just someone curious about the digital age, pull up a chair, relax, and let's get into the heart of data, unplugged style! In this episode #42, titled "Unraveling the Fabric of Data: Microsoft's Ecosystem and Beyond," we're joined once again by the tech maestro and newly minted Microsoft MVP, Sam Debruyn. Sam brings to the table a bevy of updates from his recent accolades to the intricacies of Microsoft's data platforms and the world of SQL.

Biz Buzz: From Reddit's IPO to the performance versus utility debate in database selection, we dissect the big moves shaking up the business side of tech. Read about Reddit's IPO.Microsoft's Fabric Unraveled: Get the lowdown on Microsoft's Fabric, the one-stop AI platform, as Sam Debruyn gives us a deep dive into its capabilities and integration with Azure Databricks and Power BI. Discover more about Fabric and dive into Sam's blog.dbt Developments: Sam talks dbt and the exciting new SQL tool for data pipeline building with upcoming unit testing capabilities.Polaris Project: Delving into Microsoft's internal storage projects, including insights on Polaris and its integration with Synapse SQL. Read the paper here.AI Advances: From the release of Grok-1 and Apple's MM1 AI model to GPT-4's trillion parameters, we discuss the leaps in artificial intelligence.Stability in Motion: After OpenAI's Sora, we look at Stability AI's new venture into motion with Stable Video. Check out Stable Video.Benchmarking Debate: A critical look at performance benchmarks in database selection and the ongoing search for the 'best' database. Contemplate benchmarking perspectives.Versioning Philosophy: Hot takes on semantic versioning and what stability really means in software development. Dive into Semantic Versioning.

Generative AI has made a mark everywhere, including BI platforms, but how can you combine AI and BI together? What effects can this have across organizations? With constituent aspects such as data quality, your AI strategy, and the specific use-case you’re trying to solve, it’s important to get the full picture and tread with intent. What are the subtleties that we need to get right in order for this marriage to work to its full potential? Nick Magnuson is the Head of AI at Qlik, executing the organization’s AI strategy, solution development, and innovation. Prior to Qlik, Nick was the CEO of Big Squid, which was acquired by Qlik in 2021. Nick has previously held executive roles in customer success, product, and engineering in the field of machine learning and predictive analytics. As a practitioner in this field for over 20 years, Nick has published original research in these areas, as well as cognitive bias and other quantitative topics. He has also served as an advisor to other analytics platforms and start-ups. A long-time investment professional, Nick continues to hold his Chartered Financial Analyst designation and is a past member of the Chicago Quantitative Alliance and Society of Quantitative Analysts.  In the episode, Richie and Nick explore what Qlik offers, including products like Sense and Staige, how Staige uses AI to enhance customer capabilities, use cases of generative AI, advice on data privacy and security when using AI, data quality and its effect on the success of AI tools, AI strategy and leadership, how data roles are changing and the emergence of new positions, and much more. 

Links Mentioned in the Show: QlikQlik StaigeQlik Sense[Skill Track] AI FundamentalsRelated Episode: Adapting to the AI Era with Jason Feifer, Editor in Chief of Entrepreneur MagazineSign up to RADAR: The Analytics Edition

New to DataCamp? Learn on the go using the DataCamp mobile app Empower your business with world-class data and AI skills with DataCamp for business

This week on Experiencing Data, something new as promised at the beginning of the year. Today, I’m exploring the world of embedded analytics with Zalak Trivedi from Sigma Computing—and this is also the first approved Promoted Episode on the podcast. In today’s episode, Zalak shares his journey as the product lead for Sigma’s embedded analytics and reporting solution which seeks to accelerate and simplify the deployment of decision support dashboards to their SAAS companies’ customers. Right there, we have the first challenge that Zalak was willing to dig into with me: designing a platform UX when we have multiple stakeholder and user types. In Sigma’s case, this means Sigma’s buyers, the developers that work at these SAAS companies to integrate Sigma into their products, and then the actual customers of these SAAS companies who will be the final end users of the resulting dashboards.  also discuss the challenges of creating products that serve both beginners and experts and how AI is being used in the BI industry.  

Highlights/ Skip to:

I introduce Zalak Trivedi from Sigma Computing onto the show (03:15) Zalak shares his journey leading the vision for embedded analytics at Sigma and explains what Sigma looks like when implemented into a customer’s SAAS product . (03:54) Zalak and I discuss the challenge of integrating Sigma's analytics into various companies' software, since they need to account for a variety of stakeholders. (09:53) We explore Sigma's team approach to user experience with product management, design, and technical writing (15:14) Zalak reveals how Sigma leverages telemetry to understand and improve user interactions with their products (19:54) Zalak outlines why Sigma is a faster and more supportive alternative to building your own analytics (27:21) We cover data monetization, specifically looking at how SAAS companies can monetize analytics and insights (32:05) Zalak highlights how Sigma is integratingAI into their BI solution (36:15) Zalak share his customers' current pain points and interests (40:25)  We wrap up with final thoughts and ways to connect with Zalak and learn more about Sigma (49:41) 

Quotes from Today’s Episode "Something I’m really excited about personally that we are working on is [moving] beyond analytics to help customers build entire data applications within Sigma. This is something we are really excited about as a company, and marching towards [achieving] this year." - Zalak Trivedi (04:04)

“The whole point of an embedded analytics application is that it should look and feel exactly like the application it’s embedded in, and the workflow should be seamless.” - Zalak Trivedi (09:29) 

“We [at Sigma] had to switch the way that we were thinking about personas. It was not just about the analysts or the data teams; it was more about how do we give the right tools to the [SAAS] product managers and developers to embed Sigma into their product.” - Zalak Trivedi (11:30)  “You can’t not have a design, and you can’t not have a user experience. There’s always an experience with every tool, solution, product that we use, whether it emerged organically as a byproduct, or it was intentionally created through knowledge data... it was intentional” - Brian O’Neill (14:52) 

“If we find that [in] certain user experiences,people are tripping up, and they’re not able to complete an entire workflow, we flag that, and then we work with the product managers, or [with] our customers essentially, and figure out how we can actually simplify these experiences.” - Zalak Trivedi (20:54)

“We were able to convince many small to medium businesses and startups to sign up with Sigma. The success they experienced after embedding Sigma was tremendous. Many of our customers managed to monetize their existing data within weeks, or at most, a couple of months, with lean development teams of two to three developers and a few business-side personnel, generating seven-figure income streams from that.” - Zalak Trivedi (32:05)

“At Sigma, our stance is, let’s not just add AI for the sake of adding AI. Let’s really identify [where] in the entire user journey does the intelligence really lie, and where are the different friction points, and let’s enhance those experiences.” - Zalak Trivedi (37:38)  “Every time [we at Sigma Computing] think about a new feature or functionality, we have to ensure it works for both the first-degree persona and the second-degree persona, and consider how it will be viewed by these different personas, because that is not the primary persona for which the foundation of the product was built." - Zalak Trivedi (48:08)

Links Sigma Computing: https://sigmacomputing.com

Email: [email protected] 

LinkedIn: https://www.linkedin.com/in/trivedizalak/

Sigma Computing Embedded: https://sigmacomputing.com/embedded

About Promoted Episodes on Experiencing Data: https://designingforanalytics.com/promoted

Despite the critical role of analytics in guiding business decisions, organizations continue to face significant challenges in harnessing its full potential. As data sets expand and deadlines shrink, the urgency to scale analytics processes becomes paramount. What data leaders now need to focus on are essential strategies for analytics at scale, including fostering a culture of continuous learning, prioritizing data governance, and leveraging generative AI. Libby Duane Adams is the Chief Advocacy Officer and co-founder of Alteryx. She is responsible for strengthening upskilling and reskilling efforts for Alteryx customers to enable a culture of analytics, scaling the presence of the Alteryx SparkED education program and furthering diversity and inclusion in the workplace. As the former Chief Customer Officer, Libby has helped many Fortune 100 executives to identify and seize market opportunities, outsmart their competitors, and drive more revenue from their current businesses using analytics.  In the episode, Richie and Libby explore the differences between analytics and business intelligence, analytics as a team sport, the importance of speed in analytics, generative AI and its implications in analytics, the role of data quality and governance, Alteryx’s AI platform, data skills as a workplace necessity, using AI to automate documentation and insights, success stories and mistakes within analytics, and much more.  Links Mentioned in the Show: AlteryxAlteryx SparkED Program[Course] Introduction to AlteryxRelated Episode: From Data Literacy to AI Literacy with Cindi Howson, Chief Data Strategy Officer at ThoughtSpotSign up to RADAR: The Analytics Edition New to DataCamp? Learn on the go using the DataCamp mobile appEmpower your business with world-class data and AI skills with DataCamp for business

Explore our four primary criteria for evaluating conversational BI products. Published at: https://www.eckerson.com/articles/genai-driven-analytics-product-evaluation-criteria-for-conversational-bi