"Data Modeling with Tableau" provides a comprehensive guide to effectively utilizing Tableau Prep and Tableau Desktop for building elegant data models that drive organizational insights. You'll explore robust data modeling strategies and governance practices tailored to Tableau's diverse toolset, empowering you to make faster and more informed decisions based on data. What this Book will help me do Understand the fundamentals of data modeling in Tableau using Prep Builder and Desktop. Learn to optimize data sources for performance and better query capabilities. Implement secure and scalable governance strategies with Tableau Server and Cloud. Use advanced Tableau features like Ask Data and Explain Data to enable powerful analytics. Apply best practices for sharing and extending data models within your organization. Author(s) Kirk Munroe is an experienced data professional with a deep understanding of Tableau-driven analytics. With years of in-field expertise, Kirk now dedicates his career to helping businesses unlock their data's potential through effective Tableau solutions. His hands-on approach ensures this book is practical and approachable. Who is it for? This book is ideal for data analysts and business analysts aiming to enhance their skills in data modeling. It is also valuable for professionals such as data stewards, looking to implement secure and performant data strategies. If you seek to make enterprise data more accessible and actionable, this book is for you.

Summary

Making effective use of data requires proper context around the information that is being used. As the size and complexity of your organization increases the difficulty of ensuring that everyone has the necessary knowledge about how to get their work done scales exponentially. Wikis and intranets are a common way to attempt to solve this problem, but they are frequently ineffective. Rehgan Avon co-founded AlignAI to help address this challenge through a more purposeful platform designed to collect and distribute the knowledge of how and why data is used in a business. In this episode she shares the strategic and tactical elements of how to make more effective use of the technical and organizational resources that are available to you for getting work done with data.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don't forget to thank them for their continued support of this show! Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan's active metadata capabilities. Push information about data freshness and quality to your business intelligence, automatically scale up and down your warehouse based on usage patterns, and let the bots answer those questions in Slack so that the humans can focus on delivering real value. Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Struggling with broken pipelines? Stale dashboards? Missing data? If this resonates with you, you’re not alone. Data engineers struggling with unreliable data need look no further than Monte Carlo, the leading end-to-end Data Observability Platform! Trusted by the data teams at Fox, JetBlue, and PagerDuty, Monte Carlo solves the costly problem of broken data pipelines. Monte Carlo monitors and alerts for data issues across your data warehouses, data lakes, dbt models, Airflow jobs, and business intelligence tools, reducing time to detection and resolution from weeks to just minutes. Monte Carlo also gives you a holistic picture of data health with automatic, end-to-end lineage from ingestion to the BI layer directly out of the box. Start trusting your data with Monte Carlo today! Visit dataengineeringpodcast.com/montecarlo to learn more. Your host is Tobias Macey and today I'm interviewing Rehgan Avon about her work at AlignAI to help organizations standardize their technical and procedural approaches to working with data

Interview

Introduction How did you get involved in the area of data management? Can you describe what AlignAI is and the story behind it? What are the core problems that you are focused on addressing?

What are the tactical ways that you are working to solve those problems?

What are some of the common and avoidable ways that analytics/AI projects go wrong?

What are some of the ways that organizational scale and complexity impacts their ability to execute on data and AI projects?

What are the ways that incomplete/unevenly distributed knowledge manifests in project design and execution? Can you describe the design and implementation of the AlignAI platform?

How have the goals and implementation of the product changed since you

Summary

With all of the messaging about treating data as a product it is becoming difficult to know what that even means. Vishal Singh is the head of products at Starburst which means that he has to spend all of his time thinking and talking about the details of product thinking and its application to data. In this episode he shares his thoughts on the strategic and tactical elements of moving your work as a data professional from being task-oriented to being product-oriented and the long term improvements in your productivity that it provides.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don't forget to thank them for their continued support of this show! Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. By the time errors have made their way into production, it’s often too late and damage is done. Datafold built automated regression testing to help data and analytics engineers deal with data quality in their pull requests. Datafold shows how a change in SQL code affects your data, both on a statistical level and down to individual rows and values before it gets merged to production. No more shipping and praying, you can now know exactly what will change in your database! Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control. Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Sign up free at dataengineeringpodcast.com/rudder Build Data Pipelines. Not DAGs. That’s the spirit behind Upsolver SQLake, a new self-service data pipeline platform that lets you build batch and streaming pipelines without falling into the black hole of DAG-based orchestration. All you do is write a query in SQL to declare your transformation, and SQLake will turn it into a continuous pipeline that scales to petabytes and delivers up to the minute fresh data. SQLake supports a broad set of transformations, including high-cardinality joins, aggregations, upserts and window operations. Output data can be streamed into a data lake for query engines like Presto, Trino or Spark SQL, a data warehouse like Snowflake or Redshift., or any other destination you choose. Pricing for SQLake is simple. You pay $99 per terabyte ingested into your data lake using SQLake, and run unlimited transformation pipelines for free. That way data engineers and data users can process to their heart’s content without worrying about their cloud bill. For data engineering podcast listeners, we’re offering a 30 day trial with unlimited data, so go to dataengineeringpodcast.com/upsolver today and see for yourself how to avoid DAG hell. Your host is Tobias Macey and today I'm interviewing Vishal Singh about his experience

Today I’m chatting with returning guest Tom Davenport, who is a Distinguished Professor at Babson College, a Visiting Professor at Oxford, a Research Fellow at MIT, and a Senior Advisor to Deloitte’s AI practice. He is also the author of three new books (!) on AI and in this episode, we’re discussing the role of product orientation in enterprise data science teams, the skills required, what he’s seeing in the wild in terms of teams adopting this approach, and the value it can create. Back in episode 26, Tom was a guest on my show and he gave the data science/analytics industry an approximate “2 out of 10” rating in terms of its ability to generate value with data. So, naturally, I asked him for an update on that rating, and he kindly obliged. How are you all doing? Listen in to find out!

Highlights / Skip to:

Tom provides an updated rating (between 1-10) as to how well he thinks data science and analytics teams are doing these days at creating economic value (00:44) Why Tom believes that “motivation is not enough for data science work” (03:06) Tom provides his definition of what data products are and some opinions on other industry definitions (04:22) How Tom views the rise of taking a product approach to data roles and why data products must be tied to value (07:55) Tom explains why he feels top down executive support is needed to drive a product orientation (11:51) Brian and Tom discuss how they feel companies should prioritize true data products versus more informal AI efforts (16:26) The trends Tom sees in the companies and teams that are implementing a data product orientation (19:18) Brian and Tom discuss the models they typically see for data teams and their key components (23:18) Tom explains the value and necessity of data product management (34:49) Tom describes his three new books (39:00)

Quotes from Today’s Episode “Data science in general, I think has been focused heavily on motivation to fit lines and curves to data points, and that particular motivation certainly isn’t enough in that even if you create a good model that fits the data, it doesn’t mean at all that is going to produce any economic value.” – Tom Davenport  (03:05)

“If data scientists don’t worry about deployment, then they’re not going to be in their jobs for terribly long because they’re not providing any value to their organizations.” – Tom Davenport (13:25)

“Product also means you got to market this thing if it’s going to be successful. You just can’t assume because it’s a brilliant algorithm with capturing a lot of area under the curve that it’s somehow going to be great for your company.” – Tom Davenport (19:04)

“[PM is] a hard thing, even for people in non-technical roles, because product management has always been a sort of ‘minister without portfolio’ sort of job, and you know, influence without formal authority, where you are responsible for a lot of things happening, but the people don’t report to you, generally.” – Tom Davenport (22:03)

“This collaboration between a human being making a decision and an AI system that might in some cases come up with a different decision but can’t explain itself, that’s a really tough thing to do [well].” – Tom Davenport (28:04)

“This idea that we’re going to use externally-sourced systems for ML is not likely to succeed in many cases because, you know, those vendors didn’t work closely with everybody in your organization” – Tom Davenport (30:21)

“I think it’s unlikely that [organizational gaps] are going to be successfully addressed by merging everybody together in one organization. I think that’s what product managers do is they try to address those gaps in the organization and develop a process that makes coordination at least possible, if not true, all the time.” – Tom Davenport (36:49)

Links Tom’s LinkedIn: https://www.linkedin.com/in/davenporttom/ Tom’s Twitter: https://twitter.com/tdav All-in On AI by Thomas Davenport & Nitin Mittal, 2023 Working With AI by Thomas Davenport & Stephen Miller, 2022 Advanced Introduction to AI in Healthcare by Thomas Davenport, John Glaser, & Elizabeth Gardner, 2022 Competing On Analytics by Thomas Davenport & Jeanne G. Harris, 2007

It's that one-time-of-the-year when we do a little bit of navel-gazing, a little bit of prognostication, and, when the year is a year like 2022, a little more cursing than usual. Not only did the podcast hit a fairly meaningless vanity metric milestone this year, but we also maintained our explicit rating! Executive producer Josh Crowhurst joined us to look back on the podcast and the analytics industry in 2022, as well as to do a little bit of crystal ball gazing into 2023 and beyond! For complete show notes, including links to items mentioned in this episode and a transcript of the show, visit the show page.

Summary

Encryption and security are critical elements in data analytics and machine learning applications. We have well developed protocols and practices around data that is at rest and in motion, but security around data in use is still severely lacking. Recognizing this shortcoming and the capabilities that could be unlocked by a robust solution Rishabh Poddar helped to create Opaque Systems as an outgrowth of his PhD studies. In this episode he shares the work that he and his team have done to simplify integration of secure enclaves and trusted computing environments into analytical workflows and how you can start using it without re-engineering your existing systems.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don't forget to thank them for their continued support of this show! Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. By the time errors have made their way into production, it’s often too late and damage is done. Datafold built automated regression testing to help data and analytics engineers deal with data quality in their pull requests. Datafold shows how a change in SQL code affects your data, both on a statistical level and down to individual rows and values before it gets merged to production. No more shipping and praying, you can now know exactly what will change in your database! Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control. Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Sign up free at dataengineeringpodcast.com/rudder Build Data Pipelines. Not DAGs. That’s the spirit behind Upsolver SQLake, a new self-service data pipeline platform that lets you build batch and streaming pipelines without falling into the black hole of DAG-based orchestration. All you do is write a query in SQL to declare your transformation, and SQLake will turn it into a continuous pipeline that scales to petabytes and delivers up to the minute fresh data. SQLake supports a broad set of transformations, including high-cardinality joins, aggregations, upserts and window operations. Output data can be streamed into a data lake for query engines like Presto, Trino or Spark SQL, a data warehouse like Snowflake or Redshift., or any other destination you choose. Pricing for SQLake is simple. You pay $99 per terabyte ingested into your data lake using SQLake, and run unlimited transformation pipelines for free. That way data engineers and data users can process to their heart’s content without worrying about their cloud bill. For data engineering podcast listeners, we’re offering a 30 day trial with unlimited data, so go to dataengineeringpodcast.com/upsolver today an

Upgrade your knowledge to learn S/4HANA, the latest version of the SAP ERP system, with its built-in intelligent technologies, including AI, machine learning, and advanced analytics. Since the first edition of this book published as SAP ERP Financial and Controlling: Configuration and Use Management, the perspective has changed significantly as S/4HANA now comes with new features, such as FIORI (new GUI), which focuses on flexible app style development and interactivity with mobile phones. It also has a universal journal, which helps in data integration in a single location, such as centralized processing, and is faster than ECC S/3. It merges FI & CO efficiently, which enables document posting in the Controlling area setup. General Ledger Accounts (FI) and Cost Element (CO) are mapped together in a way that cost elements (both primary and secondary) are part of G/L accounts. And a mandatory setup of customer-vendor integration with business partners is included vs the earlier ECC creation with separate vendor master and customer master.This updated edition presents new features in SAP S/4HANA, with in-depth coverage of the FI syllabus in SAP S/4HANA. A practical and hands-on approach includes scenarios with real-life examples and practical illustrations. There is no unnecessary jargon in this configuration and end-user manual. What You Will Learn Configure SAP FI as a pro in S/4 Master core aspects of Financial Accounting and Controlling Integrate SAP Financial with other SAP modules Gain a thorough hands-on experience with IMG (Implementation Guide) Understand and explain the functionalities of SAP FI Who This Book Is For FI consultants, trainers, developers, accountants, and SAP FI support organizations will find the book an excellent reference guide. Beginners without prior FI configuration experience will find the step-by-step illustrations to be practical and great hands-on experience.

Colleague, Scott Hoyt, joins to discuss where the American consumer stands and how that differs by income group. A shrinking savings rate and sputtering retail sales won't break the American consumer or Mark's good mood. The group differs on the odds of recession, but is in agreement a slowcession is underway. Happy Holidays to all our listeners. Full episode transcript Follow Mark Zandi @MarkZandi, Cris deRitis @MiddleWayEcon, and Marisa DiNatale on LinkedIn for additional insight

Questions or Comments, please email us at [email protected]. We would love to hear from you.    To stay informed and follow the insights of Moody's Analytics economists, visit Economic View.

The "CompTIA Data+: DAO-001 Certification Guide" is your complete resource to approaching and passing the CompTIA Data+ certification exam. This book offers clear explanations, step-by-step exercises, and practical examples designed to help you master the domain concepts essential for the DAO-001 exam. Prepare confidently and expand your career opportunities in data analytics. What this Book will help me do Understand and apply the five domains covered in the DAO-001 certification exam. Learn data preparation techniques such as collection, cleaning, and wrangling. Master descriptive statistical methods and hypothesis testing to analyze data. Create insightful visualizations and professional reports for stakeholders. Grasp the fundamentals of data governance, including data quality standards. Author(s) Cameron Dodd is an experienced data analyst and educator passionate about breaking down complex concepts. With years of teaching and hands-on analytics expertise, he has developed a student-centric approach to helping professionals achieve certification and career advancement. His structured yet relatable writing style makes learning intuitive. Who is it for? The ideal readers of this book are data professionals aiming to achieve CompTIA Data+ certification (DAO-001 exam), individuals entering the growing field of data analytics, and professionals looking to validate or expand their skills. Whether you're starting from scratch or solidifying your knowledge, this book is designed for all levels.

Scaling data teams from zero is hard: there are no easy shortcuts, and it is hard to find clear examples to learn from. That’s why we are very excited to co-present the work we did at On Deck over the last year: starting in Summer 2021 we built a data team from scratch, relying heavily on dbt as the core data modelling environment. Come hear us talk about how we set up the team, prioritised the many different requirements from an ever expanding team of stakeholders and, after just a few months, succeeding in moving On Deck away from a no-code data architecture (with more than 400 SaaS tools) and towards a centralised data model in dbt. We think the lessons (and especially the pitfalls) are worth telling!

Coalesce 2023 is coming! Register for free at https://coalesce.getdbt.com/.

How do you efficiently scale your analytics stack when your data and data team grows 10x in 2 years? How do you even start prioritizing what gets done when there's that much growth? In this talk, Felipe Leite and Stephen Pastan of Miro unpack their shift to a Modern Data Stack and share the vital technical changes they made to build a scalable and tech-forward data stack. Come join them in learning how they got to where they are today and what they’re working on for the future. More details coming soon!

Check the slides here: https://docs.google.com/presentation/d/1lLoRBYAv8wlJQuhSrflX4F6y4lqV3SnDNAU7BYxHYR8/edit#slide=id.p

Coalesce 2023 is coming! Register for free at https://coalesce.getdbt.com/.

Summary

The data ecosystem has seen a constant flurry of activity for the past several years, and it shows no signs of slowing down. With all of the products, techniques, and buzzwords being discussed it can be easy to be overcome by the hype. In this episode Juan Sequeda and Tim Gasper from data.world share their views on the core principles that you can use to ground your work and avoid getting caught in the hype cycles.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don't forget to thank them for their continued support of this show! Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. By the time errors have made their way into production, it’s often too late and damage is done. Datafold built automated regression testing to help data and analytics engineers deal with data quality in their pull requests. Datafold shows how a change in SQL code affects your data, both on a statistical level and down to individual rows and values before it gets merged to production. No more shipping and praying, you can now know exactly what will change in your database! Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control. Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Sign up free at dataengineeringpodcast.com/rudder Build Data Pipelines. Not DAGs. That’s the spirit behind Upsolver SQLake, a new self-service data pipeline platform that lets you build batch and streaming pipelines without falling into the black hole of DAG-based orchestration. All you do is write a query in SQL to declare your transformation, and SQLake will turn it into a continuous pipeline that scales to petabytes and delivers up to the minute fresh data. SQLake supports a broad set of transformations, including high-cardinality joins, aggregations, upserts and window operations. Output data can be streamed into a data lake for query engines like Presto, Trino or Spark SQL, a data warehouse like Snowflake or Redshift., or any other destination you choose. Pricing for SQLake is simple. You pay $99 per terabyte ingested into your data lake using SQLake, and run unlimited transformation pipelines for free. That way data engineers and data users can process to their heart’s content without worrying about their cloud bill. For data engineering podcast listeners, we’re offering a 30 day trial with unlimited data, so go to dataengineeringpodcast.com/upsolver today and see for yourself how to avoid DAG hell. Your host is Tobias Macey and today I'm interviewing Juan Sequeda and Tim Gasper about their views on the role of the data mesh paradigm for driving re-assessment of the foundational principles of data systems

Greg Jensen, Co-Chief Investment Officer at Bridgewater Associates, Mark and the team this week. We get into the causes and outlook for inflation and prospects for the economy and financial markets. Greg shares his dark forecast with the group. Full Episode Transcript Follow Mark Zandi @MarkZandi, Cris deRitis @MiddleWayEcon, and Marisa DiNatale on LinkedIn for additional insight

Questions or Comments, please email us at [email protected]. We would love to hear from you.    To stay informed and follow the insights of Moody's Analytics economists, visit Economic View.

Product experimentation is full of potholes for companies of any size, given the number of pieces (tooling, culture, process, persistence) that need to come together to be successful. Vijaye Raji (currently Statsig, formerly Facebook + Microsoft) and Sean Taylor (currently Motif Analytics, formerly Facebook + Lyft) have navigated these failure modes, and are here to help you (hopefully) do the same. This convo with Tristan + Julia is light on tooling + heavy on process: how to watch out for spillover effects in experiments, avoiding bias, how to run an experiment review, and why experiment throughput is a better indicator of success than individual experiment results. For full show notes and to read 6+ years of back issues of the podcast's companion newsletter, head to https://roundup.getdbt.com.  The Analytics Engineering Podcast is sponsored by dbt Labs.

In today’s episode, we’re joined by Michael Ramsbacker, Chief Product Officer at Trulioo, a leading global identity and business verification provider specializing in AML, KYC and KYB compliance.

We talk about:

• Michael’s background and the problems Trulioo is trying to solve.
• The importance of verifying identities across global markets.
• The challenges of implementing identity verification in an internet built on anonymity.
• How the role of product managers has changed as SaaS has evolved.
• The kind of culture and conditions needed for a software organization to succeed.
• How technologies like artificial intelligence and machine learning impact a SaaS product’s growth.
• The foundations that need to be in place to have trusted identity verification online.
• Key lessons for product managers.
• Striking a balance between focusing internally and externally as a CPO.

Michael Ramsbacker - https://www.linkedin.com/in/ramsbacker/ Trulioo - https://www.linkedin.com/company/trulioo/

This episode is brought to you by Qrvey

The tools you need to take action with your data, on a platform built for maximum scalability, security, and cost efficiencies. If you’re ready to reduce complexity and dramatically lower costs, contact us today at qrvey.com.

Qrvey, the modern no-code analytics solution for SaaS companies on AWS.

saas #analytics #AWS #BI

The insurance industry thrives on data from utilizing data and analytics to determine policy rates for customers to working with relevant partners in the industry to improve their products and services, data is embedded in everything that insurance companies do.

But insurance companies also have a number of hurdles to overcome, whether it’s transitioning legacy data into new processes and technology, balancing new projects and models with ever-changing regulatory standards, and balancing the ethical considerations of how to best utilize data without resulting in unintended consequences for the end user.

That’s why we’ve brought Rob Reynolds onto the show. Rob is the VP and Chief Data & Analytics Officer at W. R. Berkley, a multinational insurance holding company specializing in property and casualty insurance. Rob brings over two decades of experience in Data Science, IT, and technology leadership, with a particular expertise in building departments and establishing highly functioning teams, especially in highly dynamic environments.

In this episode, we talk in-depth about how insurance companies utilize data, the most important skills for anyone looking for data science jobs in the insurance industry, why the need for thoughtful criticism is growing in data science, and how an expertise in communication will put you ahead of the pack.

Summary One of the most critical aspects of software projects is managing its data. Managing the operational concerns for your database can be complex and expensive, especially if you need to scale to large volumes of data, high traffic, or geographically distributed usage. Planetscale is a serverless option for your MySQL workloads that lets you focus on your applications without having to worry about managing the database or fight with differences between development and production. In this episode Nick van Wiggeren explains how the Planetscale platform is implemented, their strategies for balancing maintenance and improvements of the underlying Vitess project with their business goals, and how you can start using it today to free up the time you spend on database administration.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don’t forget to thank them for their continued support of this show! Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. By the time errors have made their way into production, it’s often too late and damage is done. Datafold built automated regression testing to help data and analytics engineers deal with data quality in their pull requests. Datafold shows how a change in SQL code affects your data, both on a statistical level and down to individual rows and values before it gets merged to production. No more shipping and praying, you can now know exactly what will change in your database! Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control. Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Sign up free at dataengineeringpodcast.com/rudder Build Data Pipelines. Not DAGs. That’s the spirit behind Upsolver SQLake, a new self-service data pipeline platform that lets you build batch and streaming pipelines without falling into the black hole of DAG-based orchestration. All you do is write a query in SQL to declare your transformation, and SQLake will turn it into a continuous pipeline that scales to petabytes and delivers up to the minute fresh data. SQLake supports a broad set of transformations, including high-cardinality joins, aggregations, upserts and window operations. Output data can be streamed into a data lake for query engines like Presto, Trino or Spark SQL, a data warehouse like Snowflake or Redshift., or any other destination you choose. Pricing for SQLake is simple. You pay $99 per terabyte ingested into your data lake using SQLake, and run unlimited transformation pipelines for free. That way data engineers and data users can process to their heart’s content without worrying about their cloud bill. For data engineering podcast l

More organizations than ever understand the importance of data lake architectures for deriving value from their data. Building a robust, scalable, and performant data lake remains a complex proposition, however, with a buffet of tools and options that need to work together to provide a seamless end-to-end pipeline from data to insights. This book provides a concise yet comprehensive overview on the setup, management, and governance of a cloud data lake. Author Rukmani Gopalan, a product management leader and data enthusiast, guides data architects and engineers through the major aspects of working with a cloud data lake, from design considerations and best practices to data format optimizations, performance optimization, cost management, and governance. Learn the benefits of a cloud-based big data strategy for your organization Get guidance and best practices for designing performant and scalable data lakes Examine architecture and design choices, and data governance principles and strategies Build a data strategy that scales as your organizational and business needs increase Implement a scalable data lake in the cloud Use cloud-based advanced analytics to gain more value from your data

Justin Wolfers, Professor of Public Policy and Economics at the University of Michigan, joins the podcast to discuss inflation, monetary policy, and prospects for recession next year. Are we being too pessimistic when so much is going well with the U.S. economy?  Full episode transcript Follow Mark Zandi @MarkZandi, Cris deRitis @MiddleWayEcon, and Marisa DiNatale on LinkedIn for additional insight

Questions or Comments, please email us at [email protected]. We would love to hear from you.    To stay informed and follow the insights of Moody's Analytics economists, visit Economic View.

On today’s episode we’re joined by Manish Garg, SVP of Product and Technology at EarnUp, a tech platform that reduces risk, streamlines operations and improves the financial health of your borrowers.

We talk about:

• How EarnUp works and the problems it solves.
• How can startups successfully navigate the early stages?
• Building something new vs. improving on an existing product.
• A hybrid model between building on existing products and moving into innovation.
• How to recognize a strong product-market fit.
• The impact of AI and machine learning in business.

Manish Garg - https://www.linkedin.com/in/manishgarg/ EarnUp - https://www.linkedin.com/company/earnup/

This episode is brought to you by Qrvey

The tools you need to take action with your data, on a platform built for maximum scalability, security, and cost efficiencies. If you’re ready to reduce complexity and dramatically lower costs, contact us today at qrvey.com.

Qrvey, the modern no-code analytics solution for SaaS companies on AWS.

saas #analytics #AWS #BI