Governance is difficult for an organization of any size, and many struggle to execute on data management in an efficient manner. At Assurance, the team has utilized Starburst Galaxy to embed ownership within the data mesh framework, completely transforming the way organizations handle data. By granting data owners complete control and visibility over their data, Assurance enables a more nuanced and effective approach to data management. This approach not only fosters a sense of responsibility but also ensures that data is relevant, up-to-date, and aligned with the evolving needs of the organization. In this presentation, Shen Weng and Mitchell Polsons will discuss the strategic implementation of compute ownership in Starburst Galaxy, showing how it empowers teams to identify and resolve issues quickly, significantly improving the uptime of key computing operations. This approach is vital for achieving operational excellence, characterized by enhanced efficiency, reliability, and quality. Additionally, the new data setup has enabled the Assurance team to simplify data transformation processes using dbt and to improve data quality monitoring with Monte Carlo, further streamlining and strengthening our data management practices.
talk-data.com
Topic
Data Quality
13
tagged
Activity Trend
Top Events
Have you ever wondered how a data company does data? In this session, Isaac Obezo, Staff Data Engineer at Starburst, will take you for a peek behind the curtain into Starburst’s own data architecture built to support batch processing of telemetry data within Galaxy data pipelines. Isaac will walk you through our architecture utilizing tools like git, dbt, and Starburst Galaxy to create a CI/CD process allowing our data engineering team to iterate quickly to deploy new models, develop and land data, and create and improve existing models in the data lake. Isaac will also discuss Starburst’s mentality toward data quality, the use of data products, and the process toward delivering quality analytics.
Join the team from Moody's Analytics as they take you on a personal journey of optimizing their data pipelines for data quality and governance. Like many data practitioners, Ryan understands the frustration and anxiety that comes with accidentally introducing bad code into production pipelines—he's spent countless hours putting out fires caused by these unexpected changes. In this session, Ryan will recount his experiences with a previous data stack that lacked standardized testing methods and visibility into the impact of code changes on production data. He'll also share how their new data stack is safeguarded by Datafold's data diffing and continuous integration (CI) capabilities, which enables his team to work with greater confidence, peace of mind, and speed.
While everyone's talking about AI, far fewer have deployed it successfully or turned technology into business outcomes. This panel changes that, bringing together Generative AI experts for a deep dive into the practical application of generative AI. From building new customer offerings to refreshing internal processes, the panellists will reflect on the importance of data quality, data security, the responsible use of data as well as change management when it comes to embedding generative AI into the business strategy.
Drawing on his 2023 book ‘Confident Data Science’, Adam Nelson will show you how to measure your organization's data culture. Learn how to use this key metric to understand how well your organization’s culture performs along four key dimensions: Offering access to quality information about the data it has; providing the right access to the right people at the right time; investing in data skills development; and maintaining high data quality standards.
As organizations are exploring and expanding on their AI capabilities, Chief Data Officers are now responsible for governing the data for responsible and trustworthy AI. This session will cover 5 key principles to ensure successful adoption and scaling of AI initiatives that align with their company’s business strategy. From data quality to advocating for ethical AI practices, the Chief Data Officer’s mandate has expanded to compliance of new AI regulations.
Peggy Tsai, Chief Data Officer at BigID and adjunct faculty member at Carnegie Mellon University for the Chief Data Officer executive program, will provide insights into the AI governance strategies and outcomes crucial for cultivating an AI-first organization. Drawing on her extensive experience in data governance and AI, this session will be an invaluable guidance for all participants aiming to adopt industry-leading practices.
Data quality is the most important attribute of a successful data platform that can accelerate data adoption and empower any organization with data-driven decisions. However, traditional profiling-based data quality and counts-based data quality and business rules-based data quality are outdated and not practical at the scale of petabyte-scaled data platforms where billions of rows get processed every day. In this talk, Sandhya Devineni and Rajesh Gundugollu will present a framework for using machine learning to detect data quality at scale in data products. The two data leaders at Asurion will highlight the lessons learned over years of crafting the advanced state of data quality using machine learning at scale, as well as discuss the pain points and blind spots of traditional data quality processes. After sharing lessons learned, the pair will dive into their implemented framework which can be utilized to improve the accuracy and reliability of data-driven decisions by identifying bad quality data records and revolutionizing how organiations approach data-driven decision making.
Join us for an insightful session on the evolving landscape of Data Quality and Observability practices, transitioning from manual to augmented approaches driven by semantics and GenAI. Discover the framework enabling organisations to build the architecture for conversational data quality, leaving behind the limitations of traditional, resource-heavy methods and legacy technology. Learn why context is paramount in data quality and observability, and leave with actionable insights to propel your organisation into the future of data management.
When data is the most valuable asset of your company, protecting it is a non-negotiable. While Information Security professionals are focused on Bad Actors, we have data operations and data governance professionals focused on Bad Data… Are they one and the same? What’s similar and what’s different between the worlds of data integrity and data security?
Drawing from a wealth of experience and real-world challenges, Gorkem will shed light on the pivotal role of data quality in the forefront of information security. We’ll discuss opportunities for early detection, auto-detection, and the establishment of tiered rules to manage and remediate bad data effectively. Learn how proactive governance and observability can transform data management from a reactive stance to a formidable defense mechanism, ensuring the integrity and security of your data ecosystem.
It’s a tale as old as time: a data migration that was supposed to take months turns into years turns into something that no longer has an end date—all while going over budget and increasing in complexity every day. In this session, Gleb is going deep on the methods, tooling, and hard lessons learned during a years-long migration at Lyft. Specifically, he'll share how you can leverage data quality testing methodologies like cross-database diffing to accelerate a data migration without sacrificing data quality. You should walk away with practices that will allow your data team to plan, move, and audit database objects with speed and confidence during a migration.
In today's data-driven world, organizations face the challenge of not only harnessing the power of data but also ensuring its responsible and effective use. This panel discussion will delve into the critical components of embedding data governance and data literacy into the fabric of organizational culture. Data governance forms the foundation of a robust data strategy, encompassing policies, processes, and frameworks to ensure data quality, integrity, and security. However, effective governance requires more than just frameworks; it necessitates a cultural shift where data stewardship is ingrained into every aspect of organizational operations. Moreover, data literacy is paramount in enabling individuals across an organization to effectively interpret, analyze, and derive insights from data. By cultivating a culture of data literacy, organizations empower employees to make informed decisions, driving innovation and growth. This panel will explore strategies for fostering a culture of accountability, collaboration, and trust around data practices driving sustainable success in today's dynamic business landscape.
In a classic cart before the horse scenario, many companies have jumped at leveraging Generative AI and other AI technologies. However, most of those same companies haven't completed the core work of building a reliable & secure foundation that provides data accessibility, analytics speed, and ensures data quality. The resulting risk for leaders is overinvestment in AI programs that may not have accurate & secure data access, further exposing the business to harm. It is a case of slowing down to speed up - ensure the foundation is solid before you build the house. In this talk by Starburst CEO Justin Borgman, and Head of Partner Solutions Architecture, Data & Analytics - AI/ML, Subodh Kumar from AWS, you'll learn about the essential data foundations for AI success. The foundation, the plumbing, and the framing that will set businesses up for AI success.
The need for an executive responsible for an organization’s information assets today may seem obvious. But some organizations still struggle with making a business case for the role. And even existing chief data officers can be confounded about how to formally justify their existence. This session will share eye-popping findings and analyses from Mr. Laney’s study of hundreds of organizations with and without a CDO.
As any good scientist knows, and any good data scientist should know, most discoveries begin with a hypothesis. We see a lot of surveys about the CDO role but don’t really have much of a point to make or look at the impact a CDO makes. This study examined over 500 organizations to determine how businesses with a CDO operate differently.
Drawing from the study's conclusions, attendees will learn about the benefits of a CDO, and how having one affects data quality, governance, data democratization and monetization. We'll explore whether having a CDO affects an organization's ability to value its data and how investors perceive it, and look at the career path of CDOs to better understand what makes an actual C-level CDO.