Data pipelines become chaotic with pressures of agile, democratization, self-service, and organizational “pockets” of analytics. From enterprise BI to self-service analysis, data pipeline management should ensure analysis results are traceable, reproducible, and of production strength. Robust data pipelines rely on eight critical components.

Originally published at https://www.eckerson.com/articles/the-complexities-of-modern-data-pipelines

In this episode, Wayne Eckerson and Jen Underwood explore a new era of analytics. Data volumes and complexity have exceeded the limits of current manual drag-and-drop analytics solutions. Data moves at the speed of light while speed-to-insight lags farther and farther behind. It is time to explore intelligent, next generation, machine-powered analytics to retain your competitive edge. It is time to combine the best of the human mind and machine.

Underwood is an analytics expert and founder of Impact Analytic. She is a former product manager at Microsoft who spearheaded the design and development of the reinvigorated version of Power BI, which has since become a market leading BI tool. Underwood is an IBM Analytics Insider, SAS contributor, former Tableau Zen Master, Top 10 Women Influencer and active analytics community member. She is keenly interested in the intersection of data visualization and data science and writes and speaks persuasively about these topics.

In this podcast, Wayne Eckerson and James Serra discuss myths of modern data management. Some of the myths discussed include 'all you need is a data lake', 'the data warehouse is dead', 'we don’t need OLAP cubes anymore', 'cloud is too expensive and latency is too slow', 'you should always use a NoSQL product over a RDBMS.'

Serra is big data and data warehousing solutions architect at Microsoft with over thirty years of IT experience. He is a popular blogger and speaker and has presented at dozens of Microsoft PASS and other events. Prior to Microsoft, Serra was an independent data warehousing and business intelligence architect and developer.

In this podcast, Henry Eckerson interviews Dave Wells on the current health and future of the data warehouse. Wells acknowledges that data warehouses are struggling, but argues they are still necessary and cannot be replaced by data lakes. He then explains what the role of the modern data warehouse should be, practical steps forward for evolving the data warehouse, and much more.

Wells is an advisory consultant, educator, and industry analyst dedicated to building meaningful connections throughout the path from data to business value. He works at the intersection of information management and business management, driving business impact through analytics, business intelligence, and active data management. More than forty years of information systems experience combined with over ten years of business management give him a unique perspective about the connections among business, information, data, and technology. Knowledge sharing and skill building are Dave’s passions, carried out through consulting, speaking, teaching, and writing.

He is now the practice director of data management at Eckerson Group, cofounder and director of education at eLearningCurve, and a faculty member at The Data Warehousing Institute.

In this episode, Wayne Eckerson and Mike Masciandaro discuss keys to creating successful reports that users will love and use. Masciandaro provides in-depth explanations of each key (easy, drill, monitor, accurate, relevant, timely, responsive, and secure).

Masciandaro is a veteran business intelligence practitioner who recently retired from an illustrious career at Dow Chemical as director of BI. During that time, Mike saw and did just about everything there is to do in the world of BI, data, and analytics. He is now intent on sharing his hard-won knowledge with others.

In this episode, Wayne Eckerson and Lenin Gali discuss the past and future of the cloud and big data.

Gali is a data analytics practitioner who has always been on the leading edge of where business and technology intersect. He was one of the first to move data analytics to the cloud when he was BI director at ShareThis, a social media based services provider. He was instrumental in defining an enterprise analytics strategy, developing a data platform that brought games and business data together to enable thousands of data users to build better games and services by using Hadoop & Teradata while at Ubisoft. He is now spearheading the creation of a Hadoop-based data analytics platform at Quotient, a digital marketing technology firm in the retail industry.

In this episode, Mike Masciandaro and Wayne Eckerson discuss how to partner with the business and ensure high levels of customer satisfaction and adoption.

Mike is a veteran business intelligence practitioner who recently retired from an illustrious career at Dow Chemical. Mike has seen and done just about everything there is to do in the world of BI, data, and analytics. He is now intent on sharing his hard-won knowledge with others.

Dewayne Washington is back this week for part II of his Secrets of Data Analytics Leaders podcast with Eckerson Group. In part I, Dewayne and I discussed the role of the CIO. In this episode we discuss the keys to IT success.

Washington is a senior consultant with 20+ years of experience in BI and Analytics in over two dozen verticals. He is the former BI manager at Dallas/Fortworth International Airport and the current CIO at The Business of Intelligence. He is also the author of the book Get In The Stream, the ultimate guide to customer adoption, and his Data Warehousing and Mobile Solutions implementations have been featured in CIO Magazine and the Wall Street Journal. Washington is also a sought-after speaker and mentor for organizations striving to leverage BI and Analytics to meet business goals, thus earning him the title, BI Pharaoh.

In this podcast, Dewayne Washington speaks the unadulterated truth about the role of the CIO and discusses keys to success and common pitfalls. Washington is a senior consultant with 20+ years of experience in BI and Analytics in over two dozen verticals. He is the former BI manager at Dallas/Fortworth International Airport and the current CIO at The Business of Intelligence. Washington is also a sought-after speaker and mentor for organizations striving to leverage BI and Analytics to meet business goals, thus earning him the title, BI Pharaoh.

Mike Masciandaro is a veteran business intelligence practitioner who recently retired from an illustrious career at Dow Chemical. Mike has seen and done just about everything there is to do in the world of BI, data, and analytics. He is now intent on sharing his hard-won knowledge with others. You will learn the definition and purpose of a BI program, the role of subject matter experts, how to hire and retain talent, keys to delivering value as a BI program, and more.

In this podcast, Kevin Sonsky reveals the secrets to his success as a business intelligence leader at Citrix Systems. During the past 11 years, he has implemented an enterprise-wide self-service reporting environment that has delivered deeper insights into customer purchasing behavior. At the same time, he has established a grassroots governance program that has successfully standardized on dozens of key enterprise metrics and reports. Kevin is interviewed by Wayne W. Eckerson, long-time thought leader in the business analytics field.