Recent technology developments are driving urgency to modernize data management. What do you do about architecture, modeling, quality, and governance to keep up with big data, cloud, self-service, and other trends in data and technology? Examining some best practices can spark ideas of where to begin.

Originally published at https://www.eckerson.com/articles/stepping-up-to-modern-data-management

Data catalogs have become the centerpiece of modern data management. They are the means to connect self-service analysts with the right data, the “go to” technology for data curation, and the new gold standard for metadata management. With many data catalog tools available, choosing the right data catalog is an important decision.

In this episode, Wayne Eckerson asks Steve Dine about the approach needed to migrate to the Cloud and architecture required to run analytics in the Cloud. Steve Dine talks extensively about the pitfalls to avoid during Cloud migration and finishes off by saying that even though security is a big issue, most organizations will have part of their architecture in the Cloud during the next two-three years. Steve Dine is a BI and enterprise data consultant and industry thought leader who has extensive experience in designing, delivering and managing highly scalable and maintainable modern data architecture solutions.

BI teams often work extremely hard but have little to show for their effort, and they can never get ahead of a continuous backlog of requests. The business rewards their hard work by reducing their budget and staff. Even if they know what needs to be done, they have little authority to make it happen. To succeed, BI leaders need to partner with the business and deliver quick wins to turn the tide of adoption in their favor.

Originally published at https://www.eckerson.com/articles/why-bi-teams-struggle-the-tipping-point-of-success

In this Episode, Wayne Eckerson asks Charles Reeves about his organization’s Internet of Things and Big Data strategy. Reeves is senior manager of BI and analytics at Graphics Packaging International, a leader in the packaging industry with hundreds of worldwide customers. He has 25 years of professional experience in IT management including nine years in reporting, analytics, and data governance.

A data analyst workbench will inevitably integrate data catalog, data preparation, and data analysis functionality. Data analysts don’t want to jump from tool to tool when executing a workflow that is both linear and iterative. Data analysts with this kind of workbench will be more productive and foster higher levels of reuse and data literacy.

Originally published at https://www.eckerson.com/articles/self-service-triumvirate-the-new-data-analyst-workbench

In this episode, Wayne Eckerson and Shakeeb Ahkter dive into DataOps. They discuss what DataOps is, the goals and principles of DataOps, and reasons to adopt a DataOps strategy. Shakeeb also reveals the benefits gained from DataOps and what tools he uses. He is the Director of Enterprise Data Warehouse at Northwestern Medicine and is responsible for direction and oversight of data management, data engineering, and analytics.

A classic management practice dictates that a newly-appointed leader must accomplish certain things in their first 90 days. While some of this is general knowledge, there are specifics when it comes to Data Management and Analytics If you have been recently named to head any group that has to manage or facilitate the use of data, at any level in the organization, then this audio blog post is for you.

Originally published at https://www.eckerson.com/articles/what-do-you-do-first-after-being-hired-as-a-bi-analytics-data-engineering-director

In this episode, Wayne Eckerson and Rich Fox discuss what differentiates data science from analytics, why and how data science addresses business needs, why balance scorecards are relevant, and why Excel is a problem. Throughout the podcast, Fox shares many real-life examples and personal experiences.

Fox is vice president of Data Science and Analytics at Apex Parks Group, one of the largest entertainment center companies in the United States, which operates amusement parks, water parks, and family entertainment centers.

With all the hype and attention around big data and huge data platforms, there can sometimes be some data envy. There are still organizations and companies that don’t have big data: are they not poised for analytics too? Can they not get insights as well? The BI Pharaoh gives tips on how to work with your little data just like the big boys.

Originally published at https://www.eckerson.com/articles/little-data-needs-love-too

We’re at the dawn of a new era in decision making made possible by the intersection of business intelligence and artificial intelligence. Rather than replace BI, AI will make BI more pervasive. AI-infused BI tools will be easier to use, generate more useful insights, and make business users more productive. Rather than replace human decision makers, AI will free them to focus on value-added activities and make decisions with data rather than rely solely on gut instinct.

Originally published at https://www.eckerson.com/articles/the-impact-of-ai-on-analytics-machine-generated-intelligence

In this episode, Wayne Eckerson and Rich Galan discuss the obstacles to delivering timely analysis, the problems that large volumes of data create, solutions to those issues, and where BI is headed in the near future. Rich is a veteran data analytics leader with 20 years of experience in a variety of data-driven organizations.

Data analysts who sit in each business function (i.e., sales, marketing, finance) are critical to the success of a self-service analytics strategy. The problem is that most data analysts don’t receive the training and support they need to be proficient with self-service data and analytics tools. The easiest way to improve the skills and satisfaction of most data analysts is simple: bring them together into a power user network.

Originally published at https://www.eckerson.com/articles/power-user-networks-the-key-to-self-service-analytics

In this episode, Wayne Eckerson and Steve Dine discuss modernizing data architectures. They cover the definition of a modern data architecture, telltale signs you need to modernize, best practices to modernize, and much more.

Dine is the President of Datasource Consulting, an EXL company. He has extensive experience designing, delivering, and managing successful, highly scalable and maintainable modern data architecture solutions.

Data pipelines become chaotic with pressures of agile, democratization, self-service, and organizational “pockets” of analytics. From enterprise BI to self-service analysis, data pipeline management should ensure analysis results are traceable, reproducible, and of production strength. Robust data pipelines rely on eight critical components.

Originally published at https://www.eckerson.com/articles/the-complexities-of-modern-data-pipelines

In this episode, Wayne Eckerson and Jen Underwood explore a new era of analytics. Data volumes and complexity have exceeded the limits of current manual drag-and-drop analytics solutions. Data moves at the speed of light while speed-to-insight lags farther and farther behind. It is time to explore intelligent, next generation, machine-powered analytics to retain your competitive edge. It is time to combine the best of the human mind and machine.

Underwood is an analytics expert and founder of Impact Analytic. She is a former product manager at Microsoft who spearheaded the design and development of the reinvigorated version of Power BI, which has since become a market leading BI tool. Underwood is an IBM Analytics Insider, SAS contributor, former Tableau Zen Master, Top 10 Women Influencer and active analytics community member. She is keenly interested in the intersection of data visualization and data science and writes and speaks persuasively about these topics.

In this podcast, Carl Gerber and Wayne Eckerson discuss Gerber’s top five data governance best practices: Motivation, Assessment, Data Assets Catalog, CxO Alliance, and Data Quality.

Gerber is a long-time chief data officer and data leader at several large, diverse financial services and manufacturing firms, who is now an independent consultant and an Eckerson Group partner.

He helps large organizations develop data strategies, modernize analytics, and establish enterprise data governance programs that ensure data quality, operational efficiency, regulatory compliance, and business outcomes. He also mentors and coaches Chief Data Officers and fills that role on an interim basis.

With today’s unprecedented velocity of change in data and technologies, the single version of the truth (SVOT) is sometimes looked upon as a nice to have. But SVOT is not optional. Despite the difficulties in constructing a single version of the truth, it is now needed more than ever to keep CEOs happy and to stay compliant with new regulations like the GDPR.

Originally published at https://www.eckerson.com/articles/single-version-of-the-truth-not-optional

Ethics is challenging because right and good are not always clear. More data, more kinds of data, and advanced analysis of data often conflict with concerns of data privacy, security, anonymity, and ownership. Resolving these conflicts requires acknowledgment, discussion, and the hard work of defining ethics-based policies and creating a culture of ethical conduct.

Originally published at https://www.eckerson.com/articles/data-ethics-the-new-data-governance-challenge

In this episode, Wayne Eckerson and Jeff Magnusson discuss the data architecture Stitch Fix created to support its data science workloads, as well as the need to balance man and machine and art and science.

Magnusson is the vice president of data platform at Stitch Fix. He leads a team responsible for building the data platform that supports the company's team of 80+ data scientists, as well as other business users. That platform is designed to facilitate self-service among data scientists and promote velocity and innovation that differentiate Stitch Fix in the marketplace. Before Stitch Fix, Magnusson managed the data platform architecture team at Netflix where he helped design and open source many of the components of the Hadoop-based infrastructure and big data platform.

To survive and thrive in today’s information economy, organizations need to establish a compelling data strategy that advances the company’s key objectives and goals. This is not a job for the chief information officer or even chief data officer; it’s a task that the CEO must own and champion and assemble his best and brightest to execute. This audio blog examines the ten step methodology that Eckerson Group uses to help organizations create a data strategy.

Originally published at https://www.eckerson.com/articles/how-to-create-a-data-strategy-part-1-overview

Data engineering is one of the hottest and most difficult jobs to fill in the field of analytics. Breadth and depth of required skills limits the number of people qualified to work as data engineers. If you’re seeking to hire data engineers, consider the 24 skill areas identified here as guidance to shape job descriptions and to screen candidates. If you’re seeking to become a data engineer, take the skills assessment to highlight your strengths and identify your gaps.

Originally published at https://www.eckerson.com/articles/data-engineering-coming-of-age

In this podcast, Wayne Eckerson and Joe Caserta discuss data migration, compare cloud offerings from Amazon, Google, and Microsoft, and define and explain artificial intelligence.

You can contact Caserta by visiting caserta.com or by sending him an email to [email protected]. Follow him on Twitter @joe_caserta.

Caserta is President of a New York City-based consulting firm he founded in 2001 and a longtime data guy. In 2004, Joe teamed up with data warehousing legend, Ralph Kimball to write to write the book The Data Warehouse ETL Toolkit. Today he’s now one of the leading authorities on big data implementations. This makes Joe one of the few individuals with in-the-trenches experience on both sides of the data divide, traditional data warehousing on relational databases and big data implementations on Hadoop and the cloud.

Every organization that manufactures data for decision making is rethinking its data architecture. Compared to five years ago, there is a treasure trove of new technologies and techniques that promise to transform the way organizations compete and serve customers. This audio blog summarizes the major characteristics of a modern data architecture and serves a high-level guide for organizations that are in the midst of developing a new data strategy for the modern age.

Originally published at https://www.eckerson.com/articles/ten-characteristics-of-a-modern-data-architecture

In this podcast, Wayne Eckerson and James Serra discuss myths of modern data management. Some of the myths discussed include 'all you need is a data lake', 'the data warehouse is dead', 'we don’t need OLAP cubes anymore', 'cloud is too expensive and latency is too slow', 'you should always use a NoSQL product over a RDBMS.'

Serra is big data and data warehousing solutions architect at Microsoft with over thirty years of IT experience. He is a popular blogger and speaker and has presented at dozens of Microsoft PASS and other events. Prior to Microsoft, Serra was an independent data warehousing and business intelligence architect and developer.