This blog defines the governance requirements that streaming data pipelines must meet to make artificial intelligence/machine learning (AI/ML) initiatives successful. Published at: https://www.eckerson.com/articles/streaming-data-governance-three-must-have-requirements-to-support-ai-ml-innovation

This blog, the second in a series, recommends six criteria to help data leaders evaluate tools for managing streaming data pipelines. Published at: https://www.eckerson.com/articles/ultimate-guide-to-streaming-data-pipelines-six-criteria-to-evaluate-and-select-the-right-tool

This blog defines streaming data, explains why companies need it, and explores how streaming data pipelines feed multi-faceted GenAI applications. Published at: https://www.eckerson.com/articles/why-and-how-streaming-data-drives-the-success-of-generative-ai

Many machine learning (ML) use cases center on real-time calculations. This article defines streaming ML and its architectural components. Published at: https://www.eckerson.com/articles/machine-learning-and-streaming-data-pipelines-part-i-definitions-and-architecture

The modern data stack is a loose collection of technologies, often cloud-based, that collaboratively process and store data to support modern analytics. It must be automated, low code/no code, AI-assisted, graph-enabled, multimodal, streaming, distributed, meshy, converged, polyglot, open, and governed. Published at: https://www.eckerson.com/articles/twelve-must-have-characteristics-of-a-modern-data-stack

Machine learning models help respond to the time-based value and risks of business events. To achieve this on an ongoing basis, enterprises should build a streaming ML program based on sound business objectives, a cross-functional team, open platforms, and phased execution. Published at: https://www.eckerson.com/articles/machine-learning-and-streaming-data-pipelines-part-iii-guiding-principles

This audio blog discusses cloud adoption and how data teams will migrate an increasing portion of their on-premises operational and analytics workloads to the cloud. They can best meet budget and project requirements by using data streaming technologies such as change data capture (CDC), which replicates real-time updates between data source and target.

Originally published at: https://www.eckerson.com/articles/the-next-wave-of-cloud-migrations-needs-data-streaming

Learn how to achieve the DataOps objectives of improved efficiency and data quality by migrating to a streaming architecture based on Apache Kafka.

IoT has created a tidal wave that data savvy organizations can turn into profitable business solutions. Most IoT data comes from sensors, which are now attached to almost every device imaginable, from factory floor machines and agricultural fields to your cell phone and toothbrush. But IoT is forcing companies to rethink their data architectures to ingest, process, and analyze streaming data in real-time.

To help us understand the impact of IoT on data architectures, we invited Dan Graham to our show for a second time. Dan is a former product marketing manager at both IBM and Teradata, renowned for combining deep technical knowledge with industry marketing savvy. During his tenure at those companies, he was responsible for MPP data management systems, data warehouses, and data lakes, and most recently, the Internet of Things.

Processing continuous data streams is becoming increasingly important. However, traditional analytics architectures were often not built for real-time scenarios. This article will illustrate challenges and discuss how streaming-first approaches can change the way we think about analytics architectures.

Originally published at: https://www.eckerson.com/articles/streams-everywhere-towards-streaming-first-architectures

This second article in a series on modern data architectures. It focuses on what drives customers to want a modern data architecture (i.e., fear and opportunity) in the first place. It then lists ten requirements that customers desire for a modern data architecture, ranging from “cloud-first” and “streaming-first” to “best of breed” and “predictable pricing”.

Originally published at: https://www.eckerson.com/articles/ten-things-companies-want-from-a-modern-data-architecture

In this episode, Daniel Graham dissects the capabilities of data lakes and compares it to data warehouses. He talks about the primary use cases of data lakes and how they are vital for big data ecosystems. He then goes on to explain the role of data warehouses which are still responsible for timely and accurate data but don't have a central role anymore. In the end, both Wayne Eckerson and Dan Graham settle on a common definition for modern data architectures.

Daniel Graham has more than 30 years in IT, consulting, research, and product marketing, with almost 30 years at leading database management companies. Dan was a Strategy Director in IBM’s Global BI Solutions division and General Manager of Teradata’s high-end server divisions. During his tenure as a product marketer, Dan has been responsible for MPP data management systems, data warehouses, and data lakes, and most recently, the Internet of Things and streaming systems.