Business leaders can address AI bias and use it to have rational discussions about management and human bias. Published at: https://www.eckerson.com/articles/weighing-the-risk-and-reward-of-ai-a-non-technical-guide-for-business-leaders

Kevin Petrie, the Vice President of Research at Eckerson Group, and Dan O’Brien, research analyst, discussed large language models (LLMs), which are neural networks that analyze text to predict the next word or phrase. These models use training data, often from the internet, to understand word relationships and provide accurate answers to natural language questions.

Generative AI initiatives require new data pipelines that prepare text files for querying by language models. Data engineers, scientists, and other stakeholders collaborate to design and implement these pipelines, which span text sources, tokens, vectors, vector databases, and LMs. Published at: https://www.eckerson.com/articles/the-new-data-pipeline-for-generative-ai-where-and-how-it-works

Responsible AI ethical principles provide a clear, unifying purpose for the technological, business, and social goals of AI initiatives. Published at: https://www.eckerson.com/articles/the-opportunity-and-risk-of-generative-ai-part-iii-responsible-ai-ethics

Responsible AI can help data leaders comply with the fast-evolving regulatory environment of data and artificial intelligence. Published at: https://www.eckerson.com/articles/the-opportunity-and-risk-of-generative-ai-part-ii-how-responsible-ai-assists-compliance

US frontier history had races, risks, and rewards. Generative AI's future will follow a similar path. Published at: https://www.eckerson.com/articles/enterprise-data-and-the-taming-of-the-generative-ai-frontier

Dan and Jay discussed the concept of Data Fabric, an automated and AI-driven approach to managing modern data environments.

Simba Khadder and Kevin Petrie discuss strategies to overcome technical debt in implementation, the pivotal role of data in the success of ML projects, navigating regulatory compliance in machine learning, and the future of AI governance.

Generative AI brings a promise to improve lives in a blistering innovation race, but also a threat to people, corporations, and even nations. Data analytics leaders must understand the risks of generative AI, both societal and business-related, to use it positively and avoid the destructive consequences seen with nuclear energy development. Published at: https://www.eckerson.com/articles/the-opportunity-and-risk-of-generative-ai-part-i-a-nuclear-explosion

This blog recommends guiding principles for successful implementation of language models to assist data engineering. Published at: https://www.eckerson.com/articles/should-ai-bots-build-your-data-pipelines-part-iv-guiding-principles-for-success-with-language-models-and-data-engineering

An emerging approach to generative AI will help data engineering teams achieve much-needed productivity gains while controlling risk. Published at: https://www.eckerson.com/articles/should-ai-bots-build-your-data-pipelines-part-iii-the-emergence-of-small-language-models-for-data-engineering

Traditional companies must balance new and old technologies as part of an ever-modernizing data stack. This blog explores how companies strike the right balance to navigate economic uncertainty, AI disruption, and the need for tool consolidation. Published at: https://www.eckerson.com/articles/the-modernizing-data-stack-three-ways-to-balance-new-and-old

LLMs are hugely popular with data engineers because they boost productivity. But companies must adapt their data governance programs to control risks related to data quality, privacy, intellectual property, fai-Datarness, and explainability. Published at: https://www.eckerson.com/articles/should-ai-bots-build-your-data-pipelines-part-ii-risks-and-governance-approaches-for-data-engineers-to-use-large-language-models

Many data engineers already use large language models to assist data ingestion, transformation, DataOps, and orchestration. This blog commences a series that explores the emergence of ChatGPT, Bard, and LLM tools from data pipeline vendors, and their implications for the discipline of data engineering. Published at: https://www.eckerson.com/articles/should-ai-bots-build-your-data-pipelines-examining-the-role-of-chatgpt-and-large-language-models-in-data-engineering

Embeddings are a learned way of representing data in space. Vector databases make it easier to work with embeddings generated from deep learning models. They will become an essential tool in the AI stack because they reduce the time to structure data and train models. Published at: https://www.eckerson.com/articles/the-why-what-who-and-where-of-vector-databases

The modern data stack is a loose collection of technologies, often cloud-based, that collaboratively process and store data to support modern analytics. It must be automated, low code/no code, AI-assisted, graph-enabled, multimodal, streaming, distributed, meshy, converged, polyglot, open, and governed. Published at: https://www.eckerson.com/articles/twelve-must-have-characteristics-of-a-modern-data-stack

AutoML and the emerging approach of declarative ML help simplify the process of creating and refining ML models. Published at: https://www.eckerson.com/articles/automl-and-declarative-machine-learning-comparing-use-cases

Traditional techniques for managing data quality break at scale. Machine learning algorithms can automate aspects of the data quality workload, ensuring that the data the business users consume is reliable. This article profiles three tools and approaches that use ML to automate data quality. Published at: https://www.eckerson.com/articles/three-data-quality-automation-tools-you-should-consider

Synthetic data and artificial intelligence (AI) complement each other but are both subject to the risk of AI bias. Consequently, companies need to implement architectural and governance controls to reduce the bias that synthetic data can inject into AI models. Published at: https://www.eckerson.com/articles/mitigating-the-risk-of-bias-in-synthetic-data-for-ai

Many machine learning projects fail because data scientists don’t have the right data. Techniques such as synthetic data is a novel algorithmic approach to address algorithmic risks. Published at: https://www.eckerson.com/articles/synthetic-data-for-ai-definition-risks-and-strategies

Many machine learning projects fail because data scientists don’t have the right data. Techniques such as synthetic data is a novel algorithmic approach to address algorithmic risks. Published at: https://www.eckerson.com/articles/synthetic-data-for-ai-definition-risks-and-strategies

Data orchestration uses caching, APIs, and centralized metadata to help compute engines access data in hybrid or multi-cloud environments. Data platform engineers can use data orchestration to gain simple, flexible, and high-speed access to distributed data for modern analytics and AI projects. Published at: https://www.eckerson.com/articles/data-orchestration-simplifying-data-access-for-analytics

Decision AI is an emerging discipline that enables faster, smarter operational decisions by combining decision intelligence and contextual intelligence. Published at: https://www.eckerson.com/articles/decision-ai-and-the-opportunity-for-smarter-faster-actions

Machine learning models help respond to the time-based value and risks of business events. To achieve this on an ongoing basis, enterprises should build a streaming ML program based on sound business objectives, a cross-functional team, open platforms, and phased execution. Published at: https://www.eckerson.com/articles/machine-learning-and-streaming-data-pipelines-part-iii-guiding-principles

It’s tempting to dismiss observability as another overused buzzword. But this emerging discipline offers substantive methods for enterprises to monitor and optimize business metrics, IT operations, data pipelines, machine learning models, and data quality. Published at: https://www.eckerson.com/articles/the-five-shades-of-observability-business-operations-pipelines-models-and-data-quality