talk-data.com talk-data.com

Topic

data-engineering

3395

tagged

Activity Trend

1 peak/qtr
2020-Q1 2026-Q1

Activities

3395 activities · Newest first

Data Engineering with dbt

Data Engineering with dbt provides a comprehensive guide to building modern, reliable data platforms using dbt and SQL. You'll gain hands-on experience building automated ELT pipelines, using dbt Cloud with Snowflake, and embracing patterns for scalable and maintainable data solutions. What this Book will help me do Set up and manage a dbt Cloud environment and create reliable ELT pipelines. Integrate Snowflake with dbt to implement robust data engineering workflows. Transform raw data into analytics-ready data using dbt's features and SQL. Apply advanced dbt functionality such as macros and Jinja for efficient coding. Ensure data accuracy and platform reliability with built-in testing and monitoring. Author(s) None Zagni is a seasoned data engineering professional with a wealth of experience in designing scalable data platforms. Through practical insights and real-world applications, Zagni demystifies complex data engineering practices. Their approachable teaching style makes technical concepts accessible and actionable. Who is it for? This book is perfect for data engineers, analysts, and analytics engineers looking to leverage dbt for data platform development. If you're a manager or decision maker interested in fostering efficient data workflows or a professional with basic SQL knowledge aiming to deepen your expertise, this resource will be invaluable.

Geospatial Data Analytics on AWS

In "Geospatial Data Analytics on AWS," you will learn how to store, manage, and analyze geospatial data effectively using various AWS services. This book provides insight into building geospatial data lakes, leveraging AWS databases, and applying best practices to derive insights from spatial data in the cloud. What this Book will help me do Design and manage geospatial data lakes on AWS leveraging S3 and other storage solutions. Analyze geospatial data using AWS services such as Athena and Redshift. Utilize machine learning models for geospatial data processing and analytics using SageMaker. Visualize geospatial data through services like Amazon QuickSight and OpenStreetMap integration. Avoid common pitfalls when managing geospatial data in the cloud. Author(s) Scott Bateman, Janahan Gnanachandran, and Jeff DeMuth bring their extensive experience in cloud computing and geospatial analytics to this book. With backgrounds in cloud architecture, data science, and geospatial applications, they aim to make complex topics accessible. Their collaborative approach ensures readers can practically apply concepts to real-world challenges. Who is it for? This book is ideal for GIS and data professionals, including developers, analysts, and scientists. It suits readers with a basic understanding of geographical concepts but no prior AWS experience. If you're aiming to enhance your cloud-based geospatial data management and analytics skills, this is the guide for you.

SAP UI Frameworks for Enterprise Developers: A Practical Guide

Dive into the world of SAP UIs and begin your journey to becoming a development expert. In this book, you will learn the latest cutting-edge web technologies and gain a comprehensive understanding of the user experience concept that undergirds the SAP Fiori design system. SAP UI Frameworks for Enterprise Developers is filled with hands-on tutorials that will guide you through every step of the development journey, from setting up your coding editor to building your first application using UI5, SAP Fiori Elements, UI5 Web Components, or Fundamental Library Styles. Authors Marius Obert and Volker Buzek understand that staying ahead in the rapidly evolving UI technology landscape can be challenging, so they’ve included insights into upcoming industry-wide trends and future innovations. This book will provide you with the knowledge and skills needed to build intuitive applications with an eye-catching, SAP Fiori-conforming user interface.Don’t miss this opportunity to take your UI development skills to the next level! What You Will Learn Gain an overview of the technologies that can be used to implement enterprise-ready SAP Fiori UIs ready to be deployed in the cloud as well as on-premises Discover resources for connecting with domain experts to stay on top your game and learn about innovations in the SAP ecosystem Survey industry-wide technologies trends, like micro frontends and PWA, and relate these trends to both enterprise software and new projects led by SAP Who This Book Is For This book is intended for anyone interested in building intuitive enterprise user interfaces with modern web technologies in the greater SAP universe.

Data for All

Do you know what happens to your personal data when you are browsing, buying, or using apps? Discover how your data is harvested and exploited, and what you can do to access, delete, and monetize it. Data for All empowers everyone—from tech experts to the general public—to control how third parties use personal data. Read this eye-opening book to learn: The types of data you generate with every action, every day Where your data is stored, who controls it, and how much money they make from it How you can manage access and monetization of your own data Restricting data access to only companies and organizations you want to support The history of how we think about data, and why that is changing The new data ecosystem being built right now for your benefit The data you generate every day is the lifeblood of many large companies—and they make billions of dollars using it. In Data for All, bestselling author John K. Thompson outlines how this one-sided data economy is about to undergo a dramatic change. Thompson pulls back the curtain to reveal the true nature of data ownership, and how you can turn your data from a revenue stream for companies into a financial asset for your benefit. About the Technology Do you know what happens to your personal data when you’re browsing and buying? New global laws are turning the tide on companies who make billions from your clicks, searches, and likes. This eye-opening book provides an inspiring vision of how you can take back control of the data you generate every day. About the Book Data for All gives you a step-by-step plan to transform your relationship with data and start earning a “data dividend”—hundreds or thousands of dollars paid out simply for your online activities. You’ll learn how to oversee who accesses your data, how much different types of data are worth, and how to keep private details private. What's Inside The types of data you generate with every action, every day How you can manage access and monetization of your own data The history of how we think about data, and why that is changing The new data ecosystem being built right now for your benefit About the Reader For anyone who is curious or concerned about how their data is used. No technical knowledge required. About the Author John K. Thompson is an international technology executive with over 37 years of experience in the fields of data, advanced analytics, and artificial intelligence. Quotes An honest, direct, pull-no-punches source on one of the most important personal issues of our time....I changed some of my own behaviors after reading the book, and I suggest you do so as well. You have more to lose than you may think. - From the Foreword by Thomas H. Davenport, author of Competing on Analytics and The AI Advantage A must-read for anyone interested in the future of data. It helped me understand the reasons behind the current data ecosystem and the laws that are shaping its future. A great resource for both professionals and individuals. I highly recommend it. - Ravit Jain, Founder & Host of The Ravit Show, Data Science Evangelist

Building a Next-Gen SOC with IBM QRadar

In "Building a Next-Gen SOC with IBM QRadar", you'll learn how to utilize IBM QRadar to create an efficient Security Operations Center (SOC). The book covers deploying QRadar in various environments, understanding its architecture, and leveraging its powerful features to detect and respond to real-time threats with confidence, ultimately enabling advanced security practices. What this Book will help me do Understand and deploy IBM QRadar in different environments, including on-premises and cloud. Leverage QRadar's features to analyze network traffic, detect threats, and enhance security monitoring. Effectively use QRadar rules and searches to identify, correlate, and respond to security events. Integrate AI technologies with QRadar to automate and improve threat management processes. Maintain, troubleshoot, and scale the QRadar environment to meet evolving security needs. Author(s) Ashish Kothekar is an experienced cybersecurity specialist with a deep understanding of IBM QRadar and SOC operations. He has dedicated his career to helping organizations implement effective security practices. Through his accessible writing and detailed examples, he aims to empower security professionals to maximize their use of QRadar. Who is it for? This book is perfect for SOC analysts, security engineers, and cybersecurity enthusiasts who want to enhance their security skills. Readers should have a basic knowledge of networking and cybersecurity principles. If you're looking to deepen your understanding of IBM QRadar and build a next-gen SOC, this book is for you.

IBM Power System E980: Technical Overview and Introduction

This IBM® Redpaper™ publication provides a broad understanding of a new architecture of the IBM Power System E980 (9080-M9S) server that supports IBM AIX®, IBM i, and Linux operating systems (OSes). The objective of this paper is to introduce the major innovative Power E980 offerings and relevant functions: The IBM POWER9™ processor, which is available at frequencies of 3.55 - 4.0 GHz. Significantly strengthened cores and larger caches. Supports up to 64 TB memory. Integrated I/O subsystem and hot-pluggable Peripheral Component Interconnect Express (PCIe) Gen4 slots, double the bandwidth of Gen3 I/O slots. Supports EXP12SX and ESP24SX external disk drawers, which have 12 Gb SAS interfaces and double the existing EXP24S drawer bandwidth. New IBM EnergyScale™ technology offers new variable processor frequency modes that provide a significant performance boost beyond the static nominal frequency. This publication is for professionals who want to acquire a better understanding of IBM Power Systems™ products. The intended audience includes the following roles: Clients Sales and marketing professionals Technical support professionals IBM Business Partners Independent software vendors (ISVs) This paper expands the current set of IBM Power Systems documentation by providing a desktop reference that offers a detailed technical description of the Power E980 server. This paper does not replace the current marketing materials and configuration tools. It is intended as an extra source of information that, together with existing sources, can be used to enhance your knowledge of IBM server solutions.

IBM Storage Fusion Product Guide

This IBM® Redbooks® publication offers a short overview of IBM's integrated environment for container workloads, IBM Storage Fusion. The product comes in two variants, IBM Storage Fusion HCI System including all required hardware, and IBM Storage Fusion SDS (software-only) for deployment in public or private Clouds. This paper has been updated with the 2.5.1 release only for IBM Storage Fusion and the 2.5.2 release of IBM Storage Fusion HCI System.

How I Rob Banks

Follow FC as he steals from the world’s most secure banks and government facilities—without breaking a single law In How I Rob Banks: And Other Such Places, renowned ethical hacker and social engineer FC delivers a gripping and often hilarious discussion of his work: testing the limits of physical bank security by trying to “steal” money, data, and anything else he can get his hands on. In the book, you’ll explore the secretive world of physical assessments and follow FC as he breaks into banks and secure government locations to identify security flaws and loopholes. The author explains how banks and other secure facilities operate, both digitally and physically, and shows you the tools and techniques he uses to gain access to some of the world’s most locked-down buildings. You’ll also find: Strategies you can implement immediately to better secure your own company, home, and data against malicious actors Detailed photos, maps, and drawings to bring to life the unbelievable true stories contained inside An inside and candid look at a rarely examined industry through the eyes of one of its most respected penetration testers A can’t-miss account of real-life security exploits perfect for infosec pros, including red and blue teamers, pentesters, CIOs, CISSPs, and social engineers, How I Rob Banks also belongs in the hands of anyone who loves a great Ocean’s 11-style story pulled straight from the real world.

IBM Storage Fusion HCI System: Metro Sync Disaster Recovery Use Case

Metro sync disaster recovery (DR) provides two-way synchronous data replication between IBM Spectrum Fusion™ HCI clusters installed at two sites. In the event of a site disaster, applications can be failed over to the second site. The replication between the sites is synchronous, hence, the Metro sync DR solution is only available for metropolitan distance data centers with 40 millisecond latency or less. The procedures described in this paper for IBM Spectrum Fusion HCI 2.4 Metro sync DR are the same for IBM Storage Fusion HCI 2.5.2 Metro-DR. This IBM Redpaper publication will help you install and configure the new Metro sync DR function). The use case will show the end to end process with the failover and failback of the WordPress application. IBM Spectrum Fusion HCI and IBM Spectrum Fusion have become IBM Storage Fusion HCI System and IBM Storage Fusion. This edition uses the IBM Spectrum® brand names and will be updated with the next edition.

Data Modeling with Snowflake

This comprehensive guide, "Data Modeling with Snowflake", is your go-to resource for mastering the art of efficient data modeling tailored to the capabilities of the Snowflake Data Cloud. In this book, you will learn how to design agile and scalable data solutions by effectively leveraging Snowflake's unique architecture and advanced features. What this Book will help me do Understand the core principles of data modeling and how they apply to Snowflake's cloud-native environment. Learn to use Snowflake's features, such as time travel and zero-copy cloning, to create efficient data solutions. Gain hands-on experience with SQL recipes that outline practical approaches to transforming and managing Snowflake data. Discover techniques for modeling structured and semi-structured data for real-world business needs. Learn to integrate universal modeling frameworks like Star Schema and Data Vault into Snowflake implementations for scalability and maintainability. Author(s) The author, Serge Gershkovich, is a seasoned expert in database design and Snowflake architecture. With years of experience in the data management field, Serge has dedicated himself to making complex technical subjects approachable to professionals at all levels. His insights in this book are informed by practical applications and real-world experience. Who is it for? This book is targeted at data professionals, ranging from newcomers to database design to seasoned SQL developers seeking to specialize in Snowflake. If you are looking to understand and apply data modeling practices effectively within Snowflake's architecture, this book is for you. Whether you're refining your modeling skills or getting started with Snowflake, it provides the practical knowledge you need to succeed.

Modernize Applications with Apache Kafka

Application modernization has become increasingly important as older systems struggle to keep up with today's requirements. When you migrate legacy monolithic applications to microservices, easier maintenance and optimized resource utilization generally follow. But new challenges arise around communication within services and between applications. You can overcome many of these issues with the help of modern messaging technologies such as Apache Kafka. In this report, Jennifer Vargas and Richard Stroop from Red Hat explain how IT leaders and enterprise architects can use Kafka for microservices communication and then off-load operational needs through the use of Kubernetes and managed services. You'll also explore application modernization techniques that don't require you to break down your monolithic application. This report helps you: Understand the importance of migrating your monolithic applications to microservices Examine the various challenges you may face during the modernization process Explore application modernization techniques and learn the benefits of using Apache Kafka during the development process Learn how Apache Kafka can support business outcomes Understand how Kubernetes can help you overcome any difficulties you may encounter when using Kafka for application development

MySQL Crash Course

MySQL Crash Course is a fast-paced, no-nonsense introduction to relational database development. It’s filled with practical examples and expert advice that will have you up and running quickly. You’ll learn the basics of SQL, how to create a database, craft SQL queries to extract data, and work with events, procedures, and functions. You’ll see how to add constraints to tables to enforce rules about permitted data and use indexes to accelerate data retrieval. You’ll even explore how to call MySQL from PHP, Python, and Java. Three final projects will show you how to build a weather database from scratch, use triggers to prevent errors in an election database, and use views to protect sensitive data in a salary database. You’ll also learn how to: •Query database tables for specific information, order the results, comment SQL code, and deal with null values •Define table columns to hold strings, integers, and dates, and determine what data types to use •Join multiple database tables as well as use temporary tables, common table expressions, derived tables, and subqueries •Add, change, and remove data from tables, create views based on specific queries, write reusable stored routines, and automate and schedule events The perfect quick-start resource for database developers, MySQL Crash Course will arm you with the tools you need to build and manage fast, powerful, and secure MySQL-based data storage systems.

Data Fabric Architectures

The immense increase on the size and type of real time data generated across various edge computing platform results in unstructured databases and data silos. This edited book gathers together an international set of researchers to investigate the possibilities offered by data-fabric solutions; the volume focuses in particular on data architectures and on semantic changes in future data landscapes.

Uncertainty in Data Envelopment Analysis

Classical data envelopment analysis (DEA) models use crisp data to measure the inputs and outputs of a given system. In cases such as manufacturing systems, production processes, service systems, etc., the inputs and outputs may be complex and difficult to measure with classical DEA models. Crisp input and output data are fundamentally indispensable in the conventional DEA models. If these models contain complex uncertain data, then they will become more important and practical for decision makers.Uncertainty in Data Envelopment Analysis introduces methods to investigate uncertain data in DEA models, providing a deeper look into two types of uncertain DEA methods, fuzzy DEA and belief degree-based uncertainty DEA, which are based on uncertain measures. These models aim to solve problems encountered by classical data analysis in cases where the inputs and outputs of systems and processes are volatile and complex, making measurement difficult. Introduces methods to deal with uncertain data in DEA models, as a source of information and a reference book for researchers and engineers Presents DEA models that can be used for evaluating the outputs of many reallife systems in social and engineering subjects Provides fresh DEA models for efficiency evaluation from the perspective of imprecise data Applies the fuzzy set and uncertainty theories to DEA to produce a new method of dealing with the empirical data

Maturing the Snowflake Data Cloud: A Templated Approach to Delivering and Governing Snowflake in Large Enterprises

This project-oriented book gives you a hands-on approach to designing, developing, and templating your Snowflake platform delivery. Written by seasoned Snowflake practitioners, the book is full of practical guidance and advice to accelerate and mature your Snowflake journey. Working through the examples helps you develop the skill, knowledge, and expertise to expand your organization’s core Snowflake capability and prepare for later incorporation of additional Snowflake features as they become available. Your Snowflake platform will be resilient, fit for purpose, extensible, and guarantee rapid, consistent, and repeatable, pattern-based deployments ready for application delivery. When a Snowflake account is delivered there are no controls, guard rails, external monitoring, nor governance mechanisms baked in. From a large organization perspective, this book explains how to deliver your core Snowflake platform in the form of a Landing Zone, a consistent, templated approach that assumes familiarity with Snowflake core concepts and principles. The book also covers Snowflake from a governance perspective and addresses the “who can see what?” question, satisfying requirements to know for certain that your Snowflake accounts properly adhere to your organization’s data usage policies. The book provides a proven pathway to success by equipping you with skill, knowledge, and expertise to accelerate Snowflake adoption within your organization. The patterns delivered within this book are used for production deployment, and are proven in real-world use. Examples in the book help you succeed in an environment in which governance policies, processes, and procedures oversee and control every aspect of your Snowflake platform development and delivery life cycle. Your environment may not be so exacting, but you’ll still benefit from the rigorous and demanding perspective this book’s authors bring to the table. The book showsyou how to leverage what you already know and adds what you don’t know, all applied to deliver your Snowflake accounts. You will know how to position your organization to deliver consistent Snowflake accounts that are prepared and ready for immediate application development. What You Will Learn Create a common, consistent deployment framework for Snowflake in your organization Enable rapid up-skill and adoption of Snowflake, leveraging the benefits of cloud platforms Develop a deep understanding of Snowflake administration and configuration Implement consistent, approved design patterns that reduce account provisioning times Manage data consumption by monitoring and controlling access to datasets Who This Book Is For Systems administrators charged with delivering a common implementationpattern for all Snowflake accounts within an organization; senior managers looking to simplify the delivery of complex technology into their existing infrastructure; developers seeking to understand guard rails, monitoring, and controls to ensure that Snowflake meets their organization's requirements; sales executives needing to understand how their data usage can be monitored and gain insights into how their data is being consumed; governance colleagues wanting to know who can see each data set, and wanting to identify toxic role combinations, and have confidence that their Snowflake accounts are properly provisioned

Streaming Data Mesh

Data lakes and warehouses have become increasingly fragile, costly, and difficult to maintain as data gets bigger and moves faster. Data meshes can help your organization decentralize data, giving ownership back to the engineers who produced it. This book provides a concise yet comprehensive overview of data mesh patterns for streaming and real-time data services. Authors Hubert Dulay and Stephen Mooney examine the vast differences between streaming and batch data meshes. Data engineers, architects, data product owners, and those in DevOps and MLOps roles will learn steps for implementing a streaming data mesh, from defining a data domain to building a good data product. Through the course of the book, you'll create a complete self-service data platform and devise a data governance system that enables your mesh to work seamlessly. With this book, you will: Design a streaming data mesh using Kafka Learn how to identify a domain Build your first data product using self-service tools Apply data governance to the data products you create Learn the differences between synchronous and asynchronous data services Implement self-services that support decentralized data

Resilient Oracle PL/SQL

As legacy and other critical systems continue to migrate online, the need for continuous operation is imperative. Code has to handle data issues as well as hard external problems today, including outages of networks, storage systems, power, and ancillary systems. This practical guide provides system administrators, DevSecOps engineers, and cloud architects with a concise yet comprehensive overview on how to use PL/SQL to develop resilient database solutions. Integration specialist Stephen B Morris helps you understand the language, build a PL/SQL toolkit, and collect a suite of reusable components and patterns. You'll dive into the benefits of synthesizing the toolkit with a requirements-driven, feature-oriented approach and learn how to produce resilient solutions by synthesizing the PL/SQL toolkit in conjunction with a scale of resilience. Build solid PL/SQL solutions while avoiding common PL/SQL antipatterns Learn why embedding complex business logic in SQL is often a brittle proposition Learn how to recognize and improve weak PL/SQL code Verify PL/SQL code by running data-driven, in-database tests Understand the safe operation, maintenance, and modification of complex PL/SQL systems Learn the benefits of thinking about features rather than just use cases Define good requirements for PL/SQL and hybrid solutions involving PL/SQL and high level languages

IBM Power System AC922 Technical Overview and Introduction

This IBM® Redpaper™ publication is a comprehensive guide that covers the IBM Power System AC922 server (8335-GTH and 8335-GTX models). The Power AC922 server is the next generation of the IBM POWER® processor-based systems, which are designed for deep learning (DL) and artificial intelligence (AI), high-performance analytics, and high-performance computing (HPC). This paper introduces the major innovative Power AC922 server features and their relevant functions: Powerful IBM POWER9™ processors that offer up to 22 cores at up to 2.80 GHz (3.10 GHz turbo) performance with up to 2 TB of memory. IBM Coherent Accelerator Processor Interface (CAPI) 2.0, IBM OpenCAPI™, and second-generation NVIDIA NVLink 2.0 technology for exceptional processor to accelerator intercommunication. Up to six dedicated NVIDIA Tesla V100 graphics processing units (GPUs). This publication is for professionals who want to acquire a better understanding of IBM Power Systems™ products and is intended for the following audiences: Clients Sales and marketing professionals Technical support professionals IBM Business Partners Independent software vendors (ISVs) This paper expands the set of IBM Power Systems documentation by providing a desktop reference that offers a detailed technical description of the Power AC922 server. This paper does not replace the current marketing materials and configuration tools. It is intended as an extra source of information that, together with existing sources, can be used to enhance your knowledge of IBM server solutions.