talk-data.com talk-data.com

Topic

data-engineering

3395

tagged

Activity Trend

1 peak/qtr
2020-Q1 2026-Q1

Activities

3395 activities · Newest first

IBM DS8000 Copy Services: Updated for IBM DS8000 Release 9.1

This IBM® Redbooks® publication helps you plan, install, configure, and manage Copy Services on the IBM DS8000® operating in an IBM Z® or Open Systems environment. This book helps you design and implement a new Copy Services installation or migrate from an existing installation. It includes hints and tips to maximize the effectiveness of your installation, and information about tools and products to automate Copy Services functions. It is intended for anyone who needs a detailed and practical understanding of the DS8000 Copy Services. This edition is an update for the DS8900 Release 9.1. Note that the Safeguarded Copy feature is covered in IBM DS8000 Safeguarded Copy, REDP-5506.

IBM and CMTG Cyber Resiliency: Building an Automated, VMware Aware Safeguarded Copy Solution to Provide Data Resilience

This IBM Blueprint outlines how CMTG and IBM have partnered to provide cyber resilient services to their clients. CMTG is one of Australia's leading private cloud providers based in Perth, Western Australia. The solution is based on IBM Storage FlashSystem, IBM Safeguarded Copy and IBM Storage Copy Data Management. The target audience for this Blueprint is IBM Storage technical specialists and storage admins.

Deciphering Data Architectures

Data fabric, data lakehouse, and data mesh have recently appeared as viable alternatives to the modern data warehouse. These new architectures have solid benefits, but they're also surrounded by a lot of hyperbole and confusion. This practical book provides a guided tour of these architectures to help data professionals understand the pros and cons of each. James Serra, big data and data warehousing solution architect at Microsoft, examines common data architecture concepts, including how data warehouses have had to evolve to work with data lake features. You'll learn what data lakehouses can help you achieve, as well as how to distinguish data mesh hype from reality. Best of all, you'll be able to determine the most appropriate data architecture for your needs. With this book, you'll: Gain a working understanding of several data architectures Learn the strengths and weaknesses of each approach Distinguish data architecture theory from reality Pick the best architecture for your use case Understand the differences between data warehouses and data lakes Learn common data architecture concepts to help you build better solutions Explore the historical evolution and characteristics of data architectures Learn essentials of running an architecture design session, team organization, and project success factors Free from product discussions, this book will serve as a timeless resource for years to come.

IBM Storage Virtualize, IBM Storage FlashSystem, and IBM SAN Volume Controller Security Feature Checklist - For IBM Storage Virtualize 8.6

IBM® Storage Virtualize based storage systems are secure storage platforms that implement various security-related features, in terms of system-level access controls and data-level security features. This document outlines the available security features and options of IBM Storage Virtualize based storage systems. It is not intended as a "how to" or best practice document. Instead, it is a checklist of features that can be reviewed by a user security team to aid in the definition of a policy to be followed when implementing IBM FlashSystem®, IBM SAN Volume Controller, and IBM Storage Virtualize for Public Cloud. IBM Storage Virtualize features the following levels of security to protect against threats and to keep the attack surface as small as possible: The first line of defense is to offer strict verification features that stop unauthorized users from using login interfaces and gaining access to the system and its configuration. The second line of defense is to offer least privilege features that restrict the environment and limit any effect if a malicious actor does access the system configuration. The third line of defense is to run in a minimal, locked down, mode to prevent damage spreading to the kernel and rest of the operating system. The fourth line of defense is to protect the data at rest that is stored on the system from theft, loss, or corruption (malicious or accidental). The topics that are discussed in this paper can be broadly split into two categories: System security: This type of security encompasses the first three lines of defense that prevent unauthorized access to the system, protect the logical configuration of the storage system, and restrict what actions users can perform. It also ensures visibility and reporting of system level events that can be used by a Security Information and Event Management (SIEM) solution, such as IBM QRadar®. Data security: This type of security encompasses the fourth line of defense. It protects the data that is stored on the system against theft, loss, or attack. These data security features include Encryption of Data At Rest (EDAR) or IBM Safeguarded Copy (SGC). This document is correct as of IBM Storage Virtualize 8.6.

Mastering MongoDB 7.0 - Fourth Edition

Discover the many capabilities of MongoDB 7.0 with this comprehensive guide designed to take your database skills to new heights. By exploring advanced features like aggregation pipelines, role-based security, and MongoDB Atlas, you will gain in-depth expertise in modern data management. This book empowers you to create secure, high-performance database applications. What this Book will help me do Understand and implement advanced MongoDB queries for detailed data analysis. Apply optimized indexing techniques to maximize query performance. Leverage MongoDB Atlas for robust monitoring, efficient backups, and advanced integrations. Develop secure applications with role-based access control, auditing, and encryption. Create scalable and innovative solutions using the latest features in MongoDB 7.0. Author(s) Marko Aleksendrić, Arek Borucki, and their co-authors are accomplished experts in database engineering and MongoDB development. They bring collective experience in teaching and practical application of MongoDB solutions across various industries. Their goal is to simplify complex topics, making them approachable and actionable for developers worldwide. Who is it for? This book is written for developers, software engineers, and database administrators with experience in MongoDB who want to deepen their expertise. An understanding of basic database operations and queries is recommended. If you are looking to master advanced concepts and create secure, optimized, and scalable applications, this is the book for you.

IBM Storage Fusion Multicloud Object Gateway

This Redpaper provides an overview of IBM Storage Fusion Multicloud Object Gateway (MCG) and can be used as a quick reference guide for the most common use cases. The intended audience is cloud and application administrators, as well as other technical staff members who wish to learn how MCG works, how to set it up, and usage of a Backing Store or Namespace Store, as well as object caching.

Take Control of iOS & iPadOS Privacy and Security, 4th Edition

Master networking, privacy, and security for iOS and iPadOS! Version 4.2, updated January 29, 2024 Ensuring that your iPhone or iPad’s data remains secure and in your control and that your private data remains private isn’t a battle—if you know what boxes to check and how to configure iOS and iPadOS to your advantage. Take Control of iOS & iPadOS Privacy and Security takes you into the intricacies of Apple’s choices when it comes to networking, data sharing, and encryption—and protecting your personal safety. Substantially updated to cover dozens of changes and new features in iOS 17 and iPadOS 17! Your iPhone and iPad have become the center of your digital identity, and it’s easy to lose track of all the ways in which Apple and other parties access your data legitimately—or without your full knowledge and consent. While Apple nearly always errs on the side of disclosure and permission, many other firms don’t. This book comprehensively explains how to configure iOS 17, iPadOS 17, and iCloud-based services to best protect your privacy with messaging, email, browsing, and much more. The book also shows you how to ensure your devices and data are secure from intrusion from attackers of all types. You’ll get practical strategies and configuration advice to protect yourself against psychological and physical threats, including restrictions on your freedom and safety. For instance, you can now screen images that may contain nude images, while Apple has further enhanced Lockdown Mode to block potential attacks by governments, including your own. Take Control of iOS & iPadOS Privacy and Security covers how to configure the hundreds of privacy and data sharing settings Apple offers in iOS and iPadOS, and which it mediates for third-party apps. Safari now has umpteen different strategies built in by Apple to protect your web surfing habits, personal data, and identity, and new features in Safari, Mail, and Messages that block tracking of your movement across sites, actions on ads, and even when you open and view an email message. In addition to privacy and security, this book also teaches you everything you need to know about networking, whether you’re using 3G, 4G LTE, or 5G cellular, Wi-Fi or Bluetooth, or combinations of all of them; as well as about AirDrop, AirPlay, Airplane Mode, Personal Hotspot, and tethering. You’ll learn how to:

Twiddle 5G settings to ensure the best network speeds on your iPhone or iPad. Master the options for a Personal Hotspot for yourself and in a Family Sharing group. Set up a device securely from the moment you power up a new or newly restored iPhone or iPad. Manage Apple’s built-in second factor verification code generator for extra-secure website and app logins. Create groups of passwords and passkeys you can share securely with other iPhone, iPad, and Mac users. Decide whether Advanced Data Protection in iCloud, an enhanced encryption option that makes nearly all your iCloud data impossible for even Apple to view, makes sense for you. Use passkeys, a high-security but easy-to-use website login system with industry-wide support. Block unknown (and unwanted) callers, iMessage senders, and phone calls, now including FaceTime. Protect your email by using Hide My Email, a iCloud+ tool to generate an address Apple manages and relays messages through for you—now including email used with Apple Pay transactions. Use Safari’s blocking techniques and how to review websites’ attempts to track you, including the latest improvements in iOS 17 and iPadOS 17. Use Communication Safety, a way to alert your children about sensitive images—but now also a tool to keep unsolicited and unwanted images of private parts from appearing on your devices. Understand why Apple might ask for your iPhone, iPad, or Mac password when you log in on a new device using two-factor authentication. Keep yourself safe when en route to a destination by creating a Check In partner who will be alerted if you don’t reach your intended end point or don’t respond within a period of time. Dig into Private Browsing’s several new features in iOS 17/iPadOS 17, designed to let you leave no trace of your identity or actions behind, while protecting your iPhone or iPad from prying eyes, too. Manage data usage across two phone SIMs (or eSIMS) at home and while traveling. Use a hardware encryption key to strongly protect your Apple ID account. Share a Wi-Fi password with nearby contacts and via a QR Code. Differentiate between encrypted data sessions and end-to-end encryption. Stream music and video to other devices with AirPlay 2. Use iCloud+’s Private Relay, a privacy-protecting browsing service that keeps your habits and locations from prying marketing eyes. Deter brute-force cracking by relying on an Accessories timeout for devices physically being plugged in that use USB and other standards. Configure Bluetooth devices. Enjoy enhanced AirDrop options that let you tap two iPhones to transfer files and continue file transfers over the internet when you move out of range. Protect Apple ID account and iCloud data from unwanted access at a regular level and via the new Safety Check, designed to let you review or sever digital connections with people you know who may wish you harm.

Building Information Modeling

This book presents how Building Information Modeling (BIM) and the use of shared representation of built assets facilitate design, construction and operation processes (ISO 19650). The modeling of public works data disrupts the art of construction. Written by both academics and engineers who are heavily involved in the French research project Modélisation des INformations INteropérables pour les INfrastructues Durables (MINnD) as well as in international standardization projects, this book presents the challenges of BIM from theoretical and practical perspectives. It provides knowledge for evolving in an ecosystem of federated models and common data environments, which are the basis of the platforms and data spaces. BIM makes it possible to handle interoperability very concretely, using open standards, which lead to openBIM. The use of a platform allows for the merging of business software and for approaches such as a Geographic Information System (GIS) to be added to the processes. In organizations, BIM meets the life cycles of structures and circular economy. It is not only a technique that reshapes cooperation and trades around a digital twin but can also disrupt organizations and business models.

IBM SAN Volume Controller Model SV3 Product Guide (for IBM Storage Virtualize V8.6)

This IBM® Redpaper® Product Guide describes the IBM SAN Volume Controller model SV3 solution, which is a next-generation IBM SAN Volume Controller. Built with IBM Storage Virtualize software and part of the IBM Storage family, IBM SAN Volume Controller is an enterprise-class storage system. It helps organizations achieve better data economics by supporting the large-scale workloads that are critical to success. Data centers often contain a mix of storage systems. This situation can arise as a result of company mergers or as a deliberate acquisition strategy. Regardless of how they arise, mixed configurations add complexity to the data center. Different systems have different data services, which make it difficult to move data from one to another without updating automation. Different user interfaces increase the need for training and can make errors more likely. Different approaches to hybrid cloud complicate modernization strategies. Also, many different systems mean more silos of capacity, which can lead to inefficiency. To simplify the data center and to improve flexibility and efficiency in deploying storage, enterprises of all types and sizes turn to IBM SAN Volume Controller, which is built with IBM Spectrum Virtualize software. This software simplifies infrastructure and eliminates differences in management, function, and even hybrid cloud support. IBM SAN Volume Controller introduces a common approach to storage management, function, replication, and hybrid cloud that is independent of storage type. It is the key to modernizing and revitalizing your storage, but is as easy to understand. IBM SAN Volume Controller provides a rich set of software-defined storage (SDS) features that are delivered by IBM Storage Virtualize, including the following examples: Data reduction and deduplication Dynamic tiering Thin-provisioning Snapshots Cloning Replication and data copy services Data-at-rest encryption Cyber resilience Transparent Cloud Tiering IBM HyperSwap® including three-site replication for high availability (HA) This Redpaper applies to IBM Storage Virtualize V8.6.

PostgreSQL Query Optimization: The Ultimate Guide to Building Efficient Queries

Write optimized queries. This book helps you write queries that perform fast and deliver results on time. You will learn that query optimization is not a dark art practiced by a small, secretive cabal of sorcerers. Any motivated professional can learn to write efficient queries from the get-go and capably optimize existing queries. You will learn to look at the process of writing a query from the database engine’s point of view, and know how to think like the database optimizer. The book begins with a discussion of what a performant system is and progresses to measuring performance and setting performance goals. It introduces different classes of queries and optimization techniques suitable to each, such as the use of indexes and specific join algorithms. You will learn to read and understand query execution plans along with techniques for influencing those plans for better performance. The book also covers advanced topics such as the use of functions and procedures, dynamic SQL, and generated queries. All of these techniques are then used together to produce performant applications, avoiding the pitfalls of object-relational mappers. This second edition includes new examples using Postgres 15 and the newest version of the PostgresAir database. It includes additional details and clarifications about advanced topics, and covers configuration parameters in greater depth. Finally, it makes use of advancements in NORM, using automatically generated functions. What You Will Learn Identify optimization goals in OLTP and OLAP systems Read and understand PostgreSQL execution plans Distinguish between short queries and long queries Choose the right optimization technique for each query type Identify indexes that will improve query performance Optimize full table scans Avoid the pitfalls of object-relational mapping systems Optimize the entire application rather than just database queries Who This Book Is For IT professionals working in PostgreSQL who want to develop performant and scalable applications, anyone whose job title contains the words “database developer” or “database administrator" or who is a backend developer charged with programming database calls, and system architects involved in the overall design of application systems running against a PostgreSQL database

Mastering MongoDB 7.0 - Fourth Edition

Mastering MongoDB 7.0 is your in-depth resource for learning MongoDB 7.0, the powerful NoSQL database designed for developers. Gain expertise in database architecture, data management, and modern features like MongoDB Atlas. By reading this book, you'll acquire the essential skills needed for building efficient, scalable, and secure applications. What this Book will help me do Develop expert-level skills in crafting advanced queries and managing complex data tasks in MongoDB. Learn to design efficient schemas and optimize indexing to maximize database performance. Integrate applications seamlessly with MongoDB Atlas, mastering its monitoring and backup tools. Implement robust security with RBAC, auditing strategies, and comprehensive encryption. Explore the latest MongoDB 7.0 features, including Atlas Vector Search, for modern applications. Author(s) Marko Aleksendrić, Arek Borucki, and co-authors are recognized MongoDB experts with years of hands-on experience. They bring together their expertise to deliver a practical guide filled with real-world insights that help developers advance their MongoDB skills. Their collaborative writing ensures comprehensive coverage of MongoDB 7.0 tools and techniques. Who is it for? This book is written for software developers, database administrators, and engineers who have intermediate knowledge of MongoDB and want to extend their expertise. Whether you are developing scalable applications, managing data systems, or ensuring database security, this book offers advanced guidance for achieving your professional goals with MongoDB.

Data Observability for Data Engineering

"Data Observability for Data Engineering" introduces you to the foundational concepts of observing and validating data pipeline health. With real-world projects and Python code examples, you'll gain hands-on experience in improving data quality and minimizing risks, enabling you to implement strategies that ensure accuracy and reliability in your data systems. What this Book will help me do Master data observability techniques to monitor and validate data pipelines effectively. Learn to collect and analyze meaningful metrics to gauge and improve data quality. Develop skills in Python programming specific to applying data concepts such as observable data state. Address scalability challenges using state-of-the-art observability frameworks and practices. Enhance your ability to manage and optimize data workflows ensuring seamless operation from start to end. Author(s) Authors Michele Pinto and Sammy El Khammal bring a wealth of experience in data engineering and observing scalable data systems. Pinto specializes in constructing robust analytics platforms while Khammal offers insights into integrating software observability into massive pipelines. Their collaborative writing style ensures readers find both practical advice and theoretical foundations. Who is it for? This book is geared toward data engineers, architects, and scientists who seek to confidently handle pipeline challenges. Whether you're addressing specific issues or wish to introduce proactive measures in your team, this guide meets the needs of those ready to leverage observability as a key practice.

Handbook of Geospatial Artificial Intelligence

Geospatial Artificial Intelligence (GeoAI) is the integration of geospatial studies and AI using machine learning and deep learning technologies. This comprehensive handbook explains and discusses key fundamental concepts, methods, models, technologies of GeoAI, recent advances, research tools, and applications in different fields.

Redis Stack for Application Modernization

In "Redis Stack for Application Modernization," you will explore how the Redis Stack extends traditional Redis capabilities, allowing you to innovate in building real-time, scalable, multi-model applications. Through practical examples and hands-on sessions, this book equips you with skills to manage, implement, and optimize data flows and database features. What this Book will help me do Learn how to use Redis Stack for handling real-time data with JSON, hash, and other document types. Discover modern techniques for performing vector similarity searches and hybrid workflows. Become proficient in integrating Redis Stack with programming languages like Java, Python, and Node.js. Gain skills to configure Redis Stack server for scalability, security, and high availability. Master RedisInsight for data visualization, analysis, and efficient database management. Author(s) Luigi Fugaro and None Ortensi are experienced software professionals with deep expertise in database systems and application architecture. They bring years of experience working with Redis and developing real-world applications. Their hands-on approach to teaching and real-world examples make this book a valuable resource for professionals in the field. Who is it for? This book is ideal for database administrators, developers, and architects looking to leverage Redis Stack for real-time multi-model applications. It requires a basic understanding of Redis and any programming language such as Python or Java. If you wish to modernize your applications and efficiently manage databases, this book is for you.

Architecting a Modern Data Warehouse for Large Enterprises: Build Multi-cloud Modern Distributed Data Warehouses with Azure and AWS

Design and architect new generation cloud-based data warehouses using Azure and AWS. This book provides an in-depth understanding of how to build modern cloud-native data warehouses, as well as their history and evolution. The book starts by covering foundational data warehouse concepts, and introduces modern features such as distributed processing, big data storage, data streaming, and processing data on the cloud. You will gain an understanding of the synergy, relevance, and usage data warehousing standard practices in the modern world of distributed data processing. The authors walk you through the essential concepts of Data Mesh, Data Lake, Lakehouse, and Delta Lake. And they demonstrate the services and offerings available on Azure and AWS that deal with data orchestration, data democratization, data governance, data security, and business intelligence. After completing this book, you will be ready to design and architect enterprise-grade, cloud-based modern data warehouses using industry best practices and guidelines. What You Will Learn Understand the core concepts underlying modern data warehouses Design and build cloud-native data warehousesGain a practical approach to architecting and building data warehouses on Azure and AWS Implement modern data warehousing components such as Data Mesh, Data Lake, Delta Lake, and Lakehouse Process data through pandas and evaluate your model’s performance using metrics such as F1-score, precision, and recall Apply deep learning to supervised, semi-supervised, and unsupervised anomaly detection tasks for tabular datasets and time series applications Who This Book Is For Experienced developers, cloud architects, and technology enthusiasts looking to build cloud-based modern data warehouses using Azure and AWS

What is New in DFSMSrmm

DFSMSrmm is an IBM z/OS feature that is a fully functioning tape management system to manage your removable media. In the last decade, many enhancements were made to DFSMSrmm. This IBM Redbooks publication is intended to help you configure and use the newer functions and features that are now available. Discussion of the new features is included along with use cases. Hints and tips of various common DFSMSrmm problems and useful configuration and reporting JCL also are included. This publication is intended as a supplement to DFSMSrmm Primer, SG24-5983, which is still the recommended starting point for any users new to DFSMSrmm.

Elasticsearch in Action, Second Edition

Build powerful, production-ready search applications using the incredible features of Elasticsearch. In Elasticsearch in Action, Second Edition you will discover: Architecture, concepts, and fundamentals of Elasticsearch Installing, configuring, and running Elasticsearch and Kibana Creating an index with custom settings Data types, mapping fundamentals, and templates Fundamentals of text analysis and working with text analyzers Indexing, deleting, and updating documents Indexing data in bulk, and reindexing and aliasing operations Learning search concepts, relevancy scores, and similarity algorithms Elasticsearch in Action, Second Edition teaches you to build scalable search applications using Elasticsearch. This completely new edition explores Elasticsearch fundamentals from the ground up. You’ll deep dive into design principles, search architectures, and Elasticsearch’s essential APIs. Every chapter is clearly illustrated with diagrams and hands-on examples. You’ll even explore real-world use cases for full text search, data visualizations, and machine learning. Plus, its comprehensive nature means you’ll keep coming back to the book as a handy reference! About the Technology Create fully professional-grade search engines with Elasticsearch and Kibana! Rewritten for the latest version of Elasticsearch, this practical book explores Elasticsearch’s high-level architecture, reveals infrastructure patterns, and walks through the search and analytics capabilities of numerous Elasticsearch APIs. About the Book Elasticsearch in Action, Second Edition teaches you how to add modern search features to websites and applications using Elasticsearch 8. In it, you’ll quickly progress from the basics of installation and configuring clusters, to indexing documents, advanced aggregations, and putting your servers into production. You’ll especially appreciate the mix of technical detail with techniques for designing great search experiences. What's Inside Understanding search architecture Full text and term-level search queries Analytics and aggregations High-level visualizations in Kibana Configure, scale, and tune clusters About the Reader For application developers comfortable with scripting and command-line applications. About the Author Madhusudhan Konda is a full-stack lead engineer, architect, mentor, and conference speaker. He delivers live online training on Elasticsearch and the Elastic Stack. Quotes Madhu’s passion comes across in the depth and breadth of this book, the enthusiastic tone, and the hands-on examples. I hope you will take what you have read and put it ‘in action’. - From the Foreword by Shay Banon, Founder of Elasticsearch Practical and well-written. A great starting point for beginners and a comprehensive guide for more experienced professionals. - Simona Russo, Serendipity The author’s excitement is evident from the first few paragraphs. Couple that with extensive experience and technical prowess, and you have an instant classic. - Herodotos Koukkides and Semi Koen, Global Japanese Financial Institution

IBM Z Server Time Protocol Guide

Server Time Protocol (STP) is a server-wide facility that is implemented in the Licensed Internal Code (LIC) of the IBM Z® platform. It provides improved time synchronization in a sysplex or non-sysplex configuration. This IBM Redbooks® publication is intended for infrastructure architects and system programmers who need to understand the STP functions. Readers are expected to be familiar with IBM Z technology and terminology. This book provides planning and implementation information for STP functions and associated software support for the IBM z16™, IBM z15®, and IBM z14® platforms.

PostgreSQL 16 Administration Cookbook

This cookbook is a comprehensive guide to mastering PostgreSQL 16 database administration. With over 180 practical recipes, this book covers everything from query performance and backup strategies to replication and high availability. You'll gain hands-on expertise in solving real-world challenges while leveraging the new and improved features of PostgreSQL 16. What this Book will help me do Perform efficient batch processing with Postgres' SQL MERGE statement. Implement parallel transaction processes using logical replication. Enhance database backups and recovery with advanced compression techniques. Monitor and fine-tune database performance for optimal operation. Apply new PostgreSQL 16 features for secure and reliable databases. Author(s) The team of authors, including Gianni Ciolli, Boriss Mejías, Jimmy Angelakos, Vibhor Kumar, and Simon Riggs, bring years of experience in PostgreSQL database management and development. Their expertise spans professional system administration, academic research, and contributions to PostgreSQL development. Their collaborative insights enrich this comprehensive guide. Who is it for? This book is ideal for PostgreSQL database administrators seeking advanced techniques, data architects managing PostgreSQL in production, and developers interested in mastering PostgreSQL 16. Whether you're an experienced DBA upgrading to PostgreSQL 16 or a newcomer looking for practical recipes, this book provides valuable strategies and solutions.