talk-data.com talk-data.com

Topic

data-engineering

3395

tagged

Activity Trend

1 peak/qtr
2020-Q1 2026-Q1

Activities

3395 activities · Newest first

Professional Azure SQL Managed Database Administration - Third Edition

Professional Azure SQL Managed Database Administration is a comprehensive guide to mastering data management with Azure's managed database services. Packed with real-world exercises and updated to cover the latest Azure features, this book provides actionable insights into migration, performance tuning, scaling, and securing Azure SQL databases. What this Book will help me do Master the configuration and pricing options for Azure SQL databases to make cost-effective choices. Learn the processes to provision new SQL databases or migrate existing on-premises SQL databases to Azure. Acquire skills in implementing high availability and disaster recovery for ensuring data resilience. Understand the strategies for monitoring, tuning, and optimizing the performance of Azure SQL databases. Discover techniques for scaling uses through elastic pools and securing databases comprehensively. Author(s) Ahmad Osama and Shashikant Shakya are experienced professionals in SQL Server and Azure SQL technologies. With decades of combined experience in database administration and cloud computing, they bring a depth of understanding to the content of this book. Their hands-on teaching approach is evident in the practical exercises and real-world scenarios included. Who is it for? This book is specifically tailored for database administrators, developers, and application developers looking to leverage Azure SQL databases. If you are tasked with migrating applications to the cloud or ensuring top performance and resilience for cloud databases, you will find this book highly valuable. Prior experience with on-premises SQL services will help contextualize the content, making it suitable for professionals with intermediate SQL experience. Readers aiming to deepen their Azure SQL expertise will also greatly benefit.

Enhanced Cyber Resilience Solution by Threat Detection using IBM Cloud Object Storage System and IBM QRadar SIEM

This Solution Redpaper™ publication explains how the features of IBM Cloud® Object Storage System reduces the effect of incidents on business data when combined with log analysis, deep inspection, and detection of threats that IBM QRadar SIEM provides. This paper also demonstrates how to integrate IBM Cloud Object Storage's access logs with IBM QRadar SIEM. An administrator can monitor, inspect, detect, and derive insights for identifying potential threats to the data that is stored on IBM Cloud Object Storage. Also, IBM QRadar SIEM can proactively trigger cyber resiliency workflow in IBM Cloud Object Storage remotely to protect the data based on threat detection. This publication is intended for chief technology officers, solution and security architects, and systems administrators.

Securing Your Critical Workloads with IBM Hyper Protect Services

Many organizations must protect their mission-critical applications in production, but security threats can also surface during the development and pre-production phases. Also, during deployment and production, insiders who manage the infrastructure that hosts critical applications can pose a threat given their super-user credentials and level of access to secrets or encryption keys. Organizations must incorporate secure design practices in their development operations and embrace DevSecOps to protect their applications from the vulnerabilities and threat vectors that can compromise their data and potentially threaten their business. IBM® Cloud Hyper Protect Services provide built-in data-at-rest and data-in-flight protection to help developers easily build secure cloud applications by using a portfolio of cloud services that are powered by IBM LinuxONE. The LinuxONE platform ensures that client data is always encrypted, whether at rest or in transit. This feature gives customers complete authority over sensitive data and associated workloads (which restricts access, even for cloud admins) and helps them meet regulatory compliance requirements. LinuxONE also allows customers to build mission-critical applications that require quick time to market and dependable rapid expansion. The purpose of this IBM Redbooks® publication is to: Introduce the IBM Hyper Protect Services that are running on IBM LinuxONE on the IBM Cloud™ and on-premises Provide high-level design architectures Describe deployment best practices Provide guides to getting started and examples of the use of the Hyper Protect Services The target audience for this book is IBM Hyper Protect Virtual Services technical specialists, IT architects, and system administrators.

PostgreSQL 13 Cookbook

The "PostgreSQL 13 Cookbook" is your step-by-step resource for mastering PostgreSQL 13. Explore over 120 recipes, solving both common and advanced database management challenges, with a focus on high performance, fault tolerance, and cutting-edge features. What this Book will help me do Master the implementation of backup and recovery strategies tailored for PostgreSQL 13. Set up robust high availability clusters ensuring seamless failover with PostgreSQL replication features. Improve performance using optimization techniques specific to PostgreSQL 13 databases. Secure your databases with advanced authentication, encryption, and auditing measures. Analyze and monitor PostgreSQL servers to identify performance bottlenecks and maintain uptime efficiently. Author(s) Vallarapu Naga Avinash Kumar is an experienced PostgreSQL architect and developer who brings years of expertise in designing and managing enterprise-level databases. He has authored resources that simplify complex technical concepts for readers. His meticulous and straightforward writing approach empowers readers to skillfully apply PostgreSQL concepts in real-world scenarios. Who is it for? This book is perfect for database administrators, architects, and developers aiming to master PostgreSQL 13 capabilities. If you have prior experience with PostgreSQL and SQL, this cookbook will be a reliable reference to solve challenges and optimize your database solutions. If you're designing or managing databases, you'll find practical insights and actionable recipes tailored to your needs.

Snowflake Cookbook

The "Snowflake Cookbook" is your guide to mastering Snowflake's unique cloud-centric architecture. This book provides detailed recipes for building modern data pipelines, configuring efficient virtual warehouses, ensuring robust data protection, and optimizing cost-performance-all while leveraging Snowflake's distinctive features such as data sharing and time travel. What this Book will help me do Set up and configure Snowflake's architecture for optimized performance and cost efficiency. Design and implement robust data pipelines using SQL and Snowflake's specialized features. Secure, manage, and share data efficiently with built-in Snowflake capabilities. Apply performance tuning techniques to enhance your Snowflake implementations. Extend Snowflake's functionality with tools like Spark Connector for advanced workflows. Author(s) Hamid Mahmood Qureshi and Hammad Sharif are both seasoned experts in data warehousing and cloud computing technologies. With extensive experience implementing analytics solutions, they bring a hands-on approach to teaching Snowflake. They are ardent proponents of empowering readers towards creating effective and scalable data solutions. Who is it for? This book is perfect for data warehouse developers, data analysts, cloud architects, and anyone managing cloud data solutions. If you're familiar with basic database concepts or just stepping into Snowflake, you'll find practical guidance here to deepen your understanding and functional expertise in cloud data warehousing.

Implementing IBM VM Recovery Manager for IBM Power Systems

This IBM® Redbooks® publication describes the IBM VM Recovery Manager for Power Systems, and addresses topics to help answer customers' complex high availability (HA) and disaster recovery (DR) requirements for IBM AIX® and Linux on IBM Power Systems servers to help maximize systems' availability and resources, and provide technical documentation to transfer the how-to skills to users and support teams. The IBM VM Recovery Manager for Power Systems product is an easy to use and economical HA and DR solution. Automation software, installation services, and remote-based software support help you streamline the process of recovery, which raises availability and recovery testing, and maintains a state-of-the-art HA and DR solution. Built-in functions and IBM Support can decrease the need for expert-level skills and shorten your recovery time objective (RTO), improve your recovery point objective (RPO), optimize backups, and better manage growing data volumes. This book examines the IBM VM Recovery Manager solution, tools, documentation, and other resources that are available to help technical teams develop, implement, and support business resilience solutions in IBM VM Recovery Manager for IBM Power Systems environments. This publication targets technical professionals (consultants, technical support staff, IT Architects, and IT Specialists) who are responsible for providing HA and DR solutions and support for IBM Power Systems.

Learn FileMaker Pro 19: The Comprehensive Guide to Building Custom Databases

Discover how easy it is to create multi-user, cross-platform custom solutions with FileMaker Pro, the relational database platform published by Apple subsidiary Claris International, Inc. Meticulously rewritten with clearer lessons, more real-world examples and updated to include feature changes introduced in recent versions, this book makes it easier to get started planning, building and deploying a custom database solution. The material is presented in an easy to follow manner with each chapter building on the last. After an initial review of the user environment and application basics, it begins a deep exploration of the integrated development environment that seamlessly combines the full stack of data table schema, business logic and interface layers into one visual programming experience. This book includes everything needed to get started building custom databases and contains advanced material that seasoned professionals will appreciate. Written bya professional developer with decades of real-world experience, Learn FileMaker Pro 19 is your comprehensive learning and reference guide. Join millions of users and developers worldwide in achieving a new level of workflow efficiency with FileMaker Pro. What You’ll Learn Discover interface and feature changes in FileMaker 17-19 Create and maintain healthy files Plan and create custom tables, fields, relationships Write calculations using built-in and custom functions Build recursive and repeating formulas Discover advanced features using cURL, JSON, SQL, ODBC and FM URL Manipulate data files in the computer directory with scripts Deploy solutions to a server and share with desktop, iOS and web clients Who This Book Is For Casual programmers, full time consultants, and IT professionals

Privacy, Regulations, and Cybersecurity

Protect business value, stay compliant with global regulations, and meet stakeholder demands with this privacy how-to Privacy, Regulations, and Cybersecurity: The Essential Business Guide is your guide to understanding what “privacy” really means in a corporate environment: how privacy is different from cybersecurity, why privacy is essential for your business, and how to build privacy protections into your overall cybersecurity plan. First, author Chris Moschovitis walks you through our evolving definitions of privacy, from the ancient world all the way to the General Law on Data Protection (GDPR). He then explains—in friendly, accessible language—how to orient your preexisting cybersecurity program toward privacy, and how to make sure your systems are compliant with current regulations. This book—a sequel to Moschovitis’ well-received Cybersecurity Program Development for Business—explains which regulations apply in which regions, how they relate to the end goal of privacy, and how to build privacy into both new and existing cybersecurity programs. Keeping up with swiftly changing technology and business landscapes is no easy task. Moschovitis provides down-to-earth, actionable advice on how to avoid dangerous privacy leaks and protect your valuable data assets. Learn how to design your cybersecurity program with privacy in mind Apply lessons from the GDPR and other landmark laws Remain compliant and even get ahead of the curve, as privacy grows from a buzzword to a business must Learn how to protect what’s of value to your company and your stakeholders, regardless of business size or industry Understand privacy regulations from a business standpoint, including which regulations apply and what they require Think through what privacy protections will mean in the post-COVID environment Whether you’re new to cybersecurity or already have the fundamentals, this book will help you design and build a privacy-centric, regulation-compliant cybersecurity program.

Building Custom Tasks for SQL Server Integration Services: The Power of .NET for ETL for SQL Server 2019 and Beyond

Build custom SQL Server Integration Services (SSIS) tasks using Visual Studio Community Edition and C#. Bring all the power of Microsoft .NET to bear on your data integration and ETL processes, and for no added cost over what you’ve already spent on licensing SQL Server. New in this edition is a demonstration deploying a custom SSIS task to the Azure Data Factory (ADF) Azure-SSIS Integration Runtime (IR). All examples in this new edition are implemented in C#. Custom task developers are shown how to implement custom tasks using the widely accepted and default language for .NET development. Why are custom components necessary? Because even though the SSIS catalog of built-in tasks and components is a marvel of engineering, gaps remain in the available functionality. One such gap is a constraint of the built-in SSIS Execute Package Task, which does not allow SSIS developers to select SSIS packages from other projects in the SSIS Catalog. Examples in this bookshow how to create a custom Execute Catalog Package task that allows SSIS developers to execute tasks from other projects in the SSIS Catalog. Building on the examples and patterns in this book, SSIS developers may create any task to which they aspire, custom tailored to their specific data integration and ETL needs. What You Will Learn Configure and execute Visual Studio in the way that best supports SSIS task development Create a class library as the basis for an SSIS task, and reference the needed SSIS assemblies Properly sign assemblies that you create in order to invoke them from your task Implement source code control via Azure DevOps, or your own favorite tool set Troubleshoot and execute custom tasks as part of your own projects Create deployment projects (MSIs) for distributing code-complete tasks Deploy custom tasks to Azure Data Factory Azure-SSIS IRs in the cloud Create advanced editors for custom task parameters Who This Book Is For For database administrators and developers who are involved in ETL projects built around SQL Server Integration Services (SSIS). Readers do not need a background in software development with C#. Most important is a desire to optimize ETL efforts by creating custom-tailored tasks for execution in SSIS packages, on-premises or in ADF Azure-SSIS IRs.

IBM Spectrum Scale and IBM Elastic Storage System Network Guide

High-speed I/O workloads are moving away from the SAN to Ethernet and IBM® Spectrum Scale is pushing the network limits. The IBM Spectrum® Scale team discovered that many infrastructure Ethernet networks that were used for years to support various applications are not designed to provide a high-performance data path concurrently to many clients from many servers. IBM Spectrum Scale is not the first product to use Ethernet for storage access. Technologies, such as Fibre Channel over Ethernet (FCoE), scale out NAS, and IP connected storage (iSCSI and others) use Ethernet though IBM Spectrum Scale as the leader in parallel I/O performance, which provides the best performance and value when used on a high-performance network. This IBM Redpaper publication is based on lessons that were learned in the field by deploying IBM Spectrum Scale on Ethernet and InfiniBand networks. This IBM Redpaper® publication answers several questions, such as, "How can I prepare my network for high performance storage?", "How do I know when I am ready?", and "How can I tell what is wrong?" when deploying IBM Spectrum Scale and IBM Elastic Storage® Server (ESS). This document can help IT architects get the design correct from the beginning of the process. It also can help the IBM Spectrum Scale administrator work effectively with the networking team to quickly resolve issues.

Data Pipelines Pocket Reference

Data pipelines are the foundation for success in data analytics. Moving data from numerous diverse sources and transforming it to provide context is the difference between having data and actually gaining value from it. This pocket reference defines data pipelines and explains how they work in today's modern data stack. You'll learn common considerations and key decision points when implementing pipelines, such as batch versus streaming data ingestion and build versus buy. This book addresses the most common decisions made by data professionals and discusses foundational concepts that apply to open source frameworks, commercial products, and homegrown solutions. You'll learn: What a data pipeline is and how it works How data is moved and processed on modern data infrastructure, including cloud platforms Common tools and products used by data engineers to build pipelines How pipelines support analytics and reporting needs Considerations for pipeline maintenance, testing, and alerting

Mastering Kafka Streams and ksqlDB

Working with unbounded and fast-moving data streams has historically been difficult. But with Kafka Streams and ksqlDB, building stream processing applications is easy and fun. This practical guide shows data engineers how to use these tools to build highly scalable stream processing applications for moving, enriching, and transforming large amounts of data in real time. Mitch Seymour, data services engineer at Mailchimp, explains important stream processing concepts against a backdrop of several interesting business problems. You'll learn the strengths of both Kafka Streams and ksqlDB to help you choose the best tool for each unique stream processing project. Non-Java developers will find the ksqlDB path to be an especially gentle introduction to stream processing. Learn the basics of Kafka and the pub/sub communication pattern Build stateless and stateful stream processing applications using Kafka Streams and ksqlDB Perform advanced stateful operations, including windowed joins and aggregations Understand how stateful processing works under the hood Learn about ksqlDB's data integration features, powered by Kafka Connect Work with different types of collections in ksqlDB and perform push and pull queries Deploy your Kafka Streams and ksqlDB applications to production

Implementing the IBM SAN Volume Controller with IBM Spectrum Virtualize V8.3.1

This IBM® Redbooks® publication is a detailed technical guide to the IBM System Storage™ SAN Volume Controller, which is powered by IBM Spectrum® Virtualize V8.3.1. IBM SAN Volume Controller is a virtualization appliance solution that maps virtualized volumes that are visible to hosts and applications to physical volumes on storage devices. Each server within the storage area network (SAN) has its own set of virtual storage addresses that are mapped to physical addresses. If the physical addresses change, the server continues running by using the same virtual addresses that it had before. Therefore, volumes or storage can be added or moved while the server is still running. The IBM virtualization technology improves the management of information at the block level in a network, which enables applications and servers to share storage devices on a network.

IBM Integrated Synchronization: Incremental Updates Unleashed

The IBM® Db2® Analytics Accelerator (Accelerator) is a logical extension of Db2 for IBM z/OS® that provides a high-speed query engine that efficiently and cost-effectively runs analytics workloads. The Accelerator is an integrated back-end component of Db2 for z/OS. Together, they provide a hybrid workload-optimized database management system that seamlessly manages queries that are found in transactional workloads to Db2 for z/OS and queries that are found in analytics applications to Accelerator. Each query runs in its optimal environment for maximum speed and cost efficiency. The incremental update function of Db2 Analytics Accelerator for z/OS updates Accelerator-shadow tables continually. Changes to the data in original Db2 for z/OS tables are propagated to the corresponding target tables with a high frequency and a brief delay. Query results from Accelerator are always extracted from recent, close-to-real-time data. An incremental update capability that is called IBM InfoSphere® Change Data Capture (InfoSphere CDC) is provided by IBM InfoSphere Data Replication for z/OS up to Db2 Analytics Accelerator V7.5. Since then, an extra new replication protocol between Db2 for z/OS and Accelerator that is called IBM Integrated Synchronization was introduced. With Db2 Analytics Accelerator V7.5, customers can choose which one to use. IBM Integrated Synchronization is a built-in product feature that you use to set up incremental updates. It does not require InfoSphere CDC, which is bundled with IBM Db2 Analytics Accelerator. In addition, IBM Integrated Synchronization has more advantages: Simplified administration, packaging, upgrades, and support. These items are managed as part of the Db2 for z/OS maintenance stream. Updates are processed quickly. Reduced CPU consumption on the mainframe due to a streamlined, optimized design where most of the processing is done on the Accelerator. This situation provides reduced latency. Uses IBM Z® Integrated Information Processor (zIIP) on Db2 for z/OS, which leads to reduced CPU costs on IBM Z and better overall performance data, such as throughput and synchronized rows per second. On z/OS, the workload to capture the table changes was reduced, and the remainder can be handled by zIIPs. With the introduction of an enterprise-grade Hybrid Transactional Analytics Processing (HTAP) enabler that is also known as the Wait for Data protocol, the integrated low latency protocol is now enabled to support more analytical queries running against the latest committed data. IBM Db2 for z/OS Data Gate simplifies delivering data from IBM Db2 for z/OS to IBM Cloud® Pak® for Data for direct access by new applications. It uses the special-purpose integrated synchronization protocol to maintain data currency with low latency between Db2 for z/OS and dedicated target databases on IBM Cloud Pak for Data.

IBM Power Systems H922 and H924 Technical Overview and Introduction

This IBM® Redpaper™ publication is a comprehensive guide that covers the IBM Power System H922 (9223-22S), and IBM Power System H924 (9223-42S) servers that support memory-intensive workloads, such as SAP HANA, and deliver superior price and performance for mission-critical applications in IBM AIX®, IBM i, and Linux® operating systems. The goal of this paper is to provide a hardware architecture analysis and highlight the changes, new technologies, and major features that are being introduced in these systems' 2020 release, such as the following examples: Availability of new IBM POWER9™ processor configurations for the number of cores per socket. More performance by using industry-leading IBM Peripheral Component Interconnect® Express (PCIe) Gen4 slots. Enhanced internal disk configuration options, with up to 14 NVMe adapters (four U.2 NVMe plus up to 10 PCIe add-in cards). Twice as fast back-end I/O enables seamless maximum speed and throughput between on-premises and multiple public cloud infrastructures with high availability (HA). This publication is for professionals who want to acquire a better understanding of IBM Power Systems products. The intended audience includes the following roles: Clients Sales and marketing professionals Technical support professionals IBM Business Partners Independent software vendors (ISVs) This paper expands the current set of IBM Power Systems documentation by providing a desktop reference that offers a detailed technical description of the Power H922 and Power H924 systems.

MySQL Concurrency: Locking and Transactions for MySQL Developers and DBAs

Know how locks work in MySQL and how they relate to transactions. This book explains the major role that locks play in database systems, showing how locks are essential in allowing high-concurrency workloads. You will learn about lock access levels and lock granularities from the user level as well as table locks to record and gap locks. Most importantly, the book covers troubleshooting techniques when locking becomes a pain point. Several of the lock types in MySQL have a duration of a transaction. For this reason, it is important to understand how transactions work. This book covers the basics of transactions as well as transaction isolation levels and how they affect locking. The book is meant to be your go-to resource for solving locking contention and similar problems in high-performance MySQL database applications. Detecting locking issues when they occur is the first key to resolving such issues. MySQL Concurrency provides techniques for detecting locking issues such as contention. The book shows how to analyze locks that are causing contention to see why those locks are in place. A collection of six comprehensive case studies combine locking and transactional theory with realistic lock conflicts. The case studies walk you through the symptoms to look for in order to identify which issue you are facing, the cause of the conflict, its analysis, solution, and how to prevent the issue in the future. What You Will Learn Understand which lock types exist in MySQL and how they are used Choose the best transaction isolation level for a given transaction Detect and analyze lock contention when it occurs Reduce locking issues in your applications Resolve deadlocks between transactions Resolve InnoDB record-level locking issues Resolve issues from metadata and schema locks Who This Book Is For Database administrators and SQL developers who are familiar with MySQL and want to gain a better understanding of locking and transactions as well as how to work with them. While some experience with MySQL is required, no prior knowledge of locks and transactions is needed.

High Performance SQL Server: Consistent Response for Mission-Critical Applications

Design and configure SQL Server instances and databases in support of high-throughput, mission-critical applications providing consistent response times in the face of variations in numbers of users and query volumes. In this new edition, with over 100 pages of additional content, every original chapter has been updated for SQL Server 2019, and the book also includes two new chapters covering SQL Server on Linux and Intelligent Query Processing. This book shows you how to configure SQL Server and design your databases to support a given instance and workload. You will learn advanced configuration options, in-memory technologies, storage and disk configuration, and more, all aimed toward enabling your desired application performance and throughput. Configuration doesn’t stop with implementation. Workloads change over time, and other impediments can arise to thwart desired performance. High Performance SQL Server covers monitoring and troubleshooting to aid you in detecting and fixing production performance problems and minimizing application outages. You will learn about a variety of tools, ranging from the traditional wait analysis methodology to the query store or indexing, and you will learn how improving performance is an iterative process. This book is an excellent complement to query performance tuning books and provides the other half of what you need to know by focusing on configuring the instances on which mission-critical queries are executed. What You Will Learn Understand SQL Server's database engine and how it processes queries Configure instances in support of high-throughput applications Provide consistent response times to varying user numbers and query volumes Design databases for high-throughput applications with focus on performance Record performance baselines and monitor SQL Server instances against them Troubleshot and fix performance problems Who This Book Is For SQL Server database administrators, developers, and data architects. The book is also of use to system administrators who are managing and are responsible for the physical servers on which SQL Server instances are run.

Data Accelerator for AI and Analytics

This IBM® Redpaper publication focuses on data orchestration in enterprise data pipelines. It provides details about data orchestration and how to address typical challenges that customers face when dealing with large and ever-growing amounts of data for data analytics. While the amount of data increases steadily, artificial intelligence (AI) workloads must speed up to deliver insights and business value in a timely manner. This paper provides a solution that addresses these needs: Data Accelerator for AI and Analytics (DAAA). A proof of concept (PoC) is described in detail. This paper focuses on the functions that are provided by the Data Accelerator for AI and Analytics solution, which simplifies the daily work of data scientists and system administrators. This solution helps increase the efficiency of storage systems and data processing to obtain results faster while eliminating unnecessary data copies and associated data management.

Privileged Access Management for Secure Storage Administration: IBM Spectrum Scale with IBM Security Verify Privilege Vault

There is a growing insider security risk to organizations. Human error, privilege misuse, and cyberespionage are considered the top insider threats. One of the most dangerous internal security threats is the privileged user with access to critical data, which is the "crown jewels" of the organization. This data is on storage, so storage administration has critical privilege access that can cause major security breaches and jeopardize the safety of sensitive assets. Organizations must maintain tight control over whom they grant privileged identity status to for storage administration. Extra storage administration access must be shared with support and services teams when required. There also is a need to audit critical resource access that is required by compliance to standards and regulations. IBM® Security™ Verify Privilege Vault On-Premises (Verify Privilege Vault), formerly known as IBM Security™ Secret Server, is the next-generation privileged account management that integrates with IBM Storage to ensure that access to IBM Storage administration sessions is secure and monitored in real time with required recording for audit and compliance. Privilege access to storage administration sessions is centrally managed, and each session can be timebound with remote monitoring. You also can use remote termination and an approval workflow for the session. In this IBM Redpaper, we demonstrate the integration of IBM Spectrum® Scale and IBM Elastic Storage® Server (IBM ESS) with Verify Privilege Vault, and show how to use privileged access management (PAM) for secure storage administration. This paper is targeted at storage and security administrators, storage and security architects, and chief information security officers.