talk-data.com talk-data.com

Topic

data-engineering

3395

tagged

Activity Trend

1 peak/qtr
2020-Q1 2026-Q1

Activities

3395 activities · Newest first

Service Procedures for Linux on IBM Power Systems Servers

Collecting data on first occurance of the problem can id in problem determination and timely resolution of defects. At IBM®, this process of collecting data on first occurance if often referred to as First Failure Data Capture (FFDC). Gathering this data before reporting a defect helps to understand the problem more quickly and thoroughly, which saves time analyzing data and reduces the time and mission affects in fixing defects. Several diagnostic capabilities are built into the Linux operating system that enable you to determine the application level problems and system level problems. Collecting FFDC logs early, even before opening a defect report, helps to quickly determine whether: Symptoms match known problems (rediscovery) A report can be identified and resolved as a not-a-defect problem A workaround to reduce severity exists

SQL Server Data Automation Through Frameworks: Building Metadata-Driven Frameworks with T-SQL, SSIS, and Azure Data Factory

Learn to automate SQL Server operations using frameworks built from metadata-driven stored procedures and SQL Server Integration Services (SSIS). Bring all the power of Transact-SQL (T-SQL) and Microsoft .NET to bear on your repetitive data, data integration, and ETL processes. Do this for no added cost over what you’ve already spent on licensing SQL Server. The tools and methods from this book may be applied to on-premises and Azure SQL Server instances. The SSIS framework from this book works in Azure Data Factory (ADF) and provides DevOps personnel the ability to execute child packages outside a project—functionality not natively available in SSIS. Frameworks not only reduce the time required to deliver enterprise functionality, but can also accelerate troubleshooting and problem resolution. You'll learn in this book how frameworks also improve code quality by using metadata to drive processes. Much of the work performed by data professionals can be classified as “drudge work”—tasks that are repetitive and template-based. The frameworks-based approach shown in this book helps you to avoid that drudgery by turning repetitive tasks into "one and done" operations. Frameworks as described in this book also support enterprise DevOps with built-in logging functionality. What You Will Learn Create a stored procedure framework to automate SQL process execution Base your framework on a working system of stored procedures and execution logging Create an SSIS framework to reduce the complexity of executing multiple SSIS packages Deploy stored procedure and SSIS frameworks to Azure Data Factory environments in the cloud Who This Book Is For Database administrators and developers who are involved in enterprise data projects built around stored procedures and SQL Server Integration Services (SSIS). Readersshould have a background in programming along with a desire to optimize their data efforts by implementing repeatable processes that support enterprise DevOps.

EU GDPR – An international guide to compliance

This pocket guide will help you understand the Regulation, the broader principles of data protection, and what the GDPR means for businesses in Europe and beyond. Please visit https://www.itgovernancepublishing.co.uk/topic/uk-gdpr-supplemental-material to download your free Brexit supplement.

EU General Data Protection Regulation (GDPR) – An implementation and compliance guide, fourth edition

This bestselling guide is the ideal companion for anyone carrying out a GDPR (General Data Protection Regulation) compliance project. It provides comprehensive guidance and practical advice on complying with the Regulation. Visit https://www.itgovernancepublishing.co.uk/topic/uk-gdpr-supplemental-material to download your free Brexit supplement.

Security and Privacy Issues in IoT Devices and Sensor Networks

Security and Privacy Issues in IoT Devices and Sensor Networks investigates security breach issues in IoT and sensor networks, exploring various solutions. The book follows a two-fold approach, first focusing on the fundamentals and theory surrounding sensor networks and IoT security. It then explores practical solutions that can be implemented to develop security for these elements, providing case studies to enhance understanding. Machine learning techniques are covered, as well as other security paradigms, such as cloud security and cryptocurrency technologies. The book highlights how these techniques can be applied to identify attacks and vulnerabilities, preserve privacy, and enhance data security. This in-depth reference is ideal for industry professionals dealing with WSN and IoT systems who want to enhance the security of these systems. Additionally, researchers, material developers and technology specialists dealing with the multifarious aspects of data privacy and security enhancement will benefit from the book's comprehensive information. Provides insights into the latest research trends and theory in the field of sensor networks and IoT security Presents machine learning-based solutions for data security enhancement Discusses the challenges to implement various security techniques Informs on how analytics can be used in security and privacy

Oracle Database Transactions and Locking Revealed: Building High Performance Through Concurrency

Access much-needed information for building scalable, high-concurrency applications and deploying them against the Oracle Database. This new edition is updated to be current with Oracle Database 19. It includes a new chapter with troubleshooting recipes to help you quickly diagnose and resolve locking problems that are urgent and block production. Good transaction design is an important facet of highly-concurrent applications that are run by hundreds, even thousands, of users who are executing transactions at the same time. Transaction design, in turn, relies on a good understanding of how the database engine manages the locking of resources to prevent access conflicts and data loss that might otherwise result from concurrent access to data in the database. This book provides a solid and accurate explanation of how locking and concurrency are dealt with by Oracle Database. You will learn how the Oracle Database architecture accommodates user transactions, and how you can write code to mesh with the way in which Oracle Database is designed to operate. Oracle Database Transactions and Locking Revealed covers in detail the various lock types, and also different locking schemes such as pessimistic and optimistic locking. Then you will learn about transaction isolation and multi-version concurrency, and how the various lock types support Oracle Database’s transactional features. You will learn tips for transaction design, as well as some bad practices and habits to avoid. Coverage is also given to redo and undo, and their role in concurrency. The book is loaded with insightful code examples that drive home each concept. This is an important book that anyone developing highly-concurrent applications will want to have handy on their shelf. What You Will Learn Avoid application lockups due to conflicts over accessing the same resource Understand how Oracle prevents one application from overwriting another’s modifications Create transaction designs that mesh with how Oracle Database is designed Build high-throughput applications supporting thousands of concurrent users Design applications to take full advantage of Oracle’s powerful database engine Gain a fundamental knowledge of Oracle’s transaction and locking architecture Develop techniques to quickly diagnose and resolve common locking issues Who This Book Is For Oracle developers and database administrators faced with troubleshooting and solving deadlocks, locking contention, and similar problems that are encountered in high-concurrency environments; and application developers wanting to design their applications to excel at multi-user concurrency by taking full advantage of Oracle Database’s multi-versioning and concurrency support

Learn PostgreSQL

Dive into the world of PostgreSQL, one of the most powerful and versatile open-source relational databases! This book guides you through all the essentials of PostgreSQL version 12 and 13, from installation to high-performance database deployments. You'll learn how to design schemas, perform database operations efficiently, and implement advanced functionalities. What this Book will help me do Install, configure, and monitor a PostgreSQL server for optimal performance. Implement SQL and PL/pgSQL scripts to build complex database solutions. Analyze and optimize database schemas and indexes for efficiency. Secure a PostgreSQL database and manage roles and permissions effectively. Set up high-availability configurations through replication techniques. Author(s) None Ferrari and Enrico Pirozzi are seasoned database professionals with extensive experience in PostgreSQL. They bring practical expertise and a real-world perspective to the subject, ensuring you get hands-on knowledge and apply it effectively. Their approachable writing style simplifies even the most complex database concepts. Who is it for? This book is perfect for database professionals, developers, or tech enthusiasts looking to gain mastery over PostgreSQL. Whether you are new to PostgreSQL or have a fundamental understanding of databases, you'll find this book highly insightful in achieving your database management goals.

Data Lake Analytics on Microsoft Azure: A Practitioner's Guide to Big Data Engineering

Get a 360-degree view of how the journey of data analytics solutions has evolved from monolithic data stores and enterprise data warehouses to data lakes and modern data warehouses. You will This book includes comprehensive coverage of how: To architect data lake analytics solutions by choosing suitable technologies available on Microsoft Azure The advent of microservices applications covering ecommerce or modern solutions built on IoT and how real-time streaming data has completely disrupted this ecosystem These data analytics solutions have been transformed from solely understanding the trends from historical data to building predictions by infusing machine learning technologies into the solutions Data platform professionals who have been working on relational data stores, non-relational data stores, and big data technologies will find the content in this book useful. The book also can help you start your journey into the data engineer world as it provides an overview of advanced data analytics and touches on data science concepts and various artificial intelligence and machine learning technologies available on Microsoft Azure. What Will You Learn You will understand the: Concepts of data lake analytics, the modern data warehouse, and advanced data analytics Architecture patterns of the modern data warehouse and advanced data analytics solutions Phases—such as Data Ingestion, Store, Prep and Train, and Model and Serve—of data analytics solutions and technology choices available on Azure under each phase In-depth coverage of real-time and batch mode data analytics solutions architecture Various managed services available on Azure such as Synapse analytics, event hubs, Stream analytics, CosmosDB, and managed Hadoop services such as Databricks and HDInsight Who This Book Is For Data platform professionals, database architects, engineers, and solution architects

IBM Storage Solutions for SAS Analytics using IBM Spectrum Scale and IBM Elastic Storage System 3000 Version 1 Release 1

This IBM® Redpaper® publication is a blueprint for configuration, testing results, and tuning guidelines for running SAS workloads on Red Hat Enterprise Linux that use IBM Spectrum® Scale and IBM Elastic Storage® System (ESS) 3000. IBM lab validation was conducted with the Red Hat Linux nodes running with the SAS simulator scripts that are connected to the IBM Spectrum Scale and IBM ESS 3000. Simultaneous workloads are simulated across multiple x-86 nodes running with Red Hat Linux to determine scalability against the IBM Spectrum Scale clustered file system and ESS 3000 array. This paper outlines the architecture, configuration details, and performance tuning to maximize SAS application performance with the IBM Spectrum Scale 5.0.4.3 and IBM ESS 3000. This document is intended to facilitate the deployment and configuration of the SAS applications that use IBM Spectrum Scale and IBM Elastic Storage System (ESS) 3000. The information in this document is distributed on an "as is" basis without any warranty that is either expressed or implied. Support assistance for the use of this material is limited to situations where IBM Spectrum Scale or IBM ESS 3000 are supported and entitled and where the issues are specific to a blueprint implementation.

BigQuery for Data Warehousing: Managed Data Analysis in the Google Cloud

Create a data warehouse, complete with reporting and dashboards using Google’s BigQuery technology. This book takes you from the basic concepts of data warehousing through the design, build, load, and maintenance phases. You will build capabilities to capture data from the operational environment, and then mine and analyze that data for insight into making your business more successful. You will gain practical knowledge about how to use BigQuery to solve data challenges in your organization. BigQuery is a managed cloud platform from Google that provides enterprise data warehousing and reporting capabilities. Part I of this book shows you how to design and provision a data warehouse in the BigQuery platform. Part II teaches you how to load and stream your operational data into the warehouse to make it ready for analysis and reporting. Parts III and IV cover querying and maintaining, helping you keep your information relevant with other Google Cloud Platform services and advanced BigQuery. Part V takes reporting to the next level by showing you how to create dashboards to provide at-a-glance visual representations of your business situation. Part VI provides an introduction to data science with BigQuery, covering machine learning and Jupyter notebooks. What You Will Learn Design a data warehouse for your project or organization Load data from a variety of external and internal sources Integrate other Google Cloud Platform services for more complex workflows Maintain and scale your data warehouse as your organization grows Analyze, report, and create dashboards on the information in the warehouse Become familiar with machine learning techniques using BigQuery ML Who This Book Is For Developers who want to provide business users with fast, reliable, and insightful analysis from operational data, and data analysts interested in a cloud-based solution that avoids the pain of provisioning their own servers.

ETL with Azure Cookbook

ETL with Azure Cookbook is a comprehensive guide to building effective and scalable ETL solutions using the Azure cloud platform. Through hands-on recipes, this book explores the features and capabilities of Azure services for data integration and transformation, guiding you in creating efficient processes for moving and handling data. What this Book will help me do Master the basics and advanced techniques for building ETL processes on Azure. Learn practical skills in designing solutions that integrate multiple Azure services. Understand how to migrate existing on-premises ETL solutions to Azure successfully. Acquire knowledge of SQL Server and Azure Big Data Clusters for data integration. Gain experience in automating and optimizing data processes with BIML and Azure Databricks. Author(s) The authors of ETL with Azure Cookbook are experienced data engineers and Azure specialists with years of expertise in designing and implementing robust data solutions. Their professional journey includes hands-on work with SQL Server, Azure services, and scalable ETL frameworks. They aim to provide practical insights and actionable guidance to help readers achieve success in data engineering projects. Who is it for? This book is ideal for data architects, ETL developers, and IT professionals seeking to enhance their skills in data integration and transformation, particularly within the Azure ecosystem. It's suitable for individuals with some knowledge of data engineering principles, SQL, and familiarity with ETL processes who aim to adopt modern cloud-based approaches.

Empower Decision Makers with SAP Analytics Cloud: Modernize BI with SAP's Single Platform for Analytics

Discover the capabilities and features of SAP Analytics Cloud to draw actionable insights from a variety of data, as well as the functionality that enables you to meet typical business challenges. With this book, you will work with SAC and enable key decision makers within your enterprise to deliver crucial business decisions driven by data and key performance indicators. Along the way you’ll see how SAP has built a strong repertoire of analytics products and how SAC helps you analyze data to derive better business solutions. This book begins by covering the current trends in analytics and how SAP is re-shaping its solutions. Next, you will learn to analyze a typical business scenario and map expectations to the analytics solution including delivery via a single platform. Further, you will see how SAC as a solution meets each of the user expectations, starting with creation of a platform for sourcing data from multiple sources, enabling self-service for a spectrum of business roles, across time zones and devices. There’s a chapter on advanced capabilities of predictive analytics and custom analytical applications. Later there are chapters explaining the security aspects and their technical features before concluding with a chapter on SAP’s roadmap for SAC. Empower Decision Makers with SAP Analytics Cloud takes a unique approach of facilitating learning SAP Analytics Cloud by resolving the typical business challenges of an enterprise. These business expectations are mapped to specific features and capabilities of SAC, while covering its technical architecture block by block. What You Will Learn Work with the features and capabilities of SAP Analytics Cloud Analyze the requirements of a modern decision-support system Use the features of SAC that make it a single platform for decision support in a modern enterprise. See how SAC provides a secure and scalable platform hosted on the cloud Who This Book Is For Enterprise architects, SAP BI analytic solution architects, and developers.

Implementing IBM FlashSystem 9200, 9100, 7200, and 5100 Systems with IBM Spectrum Virtualize V8.3.1

Continuing its commitment to developing and delivering industry-leading storage technologies, IBM® introduces the IBM FlashSystem® solution that is powered by IBM Spectrum® Virtualize V8.3.1. This innovative storage offering delivers essential storage efficiency technologies and exceptional ease of use and performance, all integrated into a compact, modular design that is offered at a competitive, midrange price. The solution incorporates some of the top IBM technologies that are typically found only in enterprise-class storage systems, which raises the standard for storage efficiency in midrange disk systems. This cutting-edge storage system extends the comprehensive storage portfolio from IBM and can help change the way organizations address the ongoing information explosion. This IBM Redbooks® publication introduces the features and functions of an IBM Spectrum Virtualize V8.3.1 system through several examples. This book is aimed at pre-sales and post-sales technical support and marketing and storage administrators. It helps you understand the architecture, how to implement it, and how to take advantage of its industry-leading functions and features. Applicability: This edition applies to IBM Spectrum Virtualize V8.3.1 and the associated hardware and software that is detailed within. The screen captures included within this book might differ from the generally available (GA) version because parts of this book were written with pre-GA code. On 11 February 2020, IBM announced that it was simplifying its portfolio. This book was written by using previous models of the product line before the simplification; however, most of the general principles apply. If you are in any doubt as to their applicability, contact your local IBM representative. IBM Knowledge Center: In this book we provide links to Knowledge Center and a description of the relevant section that provides more information. Our starting point is the IBM FlashSystem 9200 family page, and the reader may have to select the product that applies to their environment.

Customer Data and Privacy: The Insights You Need from Harvard Business Review

Collect data and build trust. With the rise of data science and machine learning, companies are awash in customer data and powerful new ways to gain insight from that data. But in the absence of regulation and clear guidelines from most federal or state governments, it's difficult for companies to understand what qualifies as reasonable use and then determine how to act in the best interest of their customers. How do they build, not erode, trust? Customer Data and Privacy: The Insights You Need from Harvard Business Review brings you today's most essential thinking on customer data and privacy to help you understand the tangled interdependencies and complexities of this evolving issue. The lessons in this book will help you develop strategies that allow your company to be a good steward, collecting, using, and storing customer data responsibly. Business is changing. Will you adapt or be left behind? Get up to speed and deepen your understanding of the topics that are shaping your company's future with the Insights You Need from Harvard Business Review series. Featuring HBR's smartest thinking on fast-moving issues—blockchain, cybersecurity, AI, and more—each book provides the foundational introduction and practical case studies your organization needs to compete today and collects the best research, interviews, and analysis to get it ready for tomorrow. You can't afford to ignore how these issues will transform the landscape of business and society. The Insights You Need series will help you grasp these critical ideas—and prepare you and your company for the future.

Understanding Oracle APEX 20 Application Development: Think Like an Application Express Developer

This book shows developers and Oracle professionals how to build practical, non-trivial web applications using Oracle’s rapid application development environment – Application Express (APEX). This third edition Is revised to cover the new features and user interface experience found in APEX 20. Interactive grids and form regions are two of the newer aspects of APEX covered in this edition. The book is targeted at those who are new to APEX and just beginning to develop real projects for deployment, as well as those who are familiar with APEX and want a deeper understanding. The book takes you through the development of a demo web application that illustrates the concepts all APEX programmers should know. This book introduces the world of APEX properties, explaining the functionality supported by each page component as well as the techniques developers use to achieve that functionality. Topics include conditional formatting, user-customized reports, data entry forms, concurrency and lost updates, and security control. Specific attention is given in the book to the thought process involved in choosing and assembling APEX components and features to deliver a specific result. Understanding Oracle APEX 20 Application Development, 3rd Edition is the ideal book to take you from an understanding of the individual pieces of APEX to an understanding of how those pieces are assembled into polished applications. What You Will Learn Build attractive, highly functional web apps from the ground up Enhance and customize pages created by the APEX wizards Understand the security implications of page design Write PL/SQL code for process activity and verification Build complex components such as forms and interactive grids Who This Book Is For Developers new to APEXwho desire a strong fundamental understanding of how APEX applications work. For existing developers and database administrators desiring to mine the most value from APEX by improving their development techniques.

Learn MongoDB 4.x

Explore the capabilities of MongoDB 4.x with this comprehensive guide designed for developers and administrators working with NoSQL databases. Dive into topics such as database design, advanced query handling, and security configuration, and gain hands-on experience through practical examples and insights. What this Book will help me do Learn to configure and install MongoDB 4.x for development and administration. Understand the principles of NoSQL schema design for optimal performance. Perform complex queries and operations to manage your MongoDB databases. Secure your MongoDB setup with role-based access control and encryption techniques. Monitor and optimize database performance for production environments. Author(s) None Bierer, the author of 'Learn MongoDB 4.x,' is a seasoned database expert with extensive experience in NoSQL technologies. With a focus on practicality and clear explanations, None brings deep insights into MongoDB's development and administration. Who is it for? This book is ideal for early-career developers, system administrators, and database enthusiasts eager to break into NoSQL technologies. If you are familiar with Python and basic database concepts, this book will guide you through mastering MongoDB. It's perfect for those building dynamic backend systems.

SQL Server 2019 Administrator's Guide - Second Edition

SQL Server 2019 Administrator's Guide provides a complete walkthrough of administering, managing, and optimizing SQL Server 2019. You'll gain the expertise needed to implement secure and efficient database solutions suitable for enterprise-scale environments. This book systematically explores the tools, techniques, and best practices essential to mastering SQL Server 2019. What this Book will help me do Optimize database queries and design using indexing techniques to resolve performance issues effectively. Implement robust backup and recovery mechanisms following advanced security policies. Utilize SQL Server 2019 tools for automation in monitoring, maintaining, and managing health checks. Integrate SQL Server with Azure for Big Data processing and scalability. Set up highly available and stable Always-On environments for enterprise databases. Author(s) Marek Chmel and Vladimír Mužný are seasoned database administrators with years of hands-on experience in SQL Server and database infrastructure. Their collaborative writing approach emphasizes real-world scenarios and examples that make technical concepts accessible. With accolades in professional database education and a passion for teaching, they provide a guiding hand through complex database subjects. Who is it for? This book is ideal for database administrators, developers, and IT professionals who seek to enhance their expertise with SQL Server 2019. Readers should have a basic understanding of database principles and familiarity with prior versions of SQL Server. Whether you're stepping into advanced administration or seeking to fine-tune your enterprise database infrastructure, this book is tailored for you.

MongoDB Topology Design: Scalability, Security, and Compliance on a Global Scale

Create a world-class MongoDB cluster that is scalable, reliable, and secure. Comply with mission-critical regulatory regimes such as the European Union’s General Data Protection Regulation (GDPR). Whether you are thinking of migrating to MongoDB or need to meet legal requirements for an existing self-managed cluster, this book has you covered. It begins with the basics of replication and sharding, and quickly scales up to cover everything you need to know to control your data and keep it safe from unexpected data loss or downtime. This book covers best practices for stable MongoDB deployments. For example, a well-designed MongoDB cluster should have no single point of failure. The book covers common use cases when only one or two data centers are available. It goes into detail about creating geopolitical sharding configurations to cover the most stringent data protection regulation compliance. The book also covers different tools and approaches for automating and monitoring a cluster with Kubernetes, Docker, and popular cloud provider containers. What You Will Learn Get started with the basics of MongoDB clusters Protect and monitor a MongoDB deployment Deepen your expertise around replication and sharding Keep effective backups and plan ahead for disaster recovery Recognize and avoid problems that can occur in distributed databases Build optimal MongoDB deployments within hardware and data center limitations Who This Book Is For Solutions architects, DevOps architects and engineers, automation and cloud engineers, and database administrators who are new to MongoDB and distributed databases or who need to scale up simple deployments. This book is a complete guide to planning a deployment for optimal resilience, performance, and scaling, and covers all the details required to meet the new set of data protection regulations such as the GDPR. This book is particularly relevant for large global organizations such as financial and medical institutions, as well as government departments that need to control data in the whole stack and are prohibited from using managed cloud services.

Privacy Optimization Meets Pandemic Tracking

Can smartphone apps help track the spread of the novel coronavirus, privately and securely? In this report, Rob Pegoraro weighs the issue of whether mobile apps can help trace and then slow the spread of COVID-19 or will end up as just another episode of botched government procurement and application of technology. Apple and Google have recently devised a system to track COVID-19 infections anonymously using Bluetooth with iOS and Android smartphones. This development points a spotlight on a needed debate about balancing privacy and collecting useful data. Do privacy-optimizing techniques, such as federated learning and differential privacy, offer useful alternatives to building centralized databases that may later invite abuse? This report takes a close look at this subject and then provides recommendations for software developers, public health authorities, and elected officials who want to build on the Apple-Google API. Understand the scope of the problem, including how contact tracing can help slow and stop outbreaks Take a closer look at Apple and Google’s proposed remedy Learn how other countries including Singapore, India, France, and Australia have traced the spread of COVID-19 Examine the risk factors for adopting and using a decentralized system like the Apple-Google app