In this paper, we outline some IBM® Spectrum Virtualize HyperSwap® SAN implementation and design best practices for optimum resiliency of the SAN Volume Controller cluster. It provides IBM Spectrum® Virtualize HyperSwap and Stretched Cluster configuration details. Note: In this book, for brevity, we use HyperSwap to refer to both HyperSwap and Stretched Cluster. The documentation there details the minimum requirements. However, it does not describe the design of the storage area network (SAN) in detail, nor does it describe the recommended way to implement those requirements on a SAN. In this IBM Redpaper publication, we outline some of the best practices for SAN design and implementation that leads to optimum resiliency of the SAN Volume Controller (SVC) cluster, and we explain why each recommendation is made. This paper is SAN vendor-neutral wherever possible. Any mention of a specific SAN switch vendor, or terms used by a specific switch vendor, is made only where relevant to a specific context, and does not imply an endorsement of a specific switch vendor. Note: Some of the figures in this document might not depict redundant fabrics or storage configurations. This was done for simplicity, and it should be assumed that any recommendations made for fabric design assume that there are two redundant fabrics.

Send us a text Want to be featured as a guest on Making Data Simple? Reach out to us at [[email protected]] and tell us why you should be next.  Abstract This week on Making Data Simple, we are joined by Wennie Allen, Director of Data Science Elite Business, and Brittany Boggle, Senior Data Scientist at IBM. Together, they provide an update on the new initiatives the DSE team are embarking on during the COVID-19 pandemic. This includes employing data optimization and AI decision-making procedures to assist ICU facilities and estimate ventilator demand. Tune-in to find out more.  Connect with Wennie LinkedIn Twitter Connect with Brittany LinkedIn Twitter Show Notes 05:52 - Click here to learn why you should consider taking up a new hobby while physical distancing.  17:52 - Discover more about the Data Science Elite team here. 33:49 - Get up to speed with agile methodology here. 39:02 - Check out the Data and AI portfolio here.  Connect with the Team Producer Liam Seston - LinkedIn. Producer Lana Cosic - LinkedIn. Producer Meighann Helene - LinkedIn.  Producer Kate Brown - LinkedIn. Producer Allison Proctor - LinkedIn. Producer Mark Simmonds - LinkedIn.  Producer Michael Sestak - LinkedIn. Host Al Martin - LinkedIn and Twitter. Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

This IBM® Redpaper publication describes IBM Spectrum® LSF® Suite best practices installation topics, application checks for workload management, and high availability configurations by using theoretical knowledge and hands-on exercises. These findings are documented by way of sample scenarios. This publication addresses topics for sellers, IT architects, IT specialists, and anyone who wants to implement and manage a high-performing workload management solution with LSF. Moreover, this guide provides documentation to transfer how-to-skills to the technical teams, and solution guidance to the sales team. This publication compliments documentation that is available at IBM Knowledge Center, and aligns with educational materials that are provided by IBM Systems.

IBM® experts recognize the need for data protection, both from hardware or software failures, and from physical relocation of hardware, theft, and retasking of existing hardware. The IBM DS8000® supports encryption-capable hard disk drives (HDDs) and flash drives. These Full Disk Encryption (FDE) drive sets are used with key management services that are provided by IBM Security Key Lifecycle Manager software or Gemalto SafeNet KeySecure to allow encryption for data at rest. Use of encryption technology involves several considerations that are critical for you to understand to maintain the security and accessibility of encrypted data. Failure to follow the requirements that are described in the IBM Redpaper can result in an encryption deadlock. Starting with Release 8.5 code, the DS8000 also supports Transparent Cloud Tiering (TCT) data object encryption. With TCT encryption, data is encrypted before it is transmitted to the cloud. The data remains encrypted in cloud storage and is decrypted after it is transmitted back to the IBM DS8000. Starting with DS8000 Release 9.0, the DS8900F provides Fibre Channel Endpoint Security when communicating with an IBM z15™, which supports link authentication and the encryption of data that is in-flight. For more information, see IBM Fibre Channel Endpoint Security for IBM DS8900F and IBM Z, SG24-8455. This edition focuses on IBM Security Key Lifecycle Manager Version 3.0.1.3 or later, which enables support Key Management Interoperability Protocol (KMIP) with the DS8000 Release 9.0 code or later and updated DS GUI for encryption functions.

This IBM® Redpaper publication delivers SAP HANA architectural concepts for successful implementation on IBM Power Systems servers. This publication addresses topics for sellers, IT architects, IT specialists, and anyone who wants to understand how to take advantage of running SAP HANA workloads on Power Systems servers. Moreover, this guide provides documentation to transfer how-to skills to the technical teams, and it provides solution guidance to the sales team. This publication complements documentation that is available at IBM Knowledge Center, and it aligns with educational materials that are provided by IBM Systems.

This IBM Redpaper™ publication introduces the IBM Spectrum Scale immutability function. It shows how to set it up and presents different ways for managing immutable and append-only files. This publication also provides guidance for implementing IT security aspects in an IBM Spectrum Scale cluster by addressing regulatory requirements. It also describes two typical use cases for managing immutable files. One use case involves applications that manage file immutability; the other use case presents a solution to automatically set files to immutable within a IBM Spectrum Scale immutable fileset.

Companies need to migrate data not only when technology needs to be replaced, but also for consolidation, load balancing, and disaster recovery (DR). Data migration is a critical operation, and this book explains the phases and steps to ensure a smooth migration. Topics range from planning and preparation to execution and validation. The book explains, from a generic standpoint, the appliance-based, storage-based, and host-based techniques that can be used to accomplish the migration. Each method is explained through practical migration scenarios and for various operating systems. This publication addresses the aspects of data migration efforts while focusing on fixed block storage systems in open environment with the IBM® FlashSystem 9100 as the target system. Therefore, the book also emphasizes various migration techniques using the Spectrum Virtualize built-in functions. This document targets storage administrators, storage network administrators, system designers, architects, and IT professionals who design, administer or plan data migrations in large data Centers. The aim is to ensure that you are aware of the current thinking, methods, and products that IBM can make available to you. These items are provided to ensure a data migration process that is as efficient and problem-free as possible. The material presented in this book was developed with versions of the referenced products as of February, 2020.

IBM® Spectrum Protect Plus is a data protection solution that provides near-instant recovery, replication, retention, and reuse for virtual machines, databases, and applications in hybrid multicloud environments. IBM Knowledge Center for IBM Spectrum® Protect Plus provides extensive documentation for installation, deployment, and usage. In addition, IBM Spectrum Protect Plus Blueprint (https://ibm.biz/IBMSpectrumProtectPlusBlueprints) provides guidance about how to build and size an IBM Spectrum Protect Plus solution. The goal of this IBM Redpaper publication is to summarize and complement the available information by providing useful hints and tips based on the authors' practical experience in installing and supporting IBM Spectrum Protect Plus in actual customer environments. Over time, our aim is to compile a set of best practices that cover all aspects of the product, from planning and installation to tuning, maintenance, and troubleshooting.

Send us a text Want to be featured as a guest on Making Data Simple? Reach out to us at [[email protected]] and tell us why you should be next.  Abstract Currently, COVID-19 is disrupting the world. In an effort to better provide updated information on the status of the pandemic, IBM and The Weather Channel have created a COVID-19 dashboard. Bill Higgins, IBM Distinguished Engineer, and Daniel Benoit, Program Director of Information Governance have come on the podcast this week to discuss this new initiative. Together, with host Al Martin, they discuss the purpose of this project, their current findings, and how they personally have been impacted.  Check out the dashboard here.  Connect with Bill LinkedIn Connect with Daniel LinkedIn Show Notes 10:16 - Get up to speed on Natural Language Processing here. 14:47 - Not sure what a data lake is? Find out here.   21:49 - Learn more on why extensibility is important to your API's here.  Connect with the Team Producer Liam Seston - LinkedIn. Producer Lana Cosic - LinkedIn. Producer Meighann Helene - LinkedIn.  Producer Kate Brown - LinkedIn. Producer Allison Proctor - LinkedIn. Producer Mar Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

This IBM® Redbooks® publication introduces the latest member of the IBM Z® platform, the IBM z15™. It includes information about the Z environment and how it helps integrate data and transactions more securely. It also provides insight for faster and more accurate business decisions. The z15 is a state-of-the-art data and transaction system that delivers advanced capabilities, which are vital to any digital transformation. The z15 is designed for enhanced modularity, and occupies an industry-standard footprint. It is offered as a single air-cooled 19-inch frame called the z15 T02, or as a multi-frame (1 to 4 19-inch frames) called the z15 T01. Both z15 models excel at the following tasks: Using hybrid multicloud integration services Securing and protecting data with encryption everywhere Providing resilience with key to zero downtime Transforming a transactional platform into a data powerhouse Getting more out of the platform with IT Operational Analytics Accelerating digital transformation with agile service delivery Revolutionizing business processes Blending open source and IBM Z technologies This book explains how this system uses innovations and traditional Z strengths to satisfy growing demand for cloud, analytics, and open source technologies. With the z15 as the base, applications can run in a trusted, reliable, and secure environment that improves operations and lessens business risk.

To help you safeguard your sensitive data and provide ease of auditability and control, IBM introduced a new capability for IBM Z® called IBM Data Privacy Passports. It can help minimize the risk and impact of data loss and privacy breaches when collecting and storing sensitive data. Data Privacy Passports can manage how data is shared securely through a central control of user access. Data Privacy Passports can protect data wherever it goes. Security policies are kept and honored whenever the data is accessed. Future data access may be revoked remotely via Data Privacy Passports, long after data leaves the system of record, and sensitive data may even be made unusable simply by destroying its encryption key. Data Privacy Passports is designed to help reduce the time that is spent by staff to protect data and ensure privacy throughout its lifecycle via a central point of control. This IBM Redguide presents a business view of Data Privacy Passports, including how data privacy and protection concerns are addressed. We also explore how value is gained through various business model examples.

IBM® Spectrum Scale is a proven, scalable, high-performance data and file management solution. It provides world-class storage management with extreme scalability, flash accelerated performance, automatic policy-based storage that has tiers of flash through disk to tape. It also provides support for various protocols, such as NFS, SMB, Object, HDFS, and iSCSI. Containers can leverage the performance, information lifecycle management (ILM), scalability, and multisite data management to give the full flexibility on storage as they experience on the runtime. Container adoption is increasing in all industries, and they sprawl across multiple nodes on a cluster. The effective management of containers is necessary because their number will probably reach a far greater number than virtual machines today. Kubernetes is the standard container management platform currently being used. Data management is of ultimate importance, and often is forgotten because the first workloads containerized are ephemeral. For data management, many drivers with different specifications were available. A specification named Container Storage Interface (CSI) was created and is now adopted by all major Container Orchestrator Systems available. Although other container orchestration systems exist, Kubernetes became the standard framework for container management. It is a very flexible open source platform used as the base for most cloud providers and software companies' container orchestration systems. Red Hat OpenShift is one of the most reliable enterprise-grade container orchestration systems based on Kubernetes, designed and optimized to easily deploy web applications and services. OpenShift enables developers to focus on the code, while the platform takes care of all of the complex IT operations and processes. This IBM Redbooks® publication describes how the CSI Driver for IBM file storage enables IBM Spectrum® Scale to be used as persistent storage for stateful applications running in Kubernetes clusters. Through the Container Storage Interface Driver for IBM file storage, Kubernetes persistent volumes (PVs) can be provisioned from IBM Spectrum Scale. Therefore, the containers can be used with stateful microservices, such as database applications (MongoDB, PostgreSQL, and so on).

Send us a text NOTE: This episode was recorded before the COVID-19 outbreak. Any comments made in this episode on travel are no longer relevant or took place during ordered quarantines. Please stay home and be safe. Want to be featured as a guest on Making Data Simple? Reach out to us at [[email protected]] and tell us why you should be next.  Abstract Our guest this week is Alyse Daghelian, Global Vice President Cloud Expert Services at IBM. Alyse has gone through many role transitions, from starting in engineering to being the lead for global sales at IBM. She provides some insightful advice that can help you take the next step towards your career goals. Tune-in to learn more.   Connect with Alyse LinkedIn Twitter Show Notes 01:26 - You should always be looking for a new job. Click here for a Forbes article that explains why. 10:57 - Check out this article on medium that explains the impact of dehydration on the brain.  27:49 - Learn more about the fundamental life skills that poker can tech you here.  Connect with the Team Producer Liam Seston - LinkedIn. Producer Lana Cosic - LinkedIn. Producer Meighann Helene - LinkedIn.  Producer Kate Brown - LinkedIn. Producer Allison Proctor - LinkedIn. Producer Mark Simmonds - LinkedIn.  Producer Michael Sestak - LinkedIn. Host Al Martin - LinkedIn and Twitter. Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

IBM Storage for Red Hat OpenShift is a comprehensive container-ready solution that includes all the hardware & software components necessary to setup and/or expand your Red Hat OpenShift environment. This blueprint includes Red Hat OpenShift Container Platform and uses Container Storage Interface (CSI) standards. IBM Storage brings enterprise data services to containers. In this blueprint, learn how to: · Combine the benefits of IBM Systems with the performance of IBM Storage solutions so that you can deliver the right services to your clients today! · Build a 24 by 7 by 365 enterprise class private cloud with Red Hat OpenShift Container Platform utilizing new open source Container Storage interface (CSI) drivers · Leverage enterprise class services such as NVMe based flash performance, high data availability, and advanced container security IBM Storage for Red Hat OpenShift Container Platform is designed for your DevOps environment for on-premises deployment with easy-to-consume components built to perform and scale for your enterprise. Simplify your journey to cloud with pre-tested and validated blueprints engineered to enable rapid deployment and peace of mind as you move to a hybrid multicloud environment. You now have the capabilities.

Send us a text NOTE: This episode was recorded before the COVID-19 outbreak. Any comments made in this episode on travel are no longer relevant or took place during ordered quarantines. Please stay home and be safe. Want to be featured as a guest on Making Data Simple? Reach out to us at [[email protected]] and tell us why you should be next.  Abstract Back for part 2 of his Making Data Simple series is Jad Chehlawi, CEO and Founder of TelosTouch. This episode follows up to part 1, where Jad explains how TelosTouch is aiming to redefine the client experience of  investing. This episode focuses more on the emotional challenges that frequently alter decisions and how to better account for those changes.  Connect with Jad LinkedIn TelosTouch Show Notes 03:47 - Check out these 6 hacks for fighting financial procrastination.  10:06 - The missing link. Learn about the void TelosTouch seeks to fill here. 17:37 - Need to get up to speed on natural language processing (NLP)? Find out more here.  25:49 - Learn more about modern portfolio theory here. Connect with the Team Producer Liam Seston - LinkedIn. Producer Lana Cosic - LinkedIn. Producer Meighann Helene - LinkedIn.  Producer Kate Brown - LinkedIn. Producer Allison Proctor - LinkedIn. Producer Mark Simmonds - LinkedIn.  Producer Michael Sestak - LinkedIn. Host Al Martin - LinkedIn and Twitter. Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

This IBM® Redpaper Redbooks publication provides information and concepts about how to take advantage of SAP HANA and IBM Power Systems features to manage data and performance efficiently. The target audience of this book includes architects, IT specialists, and systems administrators who deploy SAP HANA and manage data and SAP system performance.

Send us a text NOTE: This episode was recorded before COVID-19 mandates. Comments made in this episode on travel took place before any quarantines were in effect and are no longer applicable. Please stay home and be safe. Want to be featured as a guest on Making Data Simple? Reach out to us at [[email protected]] and tell us why you should be next.  Abstract Our guest this week is Jad Chehlawi, Founder and CEO of TelosTouch. Jad has extensive experience in both the technology and finance industries. He aims to redefine the client experience by enabling real-time engagement across multiple sectors. Host Al Martin asks the tough questions regarding the state of financial advice and provides an opinionated response to current trends. Get insight on your investments and discover new methods of making the right financial decisions. Tune-in to find out.  Connect with Jad LinkedIn TelosTouch Show Notes 05:00 - Check out this article by Investopedia which analyzes the communication gap with clients.  14:17 - Get an understanding of wealth management with this Forbes article. 28:35 - Take a look at this Forbes article which discusses the future for financial advisors.   Connect with the Team Producer Liam Seston - LinkedIn. Producer Lana Cosic - LinkedIn. Producer Meighann Helene - LinkedIn.  Producer Mark Simmonds - LinkedIn.  Host Al Martin - LinkedIn and Twitter. Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

This IBM® Redpaper publication provides SAP HANA platform migration information and details for successful planning for migration to IBM Power Systems servers. This publication addresses topics for sellers, IT architects, IT specialists, and anyone who wants to migrate and manage SAP workloads on Power Systems servers. Moreover, this guide provides documentation to transfer how-to skills to the technical teams, and it provides solution guidance to the sales team. This publication complements documentation that is available at IBM Knowledge Center, and it aligns with educational materials that are provided by IBM Systems.

Send us a text Want to be featured as a guest on Making Data Simple? Reach out to us at [[email protected]] and tell us why you should be next. Abstract This week on Making Data Simple, our guest is Emmanuel Letouzé, PhD, Director and Co-Founder of Data-Pop Alliance. Emmanuel comes from an extensive academic background, specializing in subjects such as data science, political science, economics and demography. He speaks about key takeaways he's had from his research, applying the knowledge to new projects and issues, including the recent pandemic of COVID-19.  Connect with Emmanuel LinkedIn Datapopalliance.org Twitter Art Manu Cartoons Twitter Show Notes 03:37 - Get some answers to your questions on social distancing in this NYTimes article here. 16:19 - Learn more on demography here. 20:31 - AI can now make predictions on demography on only knowing your name. Find out more here. Connect with the Team Producer Liam Seston - LinkedIn. Producer Lana Cosic - LinkedIn. Producer Meighann Helene - LinkedIn.  Producer Mark Simmonds - LinkedIn.  Host Al Martin - LinkedIn and Twitter. Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

Summary CouchDB is a distributed document database built for scale and ease of operation. With a built-in synchronization protocol and a HTTP interface it has become popular as a backend for web and mobile applications. Created 15 years ago, it has accrued some technical debt which is being addressed with a refactored architecture based on FoundationDB. In this episode Adam Kocoloski shares the history of the project, how it works under the hood, and how the new design will improve the project for our new era of computation. This was an interesting conversation about the challenges of maintaining a large and mission critical project and the work being done to evolve it.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With 200Gbit private networking, scalable shared block storage, a 40Gbit public network, fast object storage, and a brand new managed Kubernetes platform, you’ve got everything you need to run a fast, reliable, and bullet-proof data platform. And for your machine learning workloads, they’ve got dedicated CPU and GPU instances. Go to dataengineeringpodcast.com/linode today to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show! Are you spending too much time maintaining your data pipeline? Snowplow empowers your business with a real-time event data pipeline running in your own cloud account without the hassle of maintenance. Snowplow takes care of everything from installing your pipeline in a couple of hours to upgrading and autoscaling so you can focus on your exciting data projects. Your team will get the most complete, accurate and ready-to-use behavioral web and mobile data, delivered into your data warehouse, data lake and real-time streams. Go to dataengineeringpodcast.com/snowplow today to find out why more than 600,000 websites run Snowplow. Set up a demo and mention you’re a listener for a special offer! Setting up and managing a data warehouse for your business analytics is a huge task. Integrating real-time data makes it even more challenging, but the insights you obtain can make or break your business growth. You deserve a data warehouse engine that outperforms the demands of your customers and simplifies your operations at a fraction of the time and cost that you might expect. You deserve ClickHouse, the open-source analytical database that deploys and scales wherever and whenever you want it to and turns data into actionable insights. And Altinity, the leading software and service provider for ClickHouse, is on a mission to help data engineers and DevOps managers tame their operational analytics. Go to dataengineeringpodcast.com/altinity for a free consultation to find out how they can help you today. You listen to this show to learn and stay up to date with what’s happening in databases, streaming platforms, big data, and everything else you need to know about modern data management. For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Corinium Global Intelligence, ODSC, and Data Council. Upcoming events include the Software Architecture Conference in NYC, Strata Data in San Jose, and PyCon US in Pittsburgh. Go to dataengineeringpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today. Your host is Tobias Macey and today I’m interviewing Adam Kocoloski about CouchDB and the work being done to migrate the storage layer to FoundationDB

Interview

Introduction How did you get involved in the area of data management? Can you starty by describing what CouchDB is?

How did you get involved in the CouchDB project and what is your current role in the community?

What are the use cases that it is well suited for? Can you share some of the history of CouchDB and its role in the NoSQL movement? How is CouchDB currently architected and how has it evolved since it was first introduced? What have been the benefits and challenges of Erlang as the runtime for CouchDB? How is the current storage engine implemented and what are its shortcomings? What problems are you trying to solve by replatforming on a new storage layer?

What were the selection criteria for the new storage engine and how did you structure the decision making process? What was the motivation for choosing FoundationDB as opposed to other options such as rocksDB, levelDB, etc.?

How is the adoption of FoundationDB going to impact the overall architecture and implementation of CouchDB? How will the use of FoundationDB impact the way that the current capabilities are implemented, such as data replication? What will the migration path be for people running an existing installation? What are some of the biggest challenges that you are facing in rearchitecting the codebase? What new capabilities will the FoundationDB storage layer enable? What are some of the most interesting/unexpected/innovative ways that you have seen CouchDB used?

What new capabilities or use cases do you anticipate once this migration is complete?

What are some of the most interesting/unexpected/challenging lessons that you have learned while working with the CouchDB project and community? What is in store for the future of CouchDB?

Contact Info

LinkedIn @kocolosk on Twitter kocolosk on GitHub

Parting Question

From your perspective, what is the biggest gap in the tooling or technology for data management today?

Links

Apache CouchDB FoundationDB

Podcast Episode

IBM Cloudant Experimental Particle Physics FPGA == Field Programmable Gate Array Apache Software Foundation CRDT == Conflict-free Replicated Data Type

Podcast Episode

Erlang Riak RabbitMQ Heisenbug Kubernetes Property Based Testing

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

Support Data Engineering Podcast