The road to AI adoption is far more complex than one can imagine. Building data science models and testing them is only one piece of the puzzle. To understand the roadblocks and best practices, Wayne Eckerson invited Nir Kaldero in our latest episode to learn why organizations need to start paying more attention to people, culture and processes to make data science projects a success and how democratization skills pays off in the long run.

Nir Kaldero is the Head of Data Science, Vice President at Galvanize Inc. and the creator of the GalvanizeU Master’s of Science in Data Science program. A tireless advocate for transforming education and reshaping the field of data science, his vision and mission is to make an impact on a wide variety of communities through education, science, and technology. In addition to his work at some of the world’s largest international corporations, Kaldero serves as a Google expert/mentor and has been named an IBM Analytics Champion 2017 & 2018, a prestigious honor given to leaders in the field of science, technology, engineering, and math (STEM).

This IBM® Redpaper™ publication reviews the architecture and operations of the IBM DS8000® Global Mirror function. The document looks at different aspects of the solution in terms of performance, infrastructure requirements, data integrity, business continuity, and impact on production. Hints and tips are provided on how to best configure the overall Global Mirror environment, in terms of connectivity, storage configuration, and specific parameters tuning. The guidelines that are provided are in general related to performance, which ultimately ensures a better recovery point objective (RPO). Therefore, we encourage you to follow those guidelines.

Having the appropriate storage for hosting business critical data and the proper analytic software for deep inspection of that data is becoming necessary to get deeper insights into the data so that users can categorize which data qualifies for compliance. This IBM® Redpaper™ publication explains why the storage features of IBM Spectrum™ Scale, when combined with the data analysis and categorization features of IBM StoredIQ®, provide an excellent platform for hosting unstructured business data that is subject to regulatory compliance guidelines, such as General Data Protection Regulation (GDPR). In this paper, we describe how IBM StoredIQ can be used to identify files that are stored in an IBM Spectrum Scale™ file system that include personal information, such as phone numbers. These files can be secured in another file system partition by encrypting those files by using IBM Spectrum Scale functions. Encrypting files prevents unauthorized access to those files because only users that can access the encryption key can decrypt those files. This paper is intended for chief technology officers, solution, and security architects and systems administrators.

Send us a text Seth Dobrin is back to kick off season 3 and reflect on data and tech in 2018. Seth Dobrin, vice president and Chief Data Officer of IBM Analytics, gives insight to leading the data science elite team, and he details the steps and strategies required to be successful in the field. Host Al Martin and Seth also make some data science predictions for 2019, letting you know what you should be looking out for in the year ahead.

Shownotes:  00:00 - Check us out on YouTube and SoundCloud.  00:10 - Connect with Producer Steve Moore on LinkedIn and Twitter.  00:15 - Connect with Producer Liam Seston on LinkedIn and Twitter.  00:20 - Connect with Producer Rachit Sharma on LinkedIn.  00:25 - Connect with Host Al Martin on LinkedIn and Twitter.  00:55 – Connect with Seth Dobrin on LinkedIn and Twitter.  02:00 – Seth Dobrin’s first podcast from January 2018.  03:30 - What is data science?  04:25 - Seth Dobrin’s Blog: Don’t let data science become a scam.  10:55 - IBM Data Science Elite Team: Kickstart, build andaccelerate   31:55 - What is AI? 37:58 - What are data pipelines?  41:55 - What is Blockchain? Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

This IBM® Redbooks® publication describes the connectivity options that are available for use within and beyond the data center for the IBM Z family of mainframes, which includes these systems: IBM z14® IBM z14 Model ZR1 IBM z13® IBM z13s™ IBM zEnterprise® EC12 (zEC12) IBM zEnterprise BC12 (zBC12) This book highlights the hardware and software components, functions, typical uses, coexistence, and relative merits of these connectivity features. It helps readers understand the connectivity alternatives that are available when planning and designing their data center infrastructures. The changes to this edition are based on the IBM Z hardware announcement dated April 10, 2018. This book is intended for data center planners, IT professionals, systems engineers, and network planners who are involved in the planning of connectivity solutions for IBM mainframes.

Send us a text Host Al Martin looks back on his top 5 favourite clips from episodes published in 2018. These conversations range from explaining the importance of data visualization, to discussing the differences between A.I. and deep learning. Thanks to all of our listeners for an incredible 2018, and prepare yourself for Season 3 of the Making Data Simple podcast!

Show Notes

00:00 - Check us out on YouTube and SoundCloud!  00:10 - Connect with producer Liam Seston on LinkedIn and Twitter.   00:15 - Connect with producer Steve Moore on LinkedIn and Twitter.  00:24 - Connect with host Al Martin on LinkedIn and Twitter.   00:55 - Listen to the full conversation with Lisa Seacat DeLuca here. 05:45 - Listen to the full conversation with John Thomas here. 09:59 - Listen to the full conversation with Jillian Lellis here.   13:39 - Listen to the full conversation with Adam Storm here. 18:36 - Listen to the full conversation with Jean Francois Puget here. Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

This IBM Redbooks® Product Guide gives an overview of the features and functions that are available with the IBM DS8880 models running microcode Release 8.51 (DS8000 License Machine Code 8.8.51.xx.xx). The IBM DS8880 architecture relies on powerful IBM POWER8® processor-based servers that manage the cache to streamline disk input/output (I/O), maximizing performance and throughput. These capabilities are further enhanced with the availability of the second generation of high-performance flash enclosures (HPFE Gen-2). The IBM DS8888, DS8886, and DS8884 models excel at supporting the IBM Z Enterprise server and IBM Power server environments, offering many synergy features.

Send us a text Happy holidays from the Making Data Simple team! Enjoy a rebroadcast of a conversation with Seth Dobrin, Vice President and Chief Data Officer for IBM Analytics, as he and Al explore the strategies and people your company needs to disrupt and succeed in the year ahead. Do you or your team members need new credentials to work in data? Seth also discusses what you need in your toolkit to be a data scientist at IBM.

Show Notes 00.30 Connect with Al Martin on Twitter and LinkedIn. 01.00 Connect with Seth Dobrin on Twitter and LinkedIn. 01.40 Read "What IBM looks for in a Data Scientist" by Seth Dobrin and Jean-Francois Puget. 06.00 Learn more about GDPR.  13.00 Learn more about master data management. 13.05 Learn more about unified governance and integration.  13.25 Learn more about machine learning.  14.00 Connect and learn more about Ginni Rometty.  14.40 Learn more about cognitive computing. 19.35 Connect with Rob Thomas on Twitter and LinkedIn. 21.00 Connect with Jean-Francois Puget on Twitter and LinkedIn. Follow @IBMAnalytics Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

In this episode, Daniel Graham dissects the capabilities of data lakes and compares it to data warehouses. He talks about the primary use cases of data lakes and how they are vital for big data ecosystems. He then goes on to explain the role of data warehouses which are still responsible for timely and accurate data but don't have a central role anymore. In the end, both Wayne Eckerson and Dan Graham settle on a common definition for modern data architectures.

Daniel Graham has more than 30 years in IT, consulting, research, and product marketing, with almost 30 years at leading database management companies. Dan was a Strategy Director in IBM’s Global BI Solutions division and General Manager of Teradata’s high-end server divisions. During his tenure as a product marketer, Dan has been responsible for MPP data management systems, data warehouses, and data lakes, and most recently, the Internet of Things and streaming systems.

Send us a text Jason Tatge, CEO, president and cofounder of Farmobile, joins the show to discuss data in the agriculture industry. The conversation touches on Jason's experience launching a startup, tips for finding success, and the value of big data from a farmer's perspective. This episode gives insight to data science for one of the oldest and most important sectors in our society.   

Show Notes

00:00 - Check us out on YouTube and SoundCloud. 00:10 - Connect with producer Liam Seston on LinkedIn and Twitter. 00:15 - Connect with producer Steve Moore on LinkedIn and Twitter. 00:24 - Connect with host Al Martin on LinkedIn and Twitter. 01:20 - Connect with guest Jason Tatge on LinkedIn and Twitter. 04:24 - Get some insights to commodity trading. 10:09 - Check out Farmobile.com. 14:21 - Here are some more reasons why data collection in farming is so important. 22:21 - How data collection in farming is driving greater efficiency. 27:33 - Learn about pipeline entrepreneurs here. Follow @IBMAnalytics Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

Abstract This IBM® Redbooks® publication presents a general introduction to the latest IBM tape and tape library technologies. Featured tape technologies include the IBM LTO Ultrium and Enterprise 3592 tape drives, and their implementation in IBM tape libraries. This 16th edition introduces the new TS1160 tape drive with up to 20 TB capacity on JE media and the latest updates to the IBM TS4500 and TS4300 tape libraries, It includes generalized sections about Small Computer System Interface (SCSI) and Fibre Channel connections, and multipath architecture configurations. This book also covers tools and techniques for library management. It is intended for anyone who wants to understand more about IBM tape products and their implementation. It is suitable for IBM clients, IBM Business Partners, IBM specialist sales representatives, and technical specialists. If you do not have a background in computer tape storage products, you might need to read other sources of information. In the interest of being concise, topics that are generally understood are not covered in detail.

Send us a text Paul Zikopolous, VP of big data cognitive systems at IBM, joins us to discuss tactics for both career and personal growth. Paul is also an established author and public speaker, and leverages experiences gained through those pursuits in the advice he gives. Have a pen and paper ready as there is a lot to take away from this enlightening conversation. Show notes 00:00 - Check us out on YouTube. 00:00 - We are now on Soundcloud. 00:10 - Add producer Liam Seston on LinkedIn and Twitter.  00:15 - Add producer Steve Moore on LinkedIn and Twitter. 00:25 - Add host Al Martin on LinkedIn and Twitter.  01:43 - Connect with Paul Zikopolous on LinkedIn and Twitter.  07:02 - Get up to speed with Watson Studio. 10:16 - Develop a continuous learning lifestyle. 14:27 - How to figure out what you want out of a job. 20:55 - How to succeed with failure.  24:50 - "Get comfortable feeling uncomfortable."  30:54 - Here are some tips to make time for the gym. 38:28 - "Don't let other people define you."  Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

This IBM® Redpaper™ publication given an overview and technical introduction to IBM Power Systems™ RAID solutions. The book is organized to start with an introduction to Redundant Array of Independent Disks (RAID), and various RAID levels with their benefits. A brief comparison of Direct Attached Storage (DAS) and networked storage systems such as SAN / NAS is provided with a focus on emerging applications that typically use the DAS model over networked storage models. The book focuses on IBM Power Systems I/O architecture and various SAS RAID adapters that are supported in IBM POWER8™ processor-based systems. A detailed description of the SAS adapters, along with their feature comparison tables, is included in Chapter 3, "RAID adapters for IBM Power Systems" on page 45. The book is aimed at readers who have the responsibility of configuring IBM Power Systems for individual solution requirements. This audience includes IT Architects, IBM Technical Sales Teams, IBM Business Partner Solution Architects and Technical Sales teams, and systems administrators who need to understand the SAS RAID hardware and RAID software solutions supported in POWER8 processor-based systems.

Summary

Apache Spark is a popular and widely used tool for a variety of data oriented projects. With the large array of capabilities, and the complexity of the underlying system, it can be difficult to understand how to get started using it. Jean George Perrin has been so impressed by the versatility of Spark that he is writing a book for data engineers to hit the ground running. In this episode he helps to make sense of what Spark is, how it works, and the various ways that you can use it. He also discusses what you need to know to get it deployed and keep it running in a production environment and how it fits into the overall data ecosystem.

Preamble

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out Linode. With 200Gbit private networking, scalable shared block storage, and a 40Gbit public network, you’ve got everything you need to run a fast, reliable, and bullet-proof data platform. If you need global distribution, they’ve got that covered too with world-wide datacenters including new ones in Toronto and Mumbai. Go to dataengineeringpodcast.com/linode today to get a $20 credit and launch a new server in under a minute. Go to dataengineeringpodcast.com to subscribe to the show, sign up for the mailing list, read the show notes, and get in touch. Join the community in the new Zulip chat workspace at dataengineeringpodcast.com/chat Your host is Tobias Macey and today I’m interviewing Jean Georges Perrin, author of the upcoming Manning book Spark In Action 2nd Edition, about the ways that Spark is used and how it fits into the data landscape

Interview

Introduction How did you get involved in the area of data management? Can you start by explaining what Spark is?

What are some of the main use cases for Spark? What are some of the problems that Spark is uniquely suited to address? Who uses Spark?

What are the tools offered to Spark users? How does it compare to some of the other streaming frameworks such as Flink, Kafka, or Storm? For someone building on top of Spark what are the main software design paradigms?

How does the design of an application change as you go from a local development environment to a production cluster?

Once your application is written, what is involved in deploying it to a production environment? What are some of the most useful strategies that you have seen for improving the efficiency and performance of a processing pipeline? What are some of the edge cases and architectural considerations that engineers should be considering as they begin to scale their deployments? What are some of the common ways that Spark is deployed, in terms of the cluster topology and the supporting technologies? What are the limitations of the Spark programming model?

What are the cases where Spark is the wrong choice?

What was your motivation for writing a book about Spark?

Who is the target audience?

What have been some of the most interesting or useful lessons that you have learned in the process of writing a book about Spark? What advice do you have for anyone who is considering or currently using Spark?

Contact Info

@jgperrin on Twitter Blog

Parting Question

From your perspective, what is the biggest gap in the tooling or technology for data management today?

Book Discount

Use the code poddataeng18 to get 40% off of all of Manning’s products at manning.com

Links

Apache Spark Spark In Action Book code examples in GitHub Informix International Informix Users Group MySQL Microsoft SQL Server ETL (Extract, Transform, Load) Spark SQL and Spark In Action‘s chapter 11 Spark ML and Spark In Action‘s chapter 18 Spark Streaming (structured) and Spark In Action‘s chapter 10 Spark GraphX Hadoop Jupyter

Podcast Interview

Zeppelin Databricks IBM Watson Studio Kafka Flink

P

Abstract The IBM® TS4500 (TS4500) tape library is a next-generation tape solution that offers higher storage density and integrated management than previous solutions. This IBM Redbooks® publication gives you a close-up view of the new IBM TS4500 tape library. In the TS4500, IBM delivers the density that today’s and tomorrow’s data growth requires. It has the cost-effectiveness and the manageability to grow with business data needs, while you preserve existing investments in IBM tape library products. Now, you can achieve both a low cost per terabyte (TB) and a high TB density per square foot because the TS4500 can store up to 11 petabytes (PB) of uncompressed data in a single frame library or scale up to 2 PB per square foot to over 350 PB. The TS4500 offers the following benefits: High availability: Dual active accessors with integrated service bays reduce inactive service space by 40%. The Elastic Capacity option can be used to completely eliminate inactive service space. Flexibility to grow: The TS4500 library can grow from the right side and the left side of the first L frame because models can be placed in any active position. Increased capacity: The TS4500 can grow from a single L frame up to another 17 expansion frames with a capacity of over 23,000 cartridges. High-density (HD) generation 1 frames from the TS3500 library can be redeployed in a TS4500. Capacity on demand (CoD): CoD is supported through entry-level, intermediate, and base-capacity configurations. Advanced Library Management System (ALMS): ALMS supports dynamic storage management, which enables users to create and change logical libraries and configure any drive for any logical library. Support for IBM TS1160 while also supporting TS1155, TS1150, and TS1140 tape drive: The TS1160 gives organizations an easy way to deliver fast access to data, improve security, and provide long-term retention, all at a lower cost than disk solutions. The TS1160 offers high-performance, flexible data storage with support for data encryption. Also, this enhanced fifth-generation drive can help protect investments in tape automation by offering compatibility with existing automation. The new TS1160 Tape Drive Model 60E delivers a dual 10 Gb or 25 Gb Ethernet host attachment interface that is optimized for cloud-based and hyperscale environments. The TS1160 Tape Drive Model 60F delivers a native data rate of 400 MBps, the same load/ready, locate speeds, and access times as the TS1155, and includes dual-port 16 Gb Fibre Channel support. Support of the IBM Linear Tape-Open (LTO) Ultrium 8 tape drive: The LTO Ultrium 8 offering represents significant improvements in capacity, performance, and reliability over the previous generation, LTO Ultrium 7, while still protecting your investment in the previous technology. Support of LTO 8 Type M cartridge (M8): The LTO Program is introducing a new capability with LTO-8 drives. The ability of the LTO-8 drive to write 9 TB on a brand new LTO-7 cartridge instead of 6 TB as specified by the LTO-7 format. Such a cartridge is called an LTO-7 initialized LTO-8 Type M cartridge. Integrated TS7700 back-end Fibre Channel (FC) switches are available. Up to four library-managed encryption (LME) key paths per logical library are available. This book describes the TS4500 components, feature codes, specifications, supported tape drives, encryption, new integrated management console (IMC), and command-line interface (CLI). You learn how to accomplish the following specific tasks: Improve storage density with increased expansion frame capacity up to 2.4 times and support 33% more tape drives per frame. Manage storage by using the ALMS feature. Improve business continuity and disaster recovery with dual active accessor, automatic control path failover, and data path failover. Help ensure security and regulatory compliance with tape-drive encryption and Write Once Read Many (WORM) media. Support IBM LTO Ultrium 8, 7, 6, and 5, IBM TS1160, TS1155, TS1150, and TS1140 tape drives. Provide a flexible upgrade path for users who want to expand their tape storage as their needs grow. Reduce the storage footprint and simplify cabling with 10 U of rack space on top of the library. This guide is for anyone who wants to understand more about the IBM TS4500 tape library. It is particularly suitable for IBM clients, IBM Business Partners, IBM specialist sales representatives, and technical specialists.

This IBM® Redpaper™ presents and positions the DS8882F. The DS8882F adds a modular rack-mountable enterprise storage system to the DS8880 family of all-flash enterprise storage systems. The modular system can be integrated into 16U contiguous space of an existing IBM z14™ Model ZR1 (z14 Model ZR1), IBM LinuxONE™ Rockhopper II (z14 Model LR1), or other standard 19-inch wide rack. The DS8882F allows you to take advantage of the performance boost of DS8880 all-flash enterprise systems and advanced features while limiting datacenter footprint and power infrastructure requirements.

Send us a text In the latest episode of "Making Data Simple," host Al Martin invites Jeff Jonas, CEO, founder and chief scientist at Senzing Inc. to discuss use cases of AI and big data. The discussion ranges from Jeff's personal achievements, his miraculous quadriplegic recovery, his completion of every global Ironman triathlon race, and the birth of his company Senzing Inc. Suit up for what is truly an engaging conversation.  Show notes 00:00 - Checkout our YouTube channel.  00:10 - Connect with producer Liam Seston on LinkedIn and Twitter. 00:15 - Connect with producer Steve Moore on LinkedIn and Twitter. 00:24 - Connect with host Al Martin on LinkedIn and Twitter. 01:28 - Connect with guest Jeff Jonas on LinkedIn and Twitter. 02:08 - Not sure what the difference between a triathlon and an Ironman triathlon is? 02:28 - Here's how NORA and other security software applications are being employed in Las Vegas. 13:22 - Here's an interesting article about parent/child naming conventions. 16:26 - Check out Jeff's keynote at IBM Think 2018. 18:55 - Check out these 6 other brands with the "try then buy" sales method. 23:30 - Try out Senzing for yourself at senzing.com. 27:41 - Get an inside look at what it's like to live in a hotel, full-time. 31:49 - Need to brush up on Context Computing? Jeff Jonas explains it here. 33:12 - Check out these 10 Ironman triathlon facts.   Want to be featured as a guest on Making Data Simple? Reach out to us at [email protected] and tell us why you should be next. The Making Data Simple Podcast is hosted by Al Martin, WW VP Technical Sales, IBM, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun.

This IBM® Redpaper™ publication describes the IBM DS8880 High-Performance Enclosure (HPFE) Gen2 architecture and configuration, as of DS8880 Release 8.51. The DS8880 HPFE Gen2 is a 2U Redundant Array of Independent Disks (RAID) flash enclosure with associated Flash RAID adapters that can be used exclusively with DS8880 models. The flash enclosure and Flash RAID adapters are installed in pairs. Each storage enclosure pair can support 16, 32, or 48 encryption-capable flash drives (2.5-inch, 63.5 mm form factor).

This IBM® Redbooks® Product Guide describes the IBM Storage Networking SAN768C-6. IBM Storage Networking SAN768C-6 has the industry's highest port density for a storage area network (SAN) director and features 768 line-rate 32 gigabits per second (Gbps) or 16 Gbps Fibre Channel ports. Designed to support multiprotocol workloads, IBM Storage Networking SAN768C-6 enables SAN consolidation and collapsed-core solutions for large enterprises, which reduces the number of managed switches and leads to easy-to-manage deployments. IBM Storage Networking SAN768C-6 supports the 48-Port 32 Gbps Fibre Channel Switching Module, the 48-Port 16 Gbps Fibre Channel Switching Module, the 48-port 10 Gbps FCoE Switching Module, the 24-port 40 Gbps FCoE switching module, and the 24/10-port SAN Extension Module. By reducing the number of front-panel ports that are used on inter-switch links (ISLs), it also offers room for future growth. IBM Storage Networking SAN768C-6 addresses the mounting storage requirements of today's large virtualized data centers. As a director-class SAN switch, IBM Storage Networking SAN768C-6 uses the same operating system and management interface as other IBM data center switches. It brings intelligent capabilities to a high-performance, protocol-independent switch fabric, and delivers uncompromising availability, security, scalability, simplified management, and the flexibility to integrate new technologies. You can use IBM Storage Networking SAN768C-6 to transparently deploy unified fabrics with Fibre Channel and Fibre Channel over Ethernet (FCoE) connectivity to achieve low total cost of ownership (TCO). For mission-critical enterprise storage networks that require secure, robust, cost-effective business-continuance services, the FCIP extension module is designed to deliver outstanding SAN extension performance, reducing latency for disk and tape operations with FCIP acceleration features, including FCIP write acceleration and FCIP tape write and read acceleration.

Abstract Continuing its commitment to developing and delivering industry-leading storage technologies, IBM® introduces Data Reduction Pools (DRP) and Deduplication powered by IBM Spectrum™ Virtualize, which are innovative storage features that deliver essential storage efficiency technologies and exceptional ease of use and performance, all integrated into a proven design. This book discusses Data Reduction Pools (DRP) and Deduplication and is intended for experienced storage administrators who are fully familiar with IBM Spectrum Virtualize, SAN Volume Controller, and the Storwize family of products.