This book is your practical and comprehensive guide to learning Google Cloud Platform (GCP) for data science, using only the free tier services offered by the platform. Data science and machine learning are increasingly becoming critical to businesses of all sizes, and the cloud provides a powerful platform for these applications. GCP offers a range of data science services that can be used to store, process, and analyze large datasets, and train and deploy machine learning models. The book is organized into seven chapters covering various topics such as GCP account setup, Google Colaboratory, Big Data and Machine Learning, Data Visualization and Business Intelligence, Data Processing and Transformation, Data Analytics and Storage, and Advanced Topics. Each chapter provides step-by-step instructions and examples illustrating how to use GCP services for data science and big data projects. Readers will learn how to set up a Google Colaboratory account and run Jupyternotebooks, access GCP services and data from Colaboratory, use BigQuery for data analytics, and deploy machine learning models using Vertex AI. The book also covers how to visualize data using Looker Data Studio, run data processing pipelines using Google Cloud Dataflow and Dataprep, and store data using Google Cloud Storage and SQL. What You Will Learn Set up a GCP account and project Explore BigQuery and its use cases, including machine learning Understand Google Cloud AI Platform and its capabilities Use Vertex AI for training and deploying machine learning models Explore Google Cloud Dataproc and its use cases for big data processing Create and share data visualizations and reports with Looker Data Studio Explore Google Cloud Dataflow and its use cases for batch and stream data processing Run data processing pipelines on Cloud Dataflow Explore Google Cloud Storageand its use cases for data storage Get an introduction to Google Cloud SQL and its use cases for relational databases Get an introduction to Google Cloud Pub/Sub and its use cases for real-time data streaming Who This Book Is For Data scientists, machine learning engineers, and analysts who want to learn how to use Google Cloud Platform (GCP) for their data science and big data projects

Gain deep insight into real-time analytics, including the features of these systems and the problems they solve. With this practical book, data engineers at organizations that use event-processing systems such as Kafka, Google Pub/Sub, and AWS Kinesis will learn how to analyze data streams in real time. The faster you derive insights, the quicker you can spot changes in your business and act accordingly. Author Mark Needham from StarTree provides an overview of the real-time analytics space and an understanding of what goes into building real-time applications. The book's second part offers a series of hands-on tutorials that show you how to combine multiple software products to build real-time analytics applications for an imaginary pizza delivery service. You will: Learn common architectures for real-time analytics Discover how event processing differs from real-time analytics Ingest event data from Apache Kafka into Apache Pinot Combine event streams with OLTP data using Debezium and Kafka Streams Write real-time queries against event data stored in Apache Pinot Build a real-time dashboard and order tracking app Learn how Uber, Stripe, and Just Eat use real-time analytics

In 'Data Engineering with Google Cloud Platform', you'll explore how to construct efficient, scalable data pipelines using GCP services. This hands-on guide covers everything from building data warehouses to deploying machine learning pipelines, helping you master GCP's ecosystem. What this Book will help me do Build comprehensive data ingestion and transformation pipelines using BigQuery, Cloud Storage, and Dataflow. Design end-to-end orchestration flows with Airflow and Cloud Composer for automated data processing. Leverage Pub/Sub for building real-time event-driven systems and streaming architectures. Gain skills to design and manage secure data systems with IAM and governance strategies. Prepare for and pass the Professional Data Engineer certification exam to elevate your career. Author(s) Adi Wijaya is a seasoned data engineer with significant experience in Google Cloud Platform products and services. His expertise in building data systems has equipped him with insights into the real-world challenges data engineers face. Adi aims to demystify technical topics and deliver practical knowledge through his writing, helping tech professionals excel. Who is it for? This book is tailored for data engineers and data analysts who want to leverage GCP for building efficient and scalable data systems. Readers should have a beginner-level understanding of topics like data science, Python, and Linux to fully benefit from the material. It is also suitable for individuals preparing for the Google Professional Data Engineer exam. The book is a practical companion for enhancing cloud and data engineering skills.

Learn how easy it is to apply sophisticated statistical and machine learning methods to real-world problems when you build using Google Cloud Platform (GCP). This hands-on guide shows data engineers and data scientists how to implement an end-to-end data pipeline with cloud native tools on GCP. Throughout this updated second edition, you'll work through a sample business decision by employing a variety of data science approaches. Follow along by building a data pipeline in your own project on GCP, and discover how to solve data science problems in a transformative and more collaborative way. You'll learn how to: Employ best practices in building highly scalable data and ML pipelines on Google Cloud Automate and schedule data ingest using Cloud Run Create and populate a dashboard in Data Studio Build a real-time analytics pipeline using Pub/Sub, Dataflow, and BigQuery Conduct interactive data exploration with BigQuery Create a Bayesian model with Spark on Cloud Dataproc Forecast time series and do anomaly detection with BigQuery ML Aggregate within time windows with Dataflow Train explainable machine learning models with Vertex AI Operationalize ML with Vertex AI Pipelines

Working with unbounded and fast-moving data streams has historically been difficult. But with Kafka Streams and ksqlDB, building stream processing applications is easy and fun. This practical guide shows data engineers how to use these tools to build highly scalable stream processing applications for moving, enriching, and transforming large amounts of data in real time. Mitch Seymour, data services engineer at Mailchimp, explains important stream processing concepts against a backdrop of several interesting business problems. You'll learn the strengths of both Kafka Streams and ksqlDB to help you choose the best tool for each unique stream processing project. Non-Java developers will find the ksqlDB path to be an especially gentle introduction to stream processing. Learn the basics of Kafka and the pub/sub communication pattern Build stateless and stateful stream processing applications using Kafka Streams and ksqlDB Perform advanced stateful operations, including windowed joins and aggregations Understand how stateful processing works under the hood Learn about ksqlDB's data integration features, powered by Kafka Connect Work with different types of collections in ksqlDB and perform push and pull queries Deploy your Kafka Streams and ksqlDB applications to production

For nearly a decade, Apache Kafka has been the go-to publish-subscribe (pub-sub) messaging system—and for good reason. It offers functionality for a wide range of enterprise use cases, along with a large ecosystem of tools and a dedicated community. But lately, upstart Apache Pulsar has been gaining ground. This detailed report explains why. Apache Pulsar takes the best parts of Kafka and expands on them to solve problems that were out of scope of Kafka’s original design. Author Chris Bartholomew shows you how Kafka and Pulsar compare and where they differ. Engineers and other technical decision makers will learn the advantages that make Pulsar a compelling alternative to Kafka. Explore the architecture and major components of Kafka and Pulsar Discover the benefits of Pulsar’s subscription model for messaging Understand how Pulsar simplifies the messaging system for organizations that need high performance pub-sub messaging, delivery guarantees, and traditional messaging patterns Learn how Pulsar’s separation of serving and storing makes it natural to run in cloud native environments like Kubernetes See how Kafka and Pulsar perform on the OpenMessage Project benchmark

The power of IBM® MQ is its flexibility combined with reliability, scalability, and security. This flexibility provides a large number of design and implementation choices. Making informed decisions from this range of choices can simplify the development of applications and the administration of an MQ messaging infrastructure. Applications that access such an infrastructure can be developed using a wide range of programming paradigms and languages. These applications can run within a substantial array of software and hardware environments. Customers can use IBM MQ to integrate and extend the capabilities of existing and varied infrastructures in the information technology (IT) system of a business. IBM MQ V8.0 was released in June 2014. Before that release, the product name was IBM WebSphere® MQ. This IBM Redbooks® publication covers the core enhancements made in IBM MQ V8 and the concepts that must be understood. A broad understanding of the product features is key to making informed design and implementation choices for both the infrastructure and the applications that access it. Details of new areas of function for IBM MQ are introduced throughout this book, such as the changes to security, publish/subscribe clusters, and IBM System z exploitation. This book is for individuals and organizations who make informed decisions about design and applications before implementing an IBM MQ infrastructure or begin development of an IBM MQ application.

IBM® MessageSight is an appliance-based messaging server that is optimized to address the massive scale requirements of machine-to-machine (m2m) and mobile user scenarios. IBM MessageSight makes it easy to connect mobile customers to your existing messaging enterprise system, enabling a substantial number of remote clients to be concurrently connected. The MQTT protocol is a lightweight messaging protocol that uses publish/subscribe architecture to deliver messages over low bandwidth or unreliable networks. A publish/subscribe architecture works well for HTML5, native, and hybrid mobile applications by removing the wait time of a request/response model. This creates a better, richer user experience. The MQTT protocol is simple, which results in a client library with a low footprint. MQTT was proposed as an Organization for the Advancement of Structured Information Standards (OASIS) standard. This book provides information about version 3.1 of the MQTT specification. This IBM Redbooks® publication provides information about how IBM MessageSight, in combination with MQTT, facilitates the expansion of enterprise systems to include mobile devices and m2m communications. This book also outlines how to connect IBM MessageSight to an existing infrastructure, either through the use of IBM WebSphere® MQ connectivity or the IBM Integration Bus (formerly known as WebSphere Message Broker). This book describes IBM MessageSight product features and facilities that are relevant to technical personnel, such as system architects, to help them make informed design decisions regarding the integration of the messaging appliance into their enterprise architecture. Using a scenario-based approach, you learn how to develop a mobile application, and how to integrate IBM MessageSight with other IBM products. This publication is intended to be of use to a wide-ranging audience.

Redis in Action introduces Redis and walks you through examples that demonstrate how to use it effectively. You'll begin by getting Redis set up properly and then exploring the key-value model. Then, you'll dive into real use cases including simple caching, distributed ad targeting, and more. You'll learn how to scale Redis from small jobs to massive datasets. Experienced developers will appreciate chapters on clustering and internal scripting to make Redis easier to use. About the Technology When you need near-real-time access to a fast-moving data stream, key-value stores like Redis are the way to go. Redis expands on the key-value pattern by accepting a wide variety of data types, including hashes, strings, lists, and other structures. It provides lightning-fast operations on in-memory datasets, and also makes it easy to persist to disk on the fly. Plus, it's free and open source. About the Book What's Inside Redis from the ground up Preprocessing real-time data Managing in-memory datasets Pub/sub and configuration Persisting to disk About the Reader Written for developers familiar with database concepts. No prior exposure to Redis or other NoSQL databases required. Appropriate for systems administrators comfortable with programming. About the Author Dr. Josiah L. Carlson is a seasoned database professional and an active contributor to the Redis community. Quotes A great addition to the Redis ecosystem. - From the Foreword by Salvatore Sanfilippo, Creator of Redis The examples, taken from real-world use cases, are one of the major strengths of the book. - Filippo Pacini, SG Consulting From beginner to expert with real and comprehensive examples. - Felipe Gutierrez, VMware/Spring Source Excellent in-depth analysis ... insightful real-world examples. - Bobby Abraham, Integri LLC Pure gold! - Leo Cassarani, Unboxed Consulting

Dive into ØMQ (aka ZeroMQ), the smart socket library that gives you fast, easy, message-based concurrency for your applications. With this quick-paced guide, you’ll learn hands-on how to use this scalable, lightweight, and highly flexible networking tool for exchanging messages among clusters, the cloud, and other multi-system environments. ØMQ maintainer Pieter Hintjens takes you on a tour of real-world applications, using extended examples in C to help you work with ØMQ’s API, sockets, and patterns. Learn how to use specific ØMQ programming techniques, build multithreaded applications, and create your own messaging architectures. You’ll discover how ØMQ works with several programming languages and most operating systems—with little or no cost. Learn ØMQ’s main patterns: request-reply, publish-subscribe, and pipeline Work with ØMQ sockets and patterns by building several small applications Explore advanced uses of ØMQ’s request-reply pattern through working examples Build reliable request-reply patterns that keep working when code or hardware fails Extend ØMQ’s core pub-sub patterns for performance, reliability, state distribution, and monitoring Learn techniques for building a distributed architecture with ØMQ Discover what’s required to build a general-purpose framework for distributed applications

RabbitMQ in Action is a fast-paced run through building and managing scalable applications using the RabbitMQ messaging server. It starts by explaining how message queuing works, its history, and how RabbitMQ fits in. Then it shows you real-world examples you can apply to your own scalability and interoperability challenges. About the Technology There's a virtual switchboard at the core of most large applications where messages race between servers, programs, and services. RabbitMQ is an efficient and easy-to-deploy queue that handles this message traffic effortlessly in all situations, from web startups to massive enterprise systems. About the Book RabbitMQ in Action teaches you to build and manage scalable applications in multiple languages using the RabbitMQ messaging server. It's a snap to get started. You'll learn how message queuing works and how RabbitMQ fits in. Then, you'll explore practical scalability and interoperability issues through many examples. By the end, you'll know how to make Rabbit run like a well-oiled machine in a 24 x 7 x 365 environment. What's Inside Learn fundamental messaging design patterns Use patterns for on-demand scalability Glue a PHP frontend to a backend written in anything Implement a PubSub-alerting service in 30 minutes flat Configure RabbitMQ's built-in clustering Monitor, manage, extend, and tune RabbitMQ About the Reader Written for developers familiar with Python, PHP, Java, .NET, or any other modern programming language. No RabbitMQ experience required. About the Authors Alvaro Videla is a developer and architect specializing in MQ-based applications. Jason J. W. Williams is CTO of DigiTar, a messaging service provider, where he directs design and development. Quotes In this outstanding work, two experts share their years of experience running large-scale RabbitMQ systems. - Alexis Richardson, VMware Well-written, thoughtful, and easy to follow. - Karsten Strøbæk, Microsoft Soup to nuts on RabbitMQ; a wide variety of in-depth examples. - Patrick Lemiuex, Voxel Internap This book will take you to a messaging wonderland. - David Dossot, Coauthor of Mule in Action

Publish and Subscribe is an effective way of disseminating information to multiple users. Publish/Subscribe applications can help to enormously simplify the task of getting business messages and transactions to a wide, dynamic and potentially large audience in a timely manner. This IBM Redbooks publication positions the MQSeries Publish/Subscribe to MQSeries Integrator Publish/ Subscribe. It will help you create, tailor and configure an application from publishing data through to subscribing via web pages. The books provides a broad understanding of a building and running an entire publish/subscribe solution. It will help give you a quick start to design and create a solution and then migrate it from MQSeries Publish/Subscribe to MQSeries Integrator Publish/Subscribe.