Geospatial data, or data with location information, is generated in huge volumes every day by billions of mobile phones, IoT sensors, drones, nanosatellites, and many other sources in an unending stream. This practical ebook introduces you to the landscape of tools and methods for making sense of all that data, and shows you how to apply geospatial analytics to a variety of issues, large and small. Authors Aurelia Moser, Jon Bruner, and Bill Day provide a complete picture of the geospatial analysis options available, including low-scale commercial desktop GIS tools, medium-scale options such as PostGIS and Lucene-based searching, and true big data solutions built on technologies such as Hadoop. You’ll learn when it makes sense to move from one type of solution to the next, taking increased costs and complexity into account. Explore the structure of basic webmaps, and the challenges and constraints involved when working with geo data Dive into low- to medium-scale mapping tools for use in backend and frontend web development Focus on tools for robust medium-scale geospatial projects that don’t quite justify a big data solution Learn about innovative platforms and software packages for solving issues of processing and storage of large-scale data Examine geodata analysis use cases, including disaster relief, urban planning, and agriculture and environmental monitoring

Why have stream-oriented data systems become so popular, when batch-oriented systems have served big data needs for many years? In this report, author Dean Wampler examines the rise of streaming systems for handling time-sensitive problems—such as detecting fraudulent financial activity as it happens. You’ll explore the characteristics of fast data architectures, along with several open source tools for implementing them. Batch-mode processing isn’t going away, but exclusive use of these systems is now a competitive disadvantage. You’ll learn that, while fast data architectures are much harder to build, they represent the state of the art for dealing with mountains of data that require immediate attention. Learn step-by-step how a basic fast data architecture works Understand why event logs are the core abstraction for streaming architectures, while message queues are the core integration tool Use methods for analyzing infinite data sets, where you don’t have all the data and never will Take a tour of open source streaming engines, and discover which ones work best for different use cases Get recommendations for making real-world streaming systems responsive, resilient, elastic, and message driven Explore an example streaming application for the IoT: telemetry ingestion and anomaly detection for home automation systems

Learn the right cutting-edge skills and knowledge to leverage Spark Streaming to implement a wide array of real-time, streaming applications. This book walks you through end-to-end real-time application development using real-world applications, data, and code. Taking an application-first approach, each chapter introduces use cases from a specific industry and uses publicly available datasets from that domain to unravel the intricacies of production-grade design and implementation. The domains covered in Pro Spark Streaming include social media, the sharing economy, finance, online advertising, telecommunication, and IoT. In the last few years, Spark has become synonymous with big data processing. DStreams enhance the underlying Spark processing engine to support streaming analysis with a novel micro-batch processing model. Pro Spark Streaming by Zubair Nabi will enable you to become a specialist of latency sensitive applications by leveraging the key features of DStreams, micro-batch processing, and functional programming. To this end, the book includes ready-to-deploy examples and actual code. Pro Spark Streaming will act as the bible of Spark Streaming. What You'll Learn Discover Spark Streaming application development and best practices Work with the low-level details of discretized streams Optimize production-grade deployments of Spark Streaming via configuration recipes and instrumentation using Graphite, collectd, and Nagios Ingest data from disparate sources including MQTT, Flume, Kafka, Twitter, and a custom HTTP receiver Integrate and couple with HBase, Cassandra, and Redis Take advantage of design patterns for side-effects and maintaining state across the Spark Streaming micro-batch model Implement real-time and scalable ETL using data frames, SparkSQL, Hive, and SparkR Use streaming machine learning, predictive analytics, and recommendations Mesh batch processing with stream processing via the Lambda architecture Who This Book Is For Data scientists, big data experts, BI analysts, and data architects.

Using log data to create value isn’t new to mechanized industries. But in today’s data-driven environment—particularly with the rise of the Internet of Things—this type of data exhaust can be converted from inactive, latent assets to critical-path components of an overall production ecosystem. In this report, Cameron Turner provides three real-world case studies in which his company, The Data Guild, served as a product co-development consultancy. You’ll learn how an energy efficiency firm, a tech company, and a healthcare organization combined their historical logs with newly generated sensor data from the IoT. By leveraging machine learning to proactively identify efficiency and opportunity through prediction and recommendation, each company was able to deploy an ROI-generating solution and gain a significant business advantage. This report also provides advice for successfully implementing IoT data, as well as key factors to consider when performing data analysis.

This book introduces the problems facing Internet of Things developers and explores current technologies and techniques to help you manage, mine, and make sense of the data being collected through the use of the world’s most popular database on the Internet - MySQL. The IoT is poised to change how we interact with and perceive the world around us, and the possibilities are nearly boundless. As more and more connected devices generate data, we will need to solve the problem of how to collect, store, and make sense of IoT data by leveraging the power of database systems. The book begins with an introduction of the MySQL database system and storage of sensor data. Detailed instructions and examples are provided to show how to add database nodes to IoT solutions including how to leverage MySQL high availability, including examples of how to protect data from node outages using advanced features of MySQL. The book closes with a comparison of raw and transformed data showing how transformed data can improve understandability and help you cut through a clutter of superfluous data toward the goal of mining nuggets of useful knowledge.

The need for fast data applications is growing rapidly, driven by the IoT, the surge in machine-to-machine (M2M) data, global mobile device proliferation, and the monetization of SaaS platforms. So how do you combine real-time, streaming analytics with real-time decisions in an architecture that’s reliable, scalable, and simple? In this O’Reilly report, Ryan Betts and John Hugg from VoltDB examine ways to develop apps for fast data, using pre-defined patterns. These patterns are general enough to suit both the do-it-yourself, hybrid batch/streaming approach, as well as the simpler, proven in-memory approach available with certain fast database offerings. Their goal is to create a collection of fast data app development recipes. We welcome your contributions, which will be tested and included in future editions of this report.

Message Queuing Telemetry Transport (MQTT) is a messaging protocol designed for the Internet of Things (IoT). It is lightweight enough to be supported by the smallest devices, yet robust enough to ensure that important messages get to their destinations every time. With MQTT devices, such as energy meters, cars, trains, mobile phones and tablets, and personal health care devices, devices can communicate with each other and with other systems or applications. IBM® MessageSight is a messaging appliance designed to handle the scale and security of a robust IoT solution. MessageSight allows you to easily secure connections, configure policies for messaging, and scale to up to a million concurrently connected devices. This IBM Redbooks® publication introduces MQTT and MessageSight through a simple key fob remote MQTT application. It then dives into the architecture and development of a robust, cross-platform Ride Share and Taxi solution (PickMeUp) with real-time voice, GPS location sharing, and chat among a variety of mobile platforms. The publication also includes an addendum describing use cases in a variety of other domains, with sample messaging topology and suggestions for design.