Summary The conventional approach to analytics involves collecting large amounts of data that can be cleaned, followed by a separate step for analysis and interpretation. Unfortunately this strategy is not viable for handling real-time, real-world use cases such as traffic management or supply chain logistics. In this episode Simon Crosby, CTO of Swim Inc., explains how the SwimOS kernel and the enterprise data fabric built on top of it enable brand new use cases for instant insights. This was an eye opening conversation about how stateful computation of data streams from edge devices can reduce cost and complexity as compared to batch oriented workflows.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With 200Gbit private networking, scalable shared block storage, and a 40Gbit public network, you’ve got everything you need to run a fast, reliable, and bullet-proof data platform. If you need global distribution, they’ve got that covered too with world-wide datacenters including new ones in Toronto and Mumbai. And for your machine learning workloads, they just announced dedicated CPU instances. Go to dataengineeringpodcast.com/linode today to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show! Listen, I’m sure you work for a ‘data driven’ company – who doesn’t these days? Does your company use Amazon Redshift? Have you ever groaned over slow queries or are just afraid that Amazon Redshift is gonna fall over at some point? Well, you’ve got to talk to the folks over at intermix.io. They have built the “missing” Amazon Redshift console – it’s an amazing analytics product for data engineers to find and re-write slow queries and gives actionable recommendations to optimize data pipelines. WeWork, Postmates, and Medium are just a few of their customers. Go to dataengineeringpodcast.com/intermix today and use promo code DEP at sign up to get a $50 discount! You listen to this show to learn and stay up to date with what’s happening in databases, streaming platforms, big data, and everything else you need to know about modern data management.For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Dataversity, Corinium Global Intelligence, and Data Council. Upcoming events include the O’Reilly AI conference, the Strata Data conference, the combined events of the Data Architecture Summit and Graphorum, and Data Council in Barcelona. Go to dataengineeringpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today. Your host is Tobias Macey and today I’m interviewing Simon Crosby about Swim.ai, a data fabric for the distributed enterprise

Interview

Introduction How did you get involved in the area of data management? Can you start by explaining what Swim.ai is and how the project and business got started?

Can you explain the differentiating factors between the SwimOS and Data Fabric platforms that you offer?

What are some of the use cases that are enabled by the Swim platform that would otherwise be impractical or intractable? How does Swim help alleviate the challenges of working with sensor oriented applications or edge computing platforms? Can you describe a typical design for an application or system being built on top of the Swim platform?

What does the developer workflow look like?

What kind of tooling do you have for diagnosing and debugging errors in an application built on top of Swim?

Can you describe the internal design for the SwimOS and ho

This podcast spends time discussing Tim O'Reilly's futuristic perspective on data, analytics, AI, jobs, and organization. He sheds light on what are somethings businesses could do to stay relevant and future proof. He discussed his book and shared some of the key insights relevant to anyone thinking of staying relevant in the World led by technology and impacting the future. A must video for anyone working!

Timeline: 00:28 Tim's journey. 06:03 Tim's current occupation. 10:50 Interesting work for interesting people. 15:08 Thinking behind the title "What's the future". 23:41 Culture and technology evolution. 26:29 Creating value for the shareholder. 35:06 Learning a new skill. 38:12 Labor and technology. 47:07 Investing in humans or technology? 56:02 The role of AI in Media. 59:45 How can an employee stay relevant? 1:04:28 Tim's favorite books. 1:09:38 Key takeaways.

Tim's Book: WTF?: What's the Future and Why It's Up to Us by Tim O'Reilly https://amzn.to/2N5WhOn

Tim's Recommended Read: AI Superpowers: China, Silicon Valley, and the New World Order by Kai-Fu Lee https://amzn.to/2N8VGLL Prediction Machines: The Simple Economics of Artificial Intelligence by Ajay Agrawal and Joshua Gans https://amzn.to/2ugQBKr The Long Twentieth Century: Money, Power and the Origins of Our Times by Giovanni Arrighi https://amzn.to/2ufhb6R Doughnut Economics: Seven Ways to Think Like a 21st-Century Economist by Kate Raworth https://amzn.to/2LcbLQc Winners Take All: The Elite Charade of Changing the World by Anand Giridharadas https://amzn.to/2utgeXF New Power: How Power Works in Our Hyperconnected World--and How to Make It Work for You by Jeremy Heimans and Henry Timms https://amzn.to/2NbBJ77 Seeing like a State: How Certain Schemes to Improve the Human Condition Have Failed by James C. Scott https://amzn.to/2ztnoRz The Struggle for Survival: An Historical, political, and Socioeconomic Perspective of St. Lucia by Anderson Reynolds https://amzn.to/2uqF22w

Podcast Link: https://futureofdata.org/discussing-jobs-data-and-whatsthefuture-with-timoreilly-futureofdata-podcast/

Tim's BIO: Tim O’Reilly is the founder and CEO of O’Reilly Media, Inc. His original business plan was “interesting work for interesting people,” which worked out pretty well. O’Reilly Media delivers online learning, publishes books, runs conferences, urges companies to create more value than they capture, and tries to change the world by spreading and amplifying the knowledge of innovators.

Tim has a history of convening conversations that reshape the computer industry. In 1993, he launched the first commercial, ad-supported site on the internet. In 1998, he organized the meeting where the term “open source software” was agreed on and helped the business world understand its importance. In 2004, with the Web 2.0 Summit, he defined how “Web 2.0” represented not only the resurgence of the web after the dot com bust, but a new model for the computer industry, based on big data, collective intelligence, and the internet as a platform. In 2009, with his “Gov 2.0 Summit,” he framed a conversation about the modernization of government technology that has shaped policy and spawned initiatives at the Federal, State, and local level and around the world. He has now turned his attention to the implications of AI, the on-demand economy, and other technologies that are transforming the nature of work and the future shape of the business world. This is the subject of his forthcoming book from Harper Business, WTF: What’s the Future and Why It’s Up to Us.

About #Podcast:

FutureOfData podcast is a conversation starter to bring leaders, influencers, and lead practitioners to discuss their journey in creating the data-driven future.

Wanna Join? If you or any you know wants to join in or sponsor, Email us @ [email protected]

Keywords:

FutureOfData #DataAnalytics #Leadership #Futurist #Podcast #BigData #Strategy

In the four years that O'Reilly Media, Inc. has produced its annual Big Data Now report, the data field has grown from infancy into young adulthood. Data is now a leader in some fields and a driver of innovation in others, and companies that use data and analytics to drive decision-making are outperforming their peers. And while access to big data tools and techniques once required significant expertise, today many tools have improved and communities have formed to share best practices. Companies have also started to emphasize the importance of processes, culture, and people. The topics in represent the major forces currently shaping the data world: Big Data Now: 2014 Edition Cognitive augmentation: predictive APIs, graph analytics, and Network Science dashboards Intelligence matters: defining AI, modeling intelligence, deep learning, and "summoning the demon" Cheap sensors, fast networks, and distributed computing: stream processing, hardware data flows, and computing at the edge Data (science) pipelines: broadening the coverage of analytic pipelines with specialized tools Evolving marketplace of big data components: SSDs, Hadoop 2, Spark; and why datacenters need operating systems Design and social science: human-centered design, wearables and real-time communications, and wearable etiquette Building a data culture: moving from prediction to real-time adaptation; and why you need to become a data skeptic Perils of big data: data redlining, intrusive data analysis, and the state of big data ethics