talk-data.com

Today I’m joined by Vishal Singh, Head of Data Products at Starburst and co-author of the newly published e-book, Data Products for Dummies. Throughout our conversation, Vishal explains how the variations in definitions for a data product actually led to the creation of the e-book, and we discuss the differences between our two definitions. Vishal gives a detailed description of how he believes Data Product Managers should be conducting their discovery and gathering feedback from end users, and how his team evaluates whether their data products are truly successful and user-friendly.

Highlights/ Skip to:

I introduce Vishal, the Head of Data Products at Starburst and contributor of the e-book Data Products for Dummies (00:37) Vishal describes how his customers at Starburst all had a common problem, but differing definitions of a data product, which led to the creation of his e-book (01:15) Vishal shares his one-sentence definition of a data product (02:50) How Vishal’s definition of a data product differs from mine, and we both expand on the possibilities between the two (05:33) The tactics Vishal uses to useful feedback to ensure the data products he develops are valuable for end users (07:48) Why Vishal finds it difficult to get one on one feedback from users during the iteration phase of data product development (11:07) The danger of sunk cost bias in the iteration phase of data product development (13:10) Vishal describes how he views the role of a DPM when it comes to doing effective initial discovery (15:27) How Vishal structures his teams and their interactions with each other and their end users (21:34) Vishal’s thoughts on how design affects both data scientists and end users (24:16) How DPMs at Starburst evaluate if the data product design is user-friendly (28:45) Vishal’s views on where Designers are valuable in the data product development process (35:00) Vishal and I discuss the importance of ensuring your products truly solve your user’s problems (44:44) Where you can learn more about Vishal’s upcoming events and the e-book, Data Products for Dummies (49:48)

Links Starburst: https://www.starburst.io/ Data Products for Dummies: https://www.starburst.io/info/data-products-for-dummies/ “How to Measure the Impact of Data Products with Doug Hubbard”: https://designingforanalytics.com/resources/episodes/080-how-to-measure-the-impact-of-data-productsand-anything-else-with-forecasting-and-measurement-expert-doug-hubbard/ Trino Summit: https://www.starburst.io/info/trinosummit2023/ Galaxy Platform: https://www.starburst.io/platform/starburst-galaxy/ Datanova Summit: https://www.starburst.io/datanova/ LinkedIn: https://www.linkedin.com/in/singhsvishal/ Twitter: https://twitter.com/vishal_singh

Summary

With all of the messaging about treating data as a product it is becoming difficult to know what that even means. Vishal Singh is the head of products at Starburst which means that he has to spend all of his time thinking and talking about the details of product thinking and its application to data. In this episode he shares his thoughts on the strategic and tactical elements of moving your work as a data professional from being task-oriented to being product-oriented and the long term improvements in your productivity that it provides.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management When you're ready to build your next pipeline, or want to test out the projects you hear about on the show, you'll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don't forget to thank them for their continued support of this show! Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. By the time errors have made their way into production, it’s often too late and damage is done. Datafold built automated regression testing to help data and analytics engineers deal with data quality in their pull requests. Datafold shows how a change in SQL code affects your data, both on a statistical level and down to individual rows and values before it gets merged to production. No more shipping and praying, you can now know exactly what will change in your database! Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control. Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Sign up free at dataengineeringpodcast.com/rudder Build Data Pipelines. Not DAGs. That’s the spirit behind Upsolver SQLake, a new self-service data pipeline platform that lets you build batch and streaming pipelines without falling into the black hole of DAG-based orchestration. All you do is write a query in SQL to declare your transformation, and SQLake will turn it into a continuous pipeline that scales to petabytes and delivers up to the minute fresh data. SQLake supports a broad set of transformations, including high-cardinality joins, aggregations, upserts and window operations. Output data can be streamed into a data lake for query engines like Presto, Trino or Spark SQL, a data warehouse like Snowflake or Redshift., or any other destination you choose. Pricing for SQLake is simple. You pay $99 per terabyte ingested into your data lake using SQLake, and run unlimited transformation pipelines for free. That way data engineers and data users can process to their heart’s content without worrying about their cloud bill. For data engineering podcast listeners, we’re offering a 30 day trial with unlimited data, so go to dataengineeringpodcast.com/upsolver today and see for yourself how to avoid DAG hell. Your host is Tobias Macey and today I'm interviewing Vishal Singh about his experience

HighlightsGrab your passports, it’s Excursion Thursday, and we’re headed to Mumbai, India’s largest city and Spotify’s largest potential market.Mission   Good morning, it’s Jason here at Chartmetric with your 3-minute Data Dump where we upload charts, artists, and playlists into your brain so you can stay up on the latest in the music data world.DateThis is your Data Dump for Thursday, May 23rd, 2019.Excursion Thursday: MumbaiOn today’s Excursion Thursday, we’re taking off to India’s most populated city, Mumbai, which has quickly become a testing ground for Spotify’s global expansion strategy. Until 1995, the “Hollywood of India” was also called Bombay, what many in India saw as a vestige of British colonialism, hence the name change. The city’s booming movie industry lends the city its other famous moniker, “Bollywood”.Mumbai is not only the wealthiest city in India, but it’s also arguably the financial, arts, and entertainment capital of the entire country with an estimated 22.5 million  Mumbaikars more than doubling the population of New York City!It’s clear why Spotify’s weathering its recent challenges in-country, as India’s population is currently at 1.4 billion and climbing — that’s almost 20 percent of everybody on earth, while North America comprises around 5 percent. So, if Spotify’s been able to acquire an estimated 50M monthly active users out of North America’s 366M people and an estimated 60M monthly active users out of Europe’s 743M people, that gives them a market penetration rate lying somewhere between 8 and 15 percent. Apply that to a population of 1.4B, and SPOT’s stock price will rise, for sure.So, based on the city’s listening profile….how’s it going? Unfortunately, it’s too early to tap into Spotify’s local monthly listeners, but we can at least look at other Western platforms that are operating there.Mumbai’s Shazam and YouTube charts definitely reflect the battle between domestic and foreign repertoire preferences.According to the Top 90 tracks by Shazam Chart Occurrences in the past month, a total of 22 bear Indian ISRC codes. That’s around 25% of total Shazam’d tracks we captured, while there are 38 US-based ISRCs present, about 40%.Moving to Shazam’s most charted artists in Mumbai over the last 30 days, American rappers Swae Lee and Lil Nas X come in 1st and 3rd with 52 and 47 chart appearances, respectively, and Puerto Rican singer Farruko in 2nd with 50. Fourth and 5th place go to film music composers Vishal-Shekhar and star singer Arijit Singh with 42 and 41 chart appearances each.Using Top Tracks by YouTube Views, we see a mixed bag at the top, with T. Swift and Brendon Urie’s “Me!” at 235K average daily views and Katy Perry and Migos’ “Bon Appétit” at 77K daily views in 1st and 3rd place respectively. Second place goes to “Aankh Mare” from Bollywood movie Simmba sitting pretty at 188K views. Genre-wise on the Shazam charts in the past month, it’s still a battle between local and foreign fare: with Hip-Hop at 11 genre tags from mostly American artists, Dance at 15 genre tags from an international artist roster, and Pop at 22 genre tags from both Western and Indian artists. Twelve of Pop genre tags are from domestic artists, suggesting there’s a slight skew in the past month  toward the local when it comes to the genre.While Spotify competes with the entrenched Indian streaming service JioSaavn, partly headquartered in Mumbai and specializing in Bollywood music , Mumbai’s demand for both Indian and Western music will prove to either be Spotify’s ace in the hole or rock in its shoe.OutroThat’s a wrap for your Daily Data Dump for Thursday, May 23nd, 2019. This is Jason from Chartmetric.Free accounts are at app.chartmetric.com/signupAnd article links and show notes are at: podcast.chartmetric.com.Hope you’re not too jet-lagged from today’s Excursion Thursday, and we’ll see you back here tomorrow!