Following on from the Building consumable data products keynote, we will dive deeper into the interactions around the data product catalog, to show how the network effect of explicit data sharing relationships starts to pay dividends to the participants. Such as:
For the product consumer:
• Searching for products, understanding content, costs, terms and conditions, licenses, quality certifications etc
• Inspecting sample data, choosing preferred data format, setting up a secure subscription, and seeing data provisioned into a database from the product catalog.
• Providing feedback and requesting help
• Reviewing own active subscriptions
• Understanding the lineage behind each product along with outstanding exceptions and future plans
For the product manager/owner:
• Setting up a new product, creating a new release of an existing product and issuing a data correction/restatement
• Reviewing a product’s active subscriptions and feedback/requests from consumers
• Interacting with the technical teams on pipeline implementations along with issues and proposed enhancements
• For the data governance team
• Viewing the network of dependencies between data products (the data mesh) to understand the data value chains and risk concentrations
• Reviewing a dashboard of metrics around the data products including popularity, errors/exceptions, subscriptions, interaction
• Show traceability from a governance policy relating to, say data sovereignty or data privacy to the product implementations.
• Building trust profiles for producers and consumers
The aim of the demonstrations and discussions is to explore the principles and patterns relating to data products, rather than push a particular implementation approach.
Having said that, all of the software used in the demonstrations is open source. Principally this is Egeria, Open Lineage and Unity Catalog from the Linux Foundation, plus Apache Airflow, Apache Kafka and Apache SuperSet from the Apache Software Foundation.
Videos of the demonstrations will be available on YouTube after the conference and the complete demo software can be downloaded and run on a laptop so you can share your experiences with your teams after the event.