How to determine whether your new data product is a success if you cannot use A/B testing techniques?

At Gousto we recently implemented our newest algorithm to route orders to sites. Comparing this to the previous algorithm using classic A/B testing techniques was not possible, because the algorithm requires a full set of orders to optimise and ensure the volume we send to sites remains stable. A routing algorithm is a high impact product. To ensure confidence in our algorithm before go-live, we came up with a different experimentation strategy. This included building a full-blown shadow system. For measuring its performance we built a set of data pipelines (including ETL) using Databricks.

Sometimes an A/B test cannot do the job. This talk will outline challenges and benefits of building a shadow system, providing the audience with an A/B testing alternative and an overview of relevant considerations in terms of choosing and building this experiment design.

Connect with us: Website: https://databricks.com Facebook: https://www.facebook.com/databricksinc Twitter: https://twitter.com/databricks LinkedIn: https://www.linkedin.com/company/data... Instagram: https://www.instagram.com/databricksinc/