Presentation Title: Data Engineering and the rise of the API

Description APIs, once the preserve of application developers or enterprise service bus integration solutions, are now commonplace for data engineers to use. With increased use of SaaS platforms holding key business data, we can no longer connect to the database directly. Rather, we need to pull data from an API, which will be unique per-platform. Join me as I work through the key characteristics of APIs, from authentication, headers, body, and parameters, through to pagination for large volumes of data. I will demonstrate how we can utilise APIs in our data pipelines to get the data we need, when we need it.