talk-data.com talk-data.com

J

Speaker

Jianmei Ye

1

talks

Sr. Software Engineer Adobe, Inc.

Jianmei is a Senior Software Engineer who has been part of Adobe Experience Platform for 7+ years, leading several critical initiatives within Identity Services—from building large-scale streaming ingestion pipelines to enabling incremental graph exports. She has deep expertise in data lakes and is a passionate Spark enthusiast. Most recently, she played a key role in enhancing the graph store’s performance by helping drive its migration to Foundation DB, a distributed key-value store designed for low-latency, high-throughput workloads. Jianmei is always eager to explore new technologies and push the boundaries of scalable data engineering.

Bio from: Data + AI Summit 2025

Filter by Event / Source

Talks & appearances

1 activities · Newest first

Search activities →
Scaling Identity Graph Ingestion to 1M Events/Sec with Spark Streaming & Delta Lake

Adobe’s Real-Time Customer Data Platform relies on the identity graph to connect over 70 billion identities and deliver personalized experiences. This session will showcase how the platform leverages Databricks, Spark Streaming and Delta Lake, along with 25+ Databricks deployments across multiple regions and clouds — Azure & AWS — to process terabytes of data daily and handle over a million records per second. The talk will highlight the platform’s ability to scale, demonstrating a 10x increase in ingestion pipeline capacity to accommodate peak traffic during events like the Super Bowl. Attendees will learn about the technical strategies employed, including migrating from Flink to Spark Streaming, optimizing data deduplication, and implementing robust monitoring and anomaly detection. Discover how these optimizations enable Adobe to deliver real-time identity resolution at scale while ensuring compliance and privacy.