talk-data.com

Modern search systems increasingly rely on various vector representations—dense embeddings for semantic similarity, sparse vectors for lexical matching, and behavioral vectors from collaborative filtering. But these vector spaces typically operate in isolation, forcing us to choose between different notions of relevance. This talk introduces the concept of "wormhole vectors"—a technique for dynamically connecting and traversing between heterogeneous vector spaces using retrieved documents as a bridge. Rather than running parallel searches and merging results (the most common hybrid search approach), we can start in one vector space (e.g., using sparse embeddings for keyword search), retrieve documents, and then derive wormhole vectors to instantly transport us to another space (e.g., using dense embeddings for semantic search) to continue searching with a fundamentally different notion of similarity. We'll explore practical uses of this approach across three vector spaces: dense semantic embeddings, sparse lexical/keyword vectors, and behavioral vectors from matrix factorization. We’ll demonstrate how document sets can dynamically materialize bridges between these vector spaces, enabling novel retrieval patterns like semantic → behavioral → lexical traversal that surface user intent patterns invisible to single-space approaches.

Search technologies, challenges, and dealing with large data sets - Atita Arora

About the event

Outline:

• Atita’s journey into search
• Search technologies, challenges, and dealing with large data sets
• NLP, voice, and personalization in search
• Scaling and evaluating search technologies

About the speaker:

Atita Arora is a seasoned and esteemed professional shaping the landscape of information retrieval systems. With a robust background spanning from her impactful contributions as a committer in various information retrieval and relevancy workbench projects including Apache OpenNLP and Quepid, to her insightful blogs on leveraging vectors in e-commerce search and an open source implementation called Chorus to pair with. Her commitment to calibrating the essence of "good" in information retrieval systems through OTB metrics and user-centric approaches reflects her dedication to enhancing user experiences. As a fervent believer in diversity and inclusion, Atita actively champions initiatives to foster inclusivity in search and data science. She's a proud Women of Search Group representative and a member of various DEI forums, striving to create a more inclusive environment in tech. Currently immersed in groundbreaking research on evaluating RAGs, Atita continues to drive innovation and positive change in information retrieval while advocating for diversity, making her a true trailblazer in technology and inclusivity. ​

DataTalks.Club is the place to talk about data. Join our slack community!