Search – talk-data.com

Title & Speakers	Event
CAPI: Cluster and Predict Latent Patches for Improved Masked Image Modeling 2025-05-06 · 16:45 Timothée Darcet – PhD student @ Meta AI and Inria Abstract: Masked Image Modeling (MIM) offers a promising approach to self-supervised representation learning, however existing MIM models still lag behind the state-of-the-art. In this talk, we systematically analyze target representations, loss functions, and architectures, to present CAPI - a novel pure-MIM framework that relies on the prediction of latent clusterings. Our approach leverages a clustering-based loss, which is stable to train, and exhibits promising scaling properties. Our ViT-L backbone, CAPI, achieves 83.8% accuracy on ImageNet and 32.1% mIoU on ADE20K with simple linear probes, substantially outperforming previous MIM methods and approaching the performance of the current state-of-the-art, DINOv2. Duration: approximately 45 minutes. masked image modeling (mim) vision transformer (vit) clustering-based loss	#17: CAPI: Cluster & Predict Patches for Improved Image Modeling by T. Darcet

Abstract: Masked Image Modeling (MIM) offers a promising approach to self-supervised representation learning, however existing MIM models still lag behind the state-of-the-art. In this talk, we systematically analyze target representations, loss functions, and architectures, to present CAPI - a novel pure-MIM framework that relies on the prediction of latent clusterings. Our approach leverages a clustering-based loss, which is stable to train, and exhibits promising scaling properties. Our ViT-L backbone, CAPI, achieves 83.8% accuracy on ImageNet and 32.1% mIoU on ADE20K with simple linear probes, substantially outperforming previous MIM methods and approaching the performance of the current state-of-the-art, DINOv2.

Duration: approximately 45 minutes.

Activities & events