Register for the Zoom
Exploring DeepSeek’s Janus-Pro Visual Question Answer (VQA) Capabilities
DeepSeek‘s Janus-Pro is an advanced multimodal model designed for both multimodal understanding and visual generation, with a particular emphasis on improvements in understanding tasks. The model’s architecture is built upon the concept of decoupled visual encoding, which allows it to handle the differing representation needs of these two types of tasks more effectively.
In this talk, we’ll explore Janus-Pro’s Visual Question Answer (VQA) capabilities using FiftyOne’s Janus-Pro VQA Plugin.
The plugin provides a seamless interface to Janus Pro’s visual question understanding capabilities within FiftyOne, offering:
- Vision-language tasks
- Hardware acceleration (CUDA/MPS) when available
- Dynamic version selection from HuggingFace
- Full integration with FiftyOne’s Dataset and UI
Can’t wait to see it for yourself? Check out the FiftyOne Quickstart with Janus-Pro.
About the Speaker
Harpreet Sahota is a hacker-in-residence and machine learning engineer with a passion for deep learning and generative AI. He’s got a deep interest in RAG, Agents, and Multimodal AI.
Getting the Most Out of FiftyOne Open-Source for Gen AI Workflows
In this talk we’ll explore how we maximize the potential of the FiftyOne open source SDK and App to efficiently store and annotate training data critical to Finegrain‘s Generative AI workflows. We will provide an overview of our cloud-based storage and hosting architecture, showcase how we leverage FiftyOne for training and applying models for semi-automatic data annotation, and demonstrate how we extend the CVAT integration to enable pixel-perfect side-by-side evaluation of our Generative AI models.
About the Speaker
Maxime Brénon is a machine learning and data engineer. An Xoogler he started his machine learning journey at Moodstocks when AlexNet was all the rage.
BIOSCAN-5M: A Multimodal Dataset for Insect Biodiversity
Measuring biodiversity is crucial for understanding global ecosystem health, especially in the face of anthropogenic environmental changes. Rates of data collection are ever increasing, but access to expert human annotation is limited, making this an ideal use-case for machine learning solutions. The newly released BIOSCAN-5M dataset features five million specimens from 47 countries around the world, with paired high-resolution images and DNA barcodes for every sample.
The dataset’s hierarchical taxonomic labels, geographic data, and long-tail distribution of rare species offer valuable resources for ecological research and AI model training. The dataset enables large-scale multimodal modelling for insect biodiversity, and poses challenging machine learning problems for fine-grained classification both for recognising known species of insects (closed-world), and handling novel species (open-world). BIOSCAN-5M represents a significant advancement in biodiversity informatics, facilitated by the International Barcode of Life and the BIOSCAN project, and is publicly available for download via Hugging Face and PyPI.
About the Speaker
Scott C. Lowe is a British machine learning researcher based at the Vector Institute in Toronto, Canada. His work is multidisciplinary, spanning several topics. Recently he has focused on biodiversity monitoring applications for both insects (BIOSCAN) and ocean habitats (BenthicNet), self-supervised learning, reasoning capabilities of LLMs, and symbolic music generation. Previously, he completed his PhD in Neuroinformatics from the University of Edinburgh.
Fine Tuning Moondream2
Stay tuned for the talk abstract!
About the Speaker
Parsa Khazaeepoul is the Head of Developer Relations at Moondream AI, where he focuses on making computer vision more accessible. A Summa Cum Laude graduate of the University of Washington’s Informatics program, Parsa also spearheaded developer relations at the AI2 Incubator and co-founded Turing Minds, a renowned speaker series featuring Turing Award winners and other leading figures in computer science. His work has impacted thousands through projects like CourseFinder and uwRMP, and he’s a recognized innovator in the Seattle tech scene, named to the Seattle Inno Under 25 Class of 2024.
