Webinar: Beyond the benchmark dataset - October 10, 2025
Wednesday 3rd September 2025
The MICCAI Special Interest Group (SIG) for Challenges presents:
Beyond the benchmark dataset: Real-world generalizability and regulatory challenges in medical AI
Friday, October 10, 2025, 10:00 am EDT / 4:00 PM CEST
Abstract:
Innovation is essential to advancing AI in healthcare, but when innovation is pursued without consideration for deployment and regulatory challenges, it often leads to solutions that consume significant resources yet fail to reach clinical use.
In the first portion of this webinar, Ghada Zamzmi will introduce a regulatory-driven approach to AI design and development—one that integrates safety, effectiveness, regulatory science throughout the entire AI lifecycle from pre-market data collection, model development, evaluation, clinical validation, and post-market monitoring. Ghada will highlight regulatory science–informed research practices that can accelerate the safe and successful deployment of AI technologies.
The second portion of the webinar, led by Jean Feng, will focus on the post-market phase, where AI systems encounter real-world variability. Even models that perform well during development may degrade after deployment, often affecting some subgroups more than others. Jean will present a systematic diagnostic framework, SHIFT, designed to detect and address performance drift, drawing on case studies such as an acute care needs prediction model and an intraoperative acute kidney injury risk model.
Together, these talks will demonstrate how bridging the gap between innovation, deployment challenges, and regulatory science enables the development of medical AI products that are not only cutting-edge, but also safe, effective, and deliver lasting value to patients and healthcare systems.
Speakers' Bios:
Ghada Zamzmi is a regulatory scientist and AI researcher with a background in medical imaging, machine learning, and regulatory science. Over the past decade, she has held roles across academia, government, and industry – bringing together AI expertise with practical regulatory insight and an understanding of real-world deployment challenges. Ghada aims to promote a regulatory-driven mindset in AI development by integrating robust evaluation and regulatory science at every stage of the AI lifecycle. Ghada is active in MICCAI and NeurIPS and has received several prestigious awards, including the MIT Innovators Under 35 and the IEEE Computational Life Sciences Award.
Jean Feng is an Associate Professor in the Department of Epidemiology and Biostatistics at the University of California, San Francisco and the UCSF-UC Berkeley Joint Program in Computational Precision Health, as well as a principal investigator at the UCSF-Stanford Center of Excellence in Regulatory Science and Innovation. She serves as the data science lead of the digital innovation taskforce for the Zuckerberg San Francisco General Hospital. Her research interests include the interpretability, reliability, and regulation of AI/ML algorithms in healthcare.