Webinar: Performance Reporting in Medical Imaging AI - June 10, 2025
Tuesday 20th May 2025
The MICCAI Special Interest Group (SIG) for Challenges presents:
Performance Reporting in Medical Imaging AI
Current practices, Strength of Outperformance Claims and Areas for Improvement
Tuesday June 10, 2025
9:00 am EDT / 3:00 pm CEST
Speakers:
Evangelia Christodoulou, Postdoctoral Scientist, German Cancer Research Center (DKFZ) in Heidelberg, Germany
Olivier Colliot, Research Director at CNRS (Division of Computer Science) and the co-head of the ARAMIS team at the Paris Brain Institute
This webinar, hosted by the MICCAI Special Interest Group (SIG) for Challenges, aims to educate medical imaging researchers on how to successfully participate in and conduct challenges. A particular focus will be placed on issues of quality control and validation using appropriate metrics from the organizers' and participants' point of view.
Abstract:
Reliable performance reporting is essential for the trustworthy validation of medical imaging AI. However, current practices often fall short in supporting robust and reproducible claims of outperformance. In this webinar, we will discuss common pitfalls in performance reporting, such as missing uncertainty quantification and whether common claims of outperformance are well substantiated. We will highlight the role of confidence intervals and other uncertainty-aware reporting techniques as critical tools to strengthen the credibility of performance claims. Finally, we will outline opportunities for improving reporting practices towards more transparent, fair, and clinically meaningful validation standards.
Speakers' Biographies:
Dr. Evangelia Christodoulou holds a background in Mathematics and Biostatistics and completed her PhD in Clinical Prediction Modelling at KU Leuven, Belgium, supervised by Prof. Ben Van Calster, where she collaborated with oncologists and statisticians to advance methods for validating predictive algorithms. In February 2021, she joined the German Cancer Research Center (DKFZ) in Heidelberg, Germany and was awarded a postdoctoral fellowship in 2022 within the AI Health Innovation Cluster, led by Prof. Dr. Lena Maier-Hein. Her current research focuses on the development of robust and reliable AI-based models for clinical outcome prediction in the context of Surgical Data Science. She also works on methodological contributions that address critical challenges in the validation of AI methods for biomedical imaging analysis, with particular emphasis on model performance uncertainty and dataset size considerations.
Dr. Olivier Colliot is a Research Director at CNRS (Division of Computer Science) and the co-head of the ARAMIS team at the Paris Brain Institute. He also holds a chair at the Paris Institute for Artificial Intelligence (PRAIRIE). He has been working for over twenty years on the design and validation of machine learning approaches to better understand, model, diagnose, predict and prevent brain disorders. He is an Associate Editor of Medical Image Analysis, IEEE Transactions on Medical Imaging and the SPIE Journal of Medical Imaging. His current research has a strong focus on statistical aspects of evaluation and benchmarking of AI models. He is a member of the special interest group on biomedical challenges of the MONAI working group on evaluation, reproducibility and benchmarking.
About the SIG for Challenges
The mission of the MICCAI Special Interest Group for Challenges is the establish the best practices for the community, take a leading role in method benchmarking, and provide assistance in challenge-related aspects. Visit our webpage here. Follow us on X @bias_sig.