This webinar series by the SIG for Challenges aims to educate medical imaging researchers on how to successfully participate in and conduct challenges. A particular focus will be placed on issues of quality control and validation using appropriate metrics from the organizers' and participants’ points of view.

(1) How to win a challenge (July 2022)

The recording of the webinar can be found online under this link.

Invited speakers:

Dr. Fabian Isensee has consistently enabled the translation of state-of-the-art algorithms into real-world applications, represented by nnU-Net, the de-facto standard for segmentation in the medical domain. The methods he developed have won multiple international segmentation competitions.

Dr. James Howard is an academic cardiologist who has published numerous research papers using AI to interpret X-rays, cardiac ultrasound, ECG, MRI and cardiac pressure waveforms. He has entered several Kaggle competitions, including the Deepfake Detection Challenge, where he won a gold medal and a $40,000 prize, against 2200 other teams.

(2) How to run a challenge (October 2022)

The recording of the webinar can be found online under this link.

Challenge platforms:

https://grand-challenge.org/ is an open source platform for running challenges. Recently added features include the possibility for challenge participants to upload algorithms that solve a challenge and give other users access to these algorithms to process their own data. Bram van Ginneken, Kiran Vaidhya Venkadesh and Anindo Saha will present how to use the platform for organizing high-profile challenges. The slides of the talk are available from the following link:  grand-challenge-slides.pdf

https://www.synapse.org/ is an open collaboration platform developed by Sage Bionetworks.  Synapse is the main platform supporting DREAM Challenges (dreamchallenges.org). Jake Albrecht from Sage will present tips for challenge organizers on how to define a successful community challenge, with examples from Synapse. The slides of the talk are available from the following link: synapse-slides.pdf

(3) Metrics Reloaded: From segmentation to calibration (February 2023)

The recording of the webinar can be found online under this link.

Invited speakers:

Paul F. Jaeger is a principal investigator at the Interactive Machine Learning Group at the German Cancer Research Center and Helmholtz Imaging. His research focuses on image analysis algorithms, with a particular focus on human interaction. Paul won numerous international competitions on biomedical image analysis and first-authored relevant contributions to the field in high impact journals and conferences like Nature Methods or ICLR. As founder and organizer of heidelberg.ai, Paul helps to connect over 2000 members of the local AI community at monthly events that attract top international scientists to Heidelberg. For his work, Paul received the “Richtzenhain Award for Translational Cancer Research” as well as the “Roland-Ernst Award for Interdisciplinary Research in Radiology”.

Annika Reinke joined the division of Intelligent Medical Systems at the German Cancer Research Center (DKFZ) to adapt mathematical concepts to societally relevant topics, like scientific benchmarking and validation. Having published disruptive findings on biomedical image analysis challenges in Nature Communications, she is a founding member of the initiative of Biomedical Image Analysis ChallengeS (BIAS) aiming for bringing biomedical image analysis challenges to the next level of quality. She serves as the secretary of the MICCAI special interest group on biomedical challenges and as an active member and taskforce lead of the MONAI working group on evaluation, reproducibility and benchmarking.

Florian Buettner is a physicist by training and earned his PhD in physics from the University of London/Institute of Cancer Research in 2011. He then focussed his research efforts on bioinformatics and machine learning at the Helmholtz Zentrum München and the European Bioinformatics Institute in Cambridge. He subsequently transitioned to industry and worked as an expert in artificial intelligence at Siemens AG. Now a professor at Goethe University Frankfurt and the German Cancer Research Center (DKFZ)/German Consortium for Translational Cancer Research (DKTK), Florian is currently doing research at the interface between (single-cell) bioinformatics, machine learning and oncology. In collaboration-driven research, he contributes to developing a better understanding of the molecular heterogeneity of cancer by developing interpretable and trustworthy machine learning methods.

Acknowledgements

This webinar series was partially initiated by the Helmholtz Association of German Research Centers in the scope of the Helmholtz Imaging Incubator (HI).