MICCAI MICCAI


News


Call for Challenges

Monday 25th October 2021

The 2022 MICCAI conference will be held in Singapore from September 18 to 22, 2022. The conference will host a variety of workshops, challenges, and tutorials. Challenges have become an integral part of MICCAI conferences in the past years. Their aim is to provide a fair and direct comparison of different methodological solutions to a common problem. Challenges should address a well-defined open problem relevant to the MICCAI, provide high-quality data for testing/training algorithms, and define a clear assessment procedure. Examples of topics from previous challenges include, but are not limited to, segmentation, registration, computer-aided detection and diagnosis, endoscopic vision, and brain connectivity. Proposals related to accessible, fair and responsible healthcare are particularly welcome.

The sessions may be designed as oral/poster presentations of the results that each participant achieved prior to the conference and/or hands-on competitions during the event. The MICCAI supports open data; hence, we encourage the organizers to use open-source data licenses and/or keep the challenge open to submissions beyond the conference, providing a sustained platform for benchmarking.

Challenges will take place on September 18 and 22, 2022. The events will be located at the main conference venue for in-person attendance. Fully virtual challenges will also be considered upon request. Each challenge may be organized as a half-day (8:00am-12:30pm or 1:30pm-5:00pm) or a full-day (8:00am-5:00pm) meeting.

Submission details

According to a recent study on common practice in biomedical image analysis [1], reproduction and adequate interpretation of challenge results are often not possible, because only a fraction of relevant information is typically reported. To address this issue, we have implemented an online platform [2] for challenge proposal submissions with structured descriptions of challenge protocols based on [3]. The parameters for challenge design have been developed in collaboration with multiple institutions worldwide.

This framework will ensure the completeness of the information for a high-quality challenge review as well as structured access to the challenge information in the future. Additionally, It may help authors improve the design of their challenge.

The proposal submission site [2] starts accepting submissions on November 1, 2021. Proposal submissions are due on December 10, 2021.

The CMT platform [4], which is specified in peer review processes, will be used for the review process this year.

Challenge registration

A recent MICCAI paper [5] showed that common practice in biomedical challenge organization can be exploited to tune challenge rankings. To prevent such incidents and to improve the quality of challenges, the MICCAI Board Challenge Working Group [6] and the MICCAI 2020 Satellite Event team decided to introduce the concept of challenge registration. Similar to how clinical trials have to be registered before starting, the complete design of accepted MICCAI challenges will be put online before the challenges take place. Changes to the design (e.g. to the metrics or ranking schemes applied) must be well-justified and officially registered online (as a new version of the challenge design).

To facilitate this process for the challenge organizers, the challenge submission system [2] has been updated in a way that allows for straightforward export of the challenge design as a PDF document. The satellite event team will export the document and have it signed by the contact person of the challenge before uploading it.

Registering challenges is an important step towards higher quality challenges. It not only has the potential to lead to more thoughtful challenge designs but also ensures that all the necessary information is available for challenge participants. Furthermore, all changes will be transparent to the community, ensuring increased quality control.

Challenge application two years in advance

Organizing a challenge based on an accepted challenge proposal may be time-consuming, especially when large-scale data collection and annotation is necessary. To address the traditionally tight schedule between MICCAI challenge acceptance and organization we also offer the possibility of an early review of MICCAI 2023 challenge proposals this year. Organizers interested in organizing a challenge in 2023 may use this year's submission system [2] and thus receive an early acceptance that leaves time for careful preparation of the challenge. For MICCAI 2022, three challenges were already accepted by this process:

Selection criteria

Due to the limited number of rooms available, and to remain relevant to the conference participants, the submitted challenge proposals will be selected based on the following criteria:

  • potential scientific, economic and social impact;
  • expected interest in the topic by the MICCAI audience;
  • quality of the challenge design.

Proposals will be reviewed by at least two reviewers. The reviewers will be chosen based on their expertise in the area of a proposal. Reviewers will provide feedback and assist the challenge organizing committee in making a final decision. To ensure a rich and diverse program, we reserve the right to ask the applicants of proposals with substantial overlaps to merge or alter their scopes. We also reserve the right to ask applicants to provide supplementary materials and to revise their proposals in order to ensure the quality and standard of the MICCAI challenges.

Copyright

Authors will assign to the MICCAI Society the right to distribute challenge materials (such as the agenda or the challenge design document) to MICCAI members and to challenge and conference attendees, independently of the copyright ownership planned by the challenge organizers.

Budget

MICCAI 2022 will not pay for travel arrangements or stipends for the organizers of satellite events nor provide concessions on registration fees. The MICCAI 2022 Satellite Events Committee has limited funds for waiving the registration fees for a number of speakers, including those from outside the MICCAI community. Organizers and speakers who only plan to attend the satellite event in which they are involved are expected to register for the event unless registration fee waivers have been requested and granted.

Important dates

November 1, 2021: Challenge submission system opens
December 10, 2021: Proposals due date
January 17, 2022: Reviewer deadline
January 24, 2022: First round of feedback
February 11, 2022: Deadline for submission of revised challenge proposals
February 25, 2022: Final notification of acceptance
April 1, 2022: Individual websites opening

URLs and references

  1. Maier-Hein et al. (2018). Why rankings of biomedical image analysis competitions should be interpreted with care. Nature Communications, 9(1), 5217. doi: https://doi.org/10.1038/s41467-018-07619-7
  2. Challenge Submission System - https://www.biomedical-challenges.org/miccai2021/
  3. Maier-Hein et al. (2020) BIAS: Transparent reporting of biomedical image analysis challenges. Medical Image Analysis, 101796. doi: https://doi.org/10.1016/j.media.2020.101796
  4. Conference Management Toolkit - https://cmt3.research.microsoft.com/
  5. Reinke et al. (2018). How to exploit weaknesses in biomedical challenge design and organization. Medical Image Computing and Computer Assisted Intervention - MICCAI 2018 (pp. 388-395). Springer International Publishing. doi: https://doi.org/10.1007/978-3-030-00937-3_45
  6. MICCAI Board Challenge Working Group - http://miccai.org/events/challenges/

Developed by iGraph