In this prestigious challenge 24 teams from all over the world submitted their algorithms and pipelines to find out who can detect new MS lesions best. mediaire’s pipelines made it to the top (rank 1 & 3)!
Longitudinal lesion segmentation in Multiple Sclerosis is a very challenging task and often there is not even consensus among expert neuroradiologists whether or not “a white spot” should be counted as a new lesion. 24 teams from 4 continents accepted this challenge and submitted their best algorithms and pipelines. The ground truth was derived from consensus reading of 4 expert neuroradiologists. The performance of the the AI-tools was compared against this ground truth using F1- & DICE-Score as metrics – see example:
In the end mediaire’s approach achieved the best results in terms of lesion detection (F1-Score) and ranked #1 and #3, respectively, while outperforming teams from Siemens, NVIDIA, several well known academic teams and an expert neuroradiologist. However, as neuroradiologists have been involved in generating the ground truth there certainly was a bias in favour of the human rater here…
The full results can be accessed here: MICCAI 2021 New lesion segmentation challenge – Details
The technology used in this challenge is partially available in our commercial product mdbrain already where it is regularly improved based on new data and feedback from our customers.