Towards Fairness & Robustness in Machine Learning for Dermatology
Speaker: Dr. Celia Cintas, IBM Research Africa - Nairobi, Nairobi, Kenya
Abstract: Recent years have seen an overwhelming body of work on fairness and robustness in Machine Learning (ML) models. This is not unexpected, as it is an increasingly important concern as ML models are used to support decision-making in high-stakes applications such as mortgage lending, hiring, and diagnosis in healthcare. Currently, most ML models assume ideal conditions and rely on the assumption that test/clinical data comes from the same distribution of the training samples. However, this assumption is not satisfied in most real-world applications; in a clinical setting, we can find different hardware devices, diverse patient populations, or samples from unknown medical conditions. On the other hand, we need to assess potential disparities in outcomes that can be translated and deepen in our ML solutions. In this presentation, we will discuss how to evaluate skin-tone representation in ML solutions for dermatology and how we can enhance the existing models' robustness by detecting out-out-distribution test samples (e.g., new clinical protocols or unknown disease types) over off-the-shelf ML models.
​
​
Effective and reproducible data visualizations, with demos in Python
Speaker: Kendra Oudyk, McGill University, Montreal, Quebec, Canada
Abstract: This talk explores the principles and practices of data visualization, with demonstrations in Python. We will cover best practices for creating transparent, shareable, and reproducible visualizations, emphasizing the importance of considering human perception when creating visualizations. Key topics include: 1) Data visualization principles and best practices; 2) Demonstrations of visualization techniques using Python libraries such as Matplotlib, Seaborn, and Plotly; and 3) Writing code for reproducible data visualization pipelines.Whether you are new to data visualization or looking to improve your skills, this talk will provide you with the knowledge and tools you need to create effective and transparent visualizations.
​
​
How can we explain medical image segmentation models?
Speaker: Dr. Caroline Petitjean, Université de Rouen Normandie, Rouen, Normandy, France
Abstract:
Explainability is a crucial field of AI, and in medical image analysis in particular, to ensure building trust with the clinicians. In computer vision, most of the explainability works have historically focused on image classification, notably to produce saliency maps that highlight the pixels that most contributed to the decision. Why and how we can produce explainability methods for medical image segmentation are open questions. From the viewpoint of a practitioner, it can be argued that an explanation is less useful, because the prediction is in the image domain, directly understandable by humans through visual analysis. However, existing literature suggests that explainability methods for segmentation models offer other benefits, including providing insights into models, detecting dataset biases, creating counterfactual examples, estimating segmentation-related uncertainties, and identifying pixel contributions to specific regions in an image. Insights from explainable models can also help in transferring knowledge to other tasks and understanding model generalizability.
In this talk, we will present a review of the literature on explainability for image segmentation, spanning from the earliest works published in 2019, and make some proposals on how to generate specific heatmaps that can be useful to assess the reliability of the segmentation model.
​
​
Generalizable and democratic AI: from classical techniques to modern neural networks
Speaker: Dr. Juan Eugenio Iglesias, Massachusetts General Hospital & Harvard Medical School / MIT / UCL
Abstract: Every year, millions of brain MRI scans are acquired in hospitals, which is a figure considerably larger than the size of any research dataset. Therefore, the ability to analyze such scans could transform neuroimaging research – particularly for underrepresented populations that research-oriented datasets like ADNI or UK BioBank often neglect. However, the potential of these scans remains untapped since no automated algorithm is robust enough to cope with the high variability in clinical acquisitions (MR contrasts, resolutions, orientations, artifacts, and subject populations). In this talk, I will present techniques developed by our group over the last few years that enable robust analysis of heterogeneous clinical datasets “in the wild”, including segmentation, registration, super-resolution, and synthesis. The talk will cover both: (i) classical Bayesian techniques, based on high-resolution atlases derived from ex vivo MRI and histology, and (ii) modern neural networks, relying on domain randomization methods for enhanced generalizability. I will present results on thousands of brain scans from our hospital with highly heterogeneous orientation, resolution, and contrast, as well as results on low-field scans acquired with a portable scanner.
​
Continuous evaluation of denoising strategies in fMRI using fMRIPrep and Nillearn
Speaker: Dr. Hao-Ting Wang, Centre de recherche de l'IUGM, Montreal, Canada
Abstract: Functional magnetic resonance imaging (fMRI) signal measures changes in neuronal activity over time. The signal can be contaminated with unwanted noise, such as movement, which can impact research results. To fix this, researchers perform two steps before data analysis: standardised preprocessing and customised denoising. Scientists consult the denoising benchmark literature for guidance. However, the relevant software is ever-evolving, and benchmarks quickly become obsolete. Here, we present a denoising benchmark that can be repeatedly executed on outputs of preprocessing software fMRIPrep, and provide up-to-date guidelines when a new version comes out. Hosted by the Neurolibre reproducible preprint server, anyone can explore the results without executing the benchmark. We contributed back to Nilearn, a popular open-source project to benefit the wider research community. The data preparation components of the benchmark have served as a prototype for two user-friendly applications for large scale data preprocessing for a deep learning project on identifying transdiagnostic brain biomarkers across various neurodegenerative disorders. Overall, the efforts help the fMRI researchers keep tools up-to-date and sustainable and create machine learning ready features to further explore topics in cognitive neuroscience.
​
Improving anatomical plausibility and auditing fairness in deep segmentation networks
Speaker: Dr. Enzo Ferrante, CONICET/Universidad Nacional del Litoral, Provincia de Santa Fe, Argentina
Abstract:
The evolution of deep segmentation networks has empowered the enhancement of extensive medical imaging datasets with automatically generated anatomical segmentation masks. In this talk we will discuss recent methods we proposed to improve anatomical plausibility in deep segmentation networks. By improving anatomical plausibility, we mean to ensure that the segmentation masks produced by our network are constrained to the actual shape and appearance of organs. We will briefly discuss some of our studies [1,2,3] which use autoencoders to learn low dimensional embeddings of anatomical structures and propose different ways in which they can be incorporated into deep learning models for segmentation and registration.
The complexity is further intensified by recent studies indicating potential biases in AI-based medical imaging models related to gender, age, and ethnicity [4,5]. Here we will share insights from our journey in developing the CheXMask large-scale database of x-ray anatomical segmentations [6]. We will delve into the strategies we implemented for automatic quality control and the methods we formulated for unsupervised bias discovery in the absence of ground-truth annotations [7].
​
[1] Mansilla L, Milone D, Ferrante E. – Neural Networks (2020)
[2] Larrazabal A, Martinez C, Glocker B, Ferrante E. – IEEE Transactions on Medical Imaging (2020) – MICCAI 2019 (conference version)
[3] Gaggion N, Mansilla L, Mosquera C, Milone D, Ferrante E. – IEEE Transactions on Medical Imaging (2022) MICCAI 2021 (conference version)
[4] Larrazabal AJ, Nieto N, Peterson V, Milone DH, Ferrante E. – Proceedings of the National Academy of Sciences (2020)
[5] Ricci Lara MA, Echeveste R, Ferrante E. – Nature communications (2022)
[6] Gaggion N, Mosquera C, Mansilla L, Aineseder M, Milone DH, Ferrante E. – arXiv preprint (2023)
[7] Gaggion N, Echeveste R, Mansilla L, Milone DH, Ferrante E. – MICCAI FAIMI Workshop (2023)
​