Publications

2026

1. 

   Augmenting Chest X-ray Datasets with Non-Expert Annotations

   Veronika Cheplygina, Cathrine Damgaard, Trine Naja Eriksen, Dovile Juodelyte, and Amelia Jiménez-Sánchez

   In Medical Image Understanding and Analysis, 2026

   Abs URL Code

   The advancement of machine learning algorithms in medical image analysis requires the expansion of training datasets. A popular and cost-effective approach is automated annotation extraction from free-text medical reports, primarily due to the high costs associated with expert clinicians annotating medical images, such as chest X-rays. However, it has been shown that the resulting datasets are susceptible to biases and shortcuts. Another strategy to increase the size of a dataset is crowdsourcing, a widely adopted practice in general computer vision with some success in medical image analysis. In a similar vein to crowdsourcing, we enhance two publicly available chest X-ray datasets by incorporating non-expert annotations. However, instead of using diagnostic labels, we annotate shortcuts in the form of tubes. We collect 3.5k chest drain annotations for NIH-CXR14, and 1k annotations for four different tube types in PadChest, and create the Non-Expert Annotations of Tubes in X-rays (NEATX) dataset. We train a chest drain detector with the non-expert annotations that generalizes well to expert labels. Moreover, we compare our annotations to those provided by experts and show “moderate” to “almost perfect” agreement. Finally, we present a pathology agreement study to raise awareness about the quality of ground truth annotations. We make our dataset available on Zenodo at https://zenodo.org/records/14944064and our code available at https://github.com/purrlab/chestxr-label-reliability.

2. 

   Learning to Harmonize Cross-Vendor X-ray Images by Non-linear Image Dynamics Correction

   Yucheng Lu, Shunxin Wang, Dovile Juodelyte, and Veronika Cheplygina

   In Medical Image Understanding and Analysis, 2026

   Code

2025

1. 

   Mask of truth: model sensitivity to unexpected regions of medical images

   Théo Sourget, Michelle Hestbek-Møller, Amelia Jiménez-Sánchez, Jack Junchi Xu, and Veronika Cheplygina

   Journal of Imaging Informatics in Medicine, 2025

   URL Code

2. 

   Exploring connections of spectral analysis and transfer learning in medical imaging

   Yucheng Lu, Dovile Juodelyte, Jonathan D Victor, and Veronika Cheplygina

   In Medical Imaging 2025: Image Processing, 2025

   Code

3. 

   In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review

   Amelia Jiménez-Sánchez, Natalia-Rozalia Avlona, Sarah Boer, Vı́ctor M. Campello, Aasa Feragen, and 24 more authors

   In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, , 2025

   Abs DOI URL

   Datasets play a critical role in medical imaging research, yet issues such as label quality, shortcuts, and metadata are often overlooked. This lack of attention may harm the generalizability of algorithms and, consequently, negatively impact patient outcomes. While existing medical imaging literature reviews mostly focus on machine learning (ML) methods, with only a few focusing on datasets for specific applications, these reviews remain static – they are published once and not updated thereafter. This fails to account for emerging evidence, such as biases, shortcuts, and additional annotations that other researchers may contribute after the dataset is published. We refer to these newly discovered findings of datasets as research artifacts. To address this gap, we propose a living review that continuously tracks public datasets and their associated research artifacts across multiple medical imaging applications. Our approach includes a framework for the living review to monitor data documentation artifacts, and an SQL database to visualize the citation relationships between research artifact and dataset. Lastly, we discuss key considerations for creating medical imaging datasets, review best practices for data annotation, discuss the significance of shortcuts and demographic diversity, and emphasize the importance of managing datasets throughout their entire lifecycle. Our demo is publicly available at http://inthepicture.itu.dk/.

2024

1. 

   On dataset transferability in medical image classification

   Dovile Juodelyte, Enzo Ferrante, Yucheng Lu, Prabhant Singh, Joaquin Vanschoren, and 1 more author

   arXiv preprint arXiv:2412.20172, 2024

   URL

2. 

   SeagrassFinder: Deep Learning for Eelgrass Detection and Coverage Estimation in the Wild

   Jannik Elsäßer, Laura Weihl, Veronika Cheplygina, and Lisbeth Tangaa Nielsen

   arXiv preprint arXiv:2412.16147, 2024

   URL

3. 

   Confidence intervals uncovered: Are we ready for real-world medical imaging AI?

   Evangelia Christodoulou, Annika Reinke, Rola Houhou, Piotr Kalinowski, Selen Erkan, and 6 more authors

   In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2024

   URL

4. 

   Dataset Distribution Impacts Model Fairness: Single Vs. Multi-task Learning

   Ralf Raumanns, Gerard Schouten, Josien PW Pluim, and Veronika Cheplygina

   In Ethics and Fairness in Medical Imaging, 2024

   URL

5. 

   Source Matters: Source Dataset Impact on Model Robustness in Medical Imaging

   Dovile Juodelyte, Yucheng Lu, Amelia Jiménez-Sánchez, Sabrina Bottazzi, Enzo Ferrante, and 1 more author

   International Workshop on Applications of Medical AI (AMAI), 2024

   URL

6. 

   [Citation needed] Data usage and citation practices in medical imaging conferences

   Théo Sourget, Ahmet Akkoç, Stinna Winther, Christine Lyngbye Galsgaard, Amelia Jiménez-Sánchez, and 3 more authors

   In Medical Imaging with Deep Learning, 2024

   URL PDF Code Poster

7. 

   Copycats: the many lives of a publicly available medical imaging dataset

   Amelia Jiménez-Sánchez, Natalia-Rozalia Avlona, Dovile Juodelyte, Théo Sourget, Caroline Vang-Larsen, and 3 more authors

   In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024

   URL PDF Poster

2023

1. 

   Detecting Shortcuts in Medical Images - A Case Study in Chest X-Rays

   Amelia Jiménez-Sánchez, Dovile Juodelyte, Bethany Chamberlain, and Veronika Cheplygina

   In 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), 2023

   DOI URL Code

2. 

   Revisiting Hidden Representations in Transfer Learning for Medical Imaging

   Dovile Juodelyte, Amelia Jiménez-Sánchez, and Veronika Cheplygina

   Transactions on Machine Learning Research, 2023

   URL

2022

1. 

   Machine learning for medical imaging: methodological failures and recommendations for the future

   Gaël Varoquaux and Veronika Cheplygina

   NPJ Digital Medicine, 2022

   DOI URL

2. 

   Detection of Furigana Text in Images

   Nikolaj Kjøller Bjerregaard, Veronika Cheplygina, and Stefan Heinrich

   arXiv preprint arXiv:2207.03960, 2022

   URL

3. 

   Predicting Bearings’ Degradation Stages for Predictive Maintenance in the Pharmaceutical Industry

   Dovile Juodelyte, Veronika Cheplygina, Therese Graversen, and Philippe Bonnet

   arXiv preprint arXiv:2203.03259, 2022

   URL

4. 

   Memory-aware curriculum federated learning for breast cancer classification

   Amelia Jiménez-Sánchez, Mickael Tardy, Miguel A González Ballester, Diana Mateus, and Gemma Piella

   Computer Methods and Programs in Biomedicine, 2022

   URL Code

2021

1. 

   ENHANCE (ENriching Health data by ANnotations of Crowd and Experts): A case study for skin lesion classification

   Ralf Raumanns, Gerard Schouten, Max Joosten, Josien PW Pluim, and Veronika Cheplygina

   MELBA (Machine Learning for Biomedical Imaging), 2021

   URL

2. 

   Cats, not CAT scans: a study of dataset similarity in transfer learning for 2D medical image classification

   Irma van den Brandt, Floris Fok, Bas Mulders, Joaquin Vanschoren, and Veronika Cheplygina

   arXiv preprint arXiv:2107.05940, 2021

   URL
3. High-level prior-based loss functions for medical image segmentation: A survey

   Rosana El Jurdi, Caroline Petitjean, Paul Honeine, Veronika Cheplygina, and Fahed Abdallah

   Computer Vision and Image Understanding, 2021

   URL
4. Crowdsourcing airway annotations in chest computed tomography images

   Veronika Cheplygina, Adria Perez-Rovira, Wieying Kuo, Harm AWM Tiddens, and Marleen Bruijne

   PLoS ONE, 2021

   URL

2020

1. Primary Tumor Origin Classification of Lung Nodules in Spectral CT using Transfer Learning

   Linde S Hesse, Pim A Jong, Josien PW Pluim, and Veronika Cheplygina

   arXiv preprint arXiv:2006.16633, 2020

   URL
2. Predicting Scores of Medical Imaging Segmentation Methods with Meta-Learning

   Tom Sonsbeek and Veronika Cheplygina

   In Interpretable and Annotation-Efficient Learning for Medical Image Computing (MICCAI LABELS), 2020

   URL
3. Risk of Training Diagnostic Algorithms on Data with Demographic Bias

   Samaneh Abbasi-Sureshjani, Ralf Raumanns, Britt EJ Michels, Gerard Schouten, and Veronika Cheplygina

   In Interpretable and Annotation-Efficient Learning for Medical Image Computing (MICCAI LABELS), 2020

   URL

2019

1. A survey of crowdsourcing in medical image analysis

   Silas Ørting, Andrew Doyle, Arno Hilten, Matthias Hirth, Oana Inel, and 4 more authors

   Human Computation Journal, 2019

   URL
2. Cats or CAT scans: Transfer learning from natural or medical image source data sets?

   Veronika Cheplygina

   Current Opinion in Biomedical Engineering, 2019

   URL
3. Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis

   Veronika Cheplygina, Marleen Bruijne, and Josien PW Pluim

   Medical Image Analysis, 2019

   URL

2018

1. Automatic Emphysema Detection using Weakly Labeled HRCT Lung Images

   Isabel Pino Pena, Veronika Cheplygina, Sofia Paschaloudi, Morten Vuust, Jesper Carl, and 3 more authors

   PLoS ONE, 2018

   URL
2. Crowd disagreement about medical images is informative

   Veronika Cheplygina and Josien P W Pluim

   In Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis (MICCAI LABELS), 2018

   URL
3. Multiple Instance Learning: A Survey of Problem Characteristics and Applications

   M.-A. Carbonneau, V. Cheplygina, E. Granger, and G. Gagnon

   Pattern Recognition, 2018

   URL
4. Transfer learning for multi-center classification of chronic obstructive pulmonary disease

   Veronika Cheplygina, Isabel Pino Peña, Jesper Holst Pedersen, David A Lynch, Lauge Sørensen, and 1 more author

   IEEE Journal of Biomedical and Health Informatics, 2018

   URL

2017

1. Exploring the similarity of medical imaging classification problems

   Veronika Cheplygina, Pim Moeskops, Mitko Veta, Behdad Dasht Bozorg, and Josien Pluim

   In Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis (MICCAI LABELS), 2017

   URL

2016

1. Dissimilarity-based ensembles for multiple instance learning

   Veronika Cheplygina, David M. J. Tax, and Marco Loog

   IEEE Transactions on Neural Networks and Learning Systems, 2016

   DOI URL

2015

1. Characterizing Multiple Instance Datasets

   Veronika Cheplygina and David M J Tax

   In Similarity-Based Pattern Recognition, 2015

   Abs DOI URL

   In many pattern recognition problems, a single feature vector is not sufficient to describe an object. In multiple instance learning (MIL), objects are represented by sets (bags) of feature vectors (instances). This requires an adaptation of standard supervised classifiers in order to train and evaluate on these bags of instances. Like for supervised classification, several benchmark datasets and numerous classifiers are available for MIL. When performing a comparison of different MIL classifiers, it is important to understand the differences of the datasets, used in the comparison. Seemingly different (based on factors such as dimensionality) datasets may elicit very similar behaviour in classifiers, and vice versa. This has implications for what kind of conclusions may be drawn from the comparison results. We aim to give an overview of the variability of available benchmark datasets and some popular MIL classifiers. We use a dataset dissimilarity measure, based on the differences between the ROC-curves obtained by different classifiers, and embed this dataset dissimilarity matrix into a low-dimensional space. Our results show that conceptually similar datasets can behave very differently. We therefore recommend examining such dataset characteristics when making comparisons between existing and new MIL classifiers. Data and other resources are available at http://www.miproblems.org.

2. On classification with bags, groups and sets

   Veronika Cheplygina, David M J Tax, and Marco Loog

   Pattern Recognition Letters, 2015

   Abs URL

   Many classification problems can be difficult to formulate directly in terms of the traditional supervised setting, where both training and test samples are individual feature vectors. There are cases in which samples are better described by sets of feature vectors, that labels are only available for sets rather than individual samples, or, if individual labels are available, that these are not independent. To better deal with such problems, several extensions of supervised learning have been proposed, where either training and/or test objects are sets of feature vectors. However, having been proposed rather independently of each other, their mutual similarities and differences have hitherto not been mapped out. In this work, we provide an overview of such learning scenarios, propose a taxonomy to illustrate the relationships between them, and discuss directions for further research in these areas.

3. Label stability in multiple instance learning

   V. Cheplygina, L. Sørensen, D. M. J. Tax, M. Bruijne, and M. Loog

   In Medical Imaging Computing and Computer Assisted Intervention (MICCAI), 2015

   URL
4. Multiple instance learning with bag dissimilarities

   Veronika Cheplygina, David M. J. Tax, and Marco Loog

   Pattern Recognition, 2015

   Abs URL

   Multiple instance learning (MIL) is concerned with learning from sets (bags) of objects (instances), where the individual instance labels are ambiguous. In this setting, supervised learning cannot be applied directly. Often, specialized MIL methods learn by making additional assumptions about the relationship of the bag labels and instance labels. Such assumptions may fit a particular dataset, but do not generalize to the whole range of MIL problems. Other MIL methods shift the focus of assumptions from the labels to the overall (dis)similarity of bags, and therefore learn from bags directly. We propose to represent each bag by a vector of its dissimilarities to other bags in the training set, and treat these dissimilarities as a feature representation. We show several alternatives to define a dissimilarity between bags and discuss which definitions are more suitable for particular MIL problems. The experimental results show that the proposed approach is computationally inexpensive, yet very competitive with state-of-the-art algorithms on a wide range of MIL datasets.