Meta-repository of Screening Mammography Classifiers

Compare machine learning performance across classifiers and datasets.

We are sharing an open-source meta-repository for deep learning screening mammography classifiers. The meta-repository gives researchers straightforward tools to compare how a given model performs across datasets and how various models perform on the same set of data, be it public or private.

Overview diagram of the meta-repository of screening mammography classifiers.
Meta-repository users can evaluate state-of-the-art machine learning models on any mammography dataset and compute metrics relevant to breast cancer exam classification.

The growing number of machine learning (ML) models for screening mammography has made implementation and comparison of their performance on different datasets challenging. We are sharing the meta-repository in order to address this challenge, promote standardization, and facilitate reproducibility in ML research on detection and diagnosis of breast cancer.

At the time of publication, the meta-repository contains five state-of-the-art machine learning models for breast cancer exam classification. Researchers can use their own data representative of particular target populations to compare how different models perform.

Developers can submit their own models to the meta-repository in order to make their work available to the machine learning research community and to compare the performance of their models with that of other models.

The meta-repository is a cross-platform solution and works on various hardware setups. Containerization of deep learning models with Docker allows users to generate predictions on most machines, with or without GPU acceleration.

Related Preprint

Stadnick B, Witowski J, Rajiv V, Chłędowski J, Shamout FE, Cho K, and Geras KJ.
Meta-repository of screening mammography classifiers.
arXiv. Preprint posted online August 10, 2021. arXiv:2108.04800 [cs.LG]

Please cite this work if you are using the meta-repository in your research.


Wu N, Phang J, Park J, Shen Y, Huang Z, Zorin M, Jastrzębski S, Févry T, Katsnelson J, Kim E, Wolfson S, Parikh U, Gaddam S, Lin LLY, Ho K, Weinstein JD, Reig B, Gao Y, Toth H, Pysarenko K, Lewin A, Lee J, Airola K, Mema E, Chung S, Hwang E, Samreen N, Kim SG, Heacock L, Moy L, Cho K, Geras KJ.
Neural Networks Improve Radiologists’ Performance in Breast Cancer Screening.
IEEE Trans Med Imaging. 2020 Apr;39(4):1184-1194. doi: 10.1109/TMI.2019.2945514

Shen Y, Wu N, Phang J, Park J, Liu K, Tyagi S, Heacock L, Kim SG, Moy L, Cho K, Geras KJ.
An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization.
Med Image Anal. 2021 Feb;68:101908. doi: 10.1016/

Liu K, Shen Y, Wu N, Chłędowski J, Fernandez-Granda C, Geras KJ.
Weakly-supervised high-resolution segmentation of mammography images for breast cancer diagnosis.
Proc Mach Learn Res. 2021. Medical Imaging with Deep Learning.


Questions about this resource may be directed to Krzysztof Geras or raised as issues on Github.