What it is

Arises from the addition of synthetic or redundant data samples to a dataset.

Why it occurs

Synthetic data is often offered as a mitigation technique to reduce the bias in that may be present real data sets. However, since synthetic data is typically generated using a procedure or algorithm that itself needs to be trained, the biases inherent in a real data set may creep into the synthetic data. In addition, the procedures used for generating synthetic data may introduce additional biases.

Why we care

One should not assume that synthetic data can solve data bias. One should probe whether synthetic data is helpful in mitigating bias for each individual task.

A real-life example relevant to MIDRC

if the synthetic data generation is based on too few samples, and those samples all contain a particular type of bias, then the bias will creep into the generated synthetic data. A hypothetical example relevant to MIDRC would be the generation of synthetic pediatric chest radiographs from a sample of actual chest radiographs of healthy children while the intended population is COVID+ pediatric patients. 

How to mitigate it

Carefully consider (e.g., with a list of known of potential biases) the kinds of biases synthetic data generation may be adding to a small or already-biased data set.

Summary

Synthetic data is often offered as a mitigation technique to reduce the bias in that may be present real data, but do not assume that synthetic data generation can automatically remedy biased data collection.

References

1. https://www.nature.com/articles/s41551-021-00751-8.pdf: "Synthetic data in machine learning for medicine and healthcare".

Belongs under

Data collection biases

What it is

(i) When specific groups of user populations are excluded from training, testing or subsequent analyses, especially if the exclusion conditions for different types of outcomes (e.g., for actually positive and actually negative cases in a classification task). (ii) It can also occur when you exclude some features from the dataset used to train an AI model.

Why it occurs

(i) Population groups may be systematically excluded and hence be underrepresented in the training of an AI model. This is especially important since it can result in distortions when patients with a particular condition are excluded differently for different types of outcomes. For example, for a classification task, patients with a certain condition are excluded when the case is negative but not excluded when the case is positive. (ii) People mistakenly think that some features are irrelevant, so a subset of the data is systematically excluded due to a particular attribute. 

Why we care

The model may not be able to perform accurately in a subset of population that's excluded from the data collection. This may result in biased algorithm training if exclusion bias happens in the training data, or it may result in incorrect conclusions if exclusion bias happens in the test data.

A real-life example relevant to MIDRC

Hypothetically, portable x-ray images may be excluded from the data set for a certain task. Since patient imaged with portable x-ray are likely to be sicker, this may result in distortion of the patient population with more severe disease.

How to mitigate it

(i) Ensure that the training sample is representative of the population the AI model is intended to be applied to. Make sure that exclusion conditions are carefully examined. (ii) Perform sufficient analysis before discarding features from training dataset. Addition of random data to training sample. 

Summary

If exclusions are not carefully described and justified, severe bias may occur both in model training and testing. Sample selection bias caused by choosing non-random data, where a subset of the data is systematically excluded. 

References

1. Horwitz RI, Feinstein AR. Exclusion bias and the false relationship of reserpine and breast cancer. Arch Intern Med 1985;145:1873–5.

2. M. Delgado-Rodriguez, Bias, Journal of Epidemiology & Community Health. 58 (2004) 635–641

Belongs under

Data collection bias, and data preparation and annotations biases

What it is

Popularity bias is often noted with AI systems that recommend movies, etc. Popularity bias may causes changes in the data collected based on the trends of the day. Such could happen in medical imaging AI.

Why it occurs

Humans make medical decisions based on both facts and perceptions.

Why we care

if findings and statistics are skewed based on current trends, long term analyses (without such temporal corrections) may give biased findings.

A real-life example relevant to MIDRC

An example might be in women deciding to be imaged (or not) for a screening mammogram. Newspaper articles might sway a woman in her decision making.

How it is measured

See ref [1].

How to mitigate it

Ref [1]: "The effect of having few data items responsible for the majority of ratings in a Collaborative Filtering recommendation, and the complement of having majority of items responsible for few ratings given by the users, are usually referred as popularity bias. The effect is known as reflecting the preference of users for popular items, but also as a consequence of methods and metrics normally applied by these systems. Variational Autoencoders (VAE) are considered today the state-of-the-art for collaborative filtering recommenders, and can handle big and sparse data entries with robustness and high accuracy. A methodology is proposed here for characterizing the popularity bias in Movielens and Netflix datasets, and when applying VAE for generating recommendations based on them. As a first step, the long tail model is applied for segmenting items and users in three different classes (Short Head, Medium Tail and Long Tail), depending on the proportion of interactions they are associated with. In addition, a real recommendation scenario is presented for measuring the proportion of unpopular items appearing among the suggestions provided by VAE. We consider characterizing the popularity in details as a very first step for providing recommenders with the desired serendipity effect, and expanding the knowledge of these systems about new and unpopular items with few ratings."

Summary

Popularity bias occurs when current trends influence patients' decision making on medical imaging, which then subsequently affect data collection. 

References

1. https://www.semanticscholar.org/paper/On-Measuring-Popularity-Bias-in-Collaborative-Data-Borges-Stefanidis/05109d22ad30e4b5f11f0b72c762ee19374cdd6a

Belongs under

Data collection biases

What it is

Arises when statistics, demographics, and user characteristics differ between the original target population and the user population represented in the actual dataset or platform.

Why it occurs

When the characteristics of a training data population are different than the characteristics of a testing data population. The characteristics may include biological differences, demographic differences, social differences (such as the impacts of socioeconomic differences to access to health care), or technical differences in image acquisition that correlate to demographic difference.

Why we care

Algorithms can be overtrained to one group, reducing their ability to be applied in a useful way to other groups. This impacts decision-making if drawn from the algorithms developed with population bias, for example increasing the numbers of false positive/false negative classification decisions. It may also foster a lack of confidence in the algorithm's utility for some populations and discourage further development and resourcing.

A real-life example relevant to MIDRC

Some socioeconomic groups may not have sought care for COVID-19 as soon as some others, and this may have made their cases more advanced before imaging was conducted. Some populations may not have had CT scanning as a part of their care, and thus their data may not be included in training models, when otherwise this would have made the dataset more diverse. More significant and serious outcomes from COVID-19 have been observed in some age ranges, races, and/or ethnicities than others.

How to mitigate it

Cross-population modeling [2]. 

Summary

Population bias in medical imaging occurs when the characteristics of a population from which data is used for training an algorithm are different than the characteristics of a population for which data is used for testing or decision making-from an algorithm. The characteristics may include biological differences, demographic differences, social differences (such as the impacts of socioeconomic differences in access to health care), or technical differences in image acquisition that correlate to demographic difference. 

References

1. https://www.nature.com/articles/s41467-020-17971-2 (interesting example of attempting to make a generalizable AI model for diagnosing COVID-19 from CT images.)

2. https://link.springer.com/content/pdf/10.1007/ s10916-020-01562-1.pdf

3. https://ieeexplore.ieee.org/abstract/document/ 9182874.

4. Obuchowski et al. define "population bias" and "population precision" in "Quantitative imaging biomarkers: A review of statistical methods for computer algorithm comparisons" (2015, Statistical Methods in Medical Research).

Belongs under

Data collection, model development, and model evaluation biases

What it is

The patient data used for training/tuning/testing a machine learning algorithm are not representative of the patient population to which the algorithm is intended to be applied.

Why it occurs

Data collection is by convenience and availability, without sufficiently considering the clinical task and patient population. It may occur in at least two ways (a) shortcut over signal problem: the algorithm learns from confounding factors instead of the pathology of interest; (b) distribution shift problem: the algorithm learns from the pathology of interest, but because of distribution shift, the performance on the intended population has systematic deviations from estimates on study samples.

Why we care

Such data collection bias would result in performance estimates that are not generalizable to the intended patient population of the AI/ML algorithm, thereby leaving effectiveness and trustworthiness of AI/ML questionable. 

A real-life example relevant to MIDRC

Data missingness correlates with disease severity; single institution as data source for a particular task.

How to mitigate it

Collection from multiple sources. Curation by matching key characteristics of the intended patient population.

Summary

The patient data used or training/tuning/testing a machine learning algorithm are not representative of the patient population to which the algorithm is intended to be applied. Such bias often arises from collection of data by convenience and availability, without sufficiently considering the clinical task and patient population. 

References

1. DeGrave, A.J., Janizek, J.D. & Lee, SI. AI for radiographic COVID-19 detection selects shortcuts over signal. Nat Mach Intell 3, 610–619 (2021). https://doi.org/10.1038/s42256-021-00338-7

Belongs under

Data collection and model evaluation biases