Science Policy
day one project

Combating Bias in Medical Innovation

04.26.22 | 17 min read | Text by Grace Wickerson

There is a crisis within healthcare technology research and development, wherein historically marginalized groups are under-researched in preclinical studiesunder-represented in clinical trialsmisunderstood by clinical practitioners, and harmed by biased medical technology. These issues in turn contribute to costly disparities in healthcare outcomes, leading to losses of $93 billion a year in excess medical-care costs, $42 billion a year in lost productivity, and $175 billion a year due to premature deaths. COVID-19 put these disparities into especially sharp focus. In December 2020, pulse oximeters, critical for healthcare monitoring during the pandemic, were shown to be much less accurate in patients with darker skin, thereby putting those patients at a greater risk of organ damage. The Food and Drug Administration (FDA) responded by issuing a safety communication, but not with any changes to regulation of pulse oximeters. 

Especially for an administration that has embedded equity throughout its policy agenda, this situation is unacceptable. The Biden-Harris Administration must act to address bias in medical technology at the development, testing and regulation, and market-deployment and evaluation phases. This will require coordinated effort across multiple agencies. In the development phase, science-funding agencies should crack down on federally funded studies that do not conduct mandatory subgroup analysis for diverse populations. Funding agencies should also expand funding for under-resourced research areas. In the testing and regulation phase, the FDA should raise the threshold for evaluation of medical technologies, make diversity requirements binding, and expand data-auditing processes. In the market-deployment and evaluation phases, the FDA should strengthen reporting mechanisms for adverse outcomes, the Federal Trade Commission (FTC) should require impact assessments of deployed technologies, and the Agency for Healthcare Research and Quality (AHRQ) should identify technologies that could address healthcare disparities.

Challenge and Opportunity

Bias is regrettably endemic in medical innovation. Drugs are incorrectly dosed to people assigned female at birth due to historical exclusion of women from clinical trials. Medical algorithms make healthcare decisions based on biased health dataclinically disputed race-based corrections, and/or model choices that exacerbate healthcare disparitiesMuch medical equipment is not accessible, thus violating the Americans with Disabilities Act. Biased studies, technology, and equipment inevitably produce disparate outcomes in U.S. healthcare.

The problem of bias in medical innovation manifests in multiple ways: cutting across technological sectors in clinical trials, pervading the commercialization pipeline, and impeding equitable access to critical healthcare advances.

Bias in medical innovation cuts across technology sectors

The 1993 National Institutes of Health (NIH) Revitalization Act required federally funded clinical studies to (i) include women and racial minorities as participants, and (ii) break down results by sex and race or ethnicity. Yet a 2019 study found that only 13.4% of NIH-funded trials performed the mandatory subgroup analysis. Moreover, the increasing share of industry-funded studies are not subject to Revitalization Act mandates — they are only governed by non-binding FDA recommendations for clinical-trial diversity. These studies frequently fail to report differences in outcomes by patient population as a result. The resulting disparities in clinical-trial representation are stark: African Americans represent 12% of the U.S. population but only 5% of clinical-trial participants, Hispanics make up 16% of the population but only 1% of clinical trial participants, and sex distribution in some trials is 67% male. Finally, many medical technologies approved prior to 1993 have not been reassessed for potential bias. One outcome of such inequitable representation is evident in drug dosing protocols: sex-aware prescribing guidelines exist for only a third of all drugs.

Bias in U.S. medical innovation — perpetuated by weak or weakly enforced federal regulations — extends beyond clinical trials. As explained below, bias pervades medical algorithms, medical devices, and the pharmaceutical sector as well. 


Regulation of medical algorithms varies based on end application, as defined in the 21st Century Cures Act. Only algorithms that (i) acquire and analyze medical data and (ii) could have adverse outcomes are subject to FDA regulation. Thus, clinical decision-support software is not regulated even though these technologies make important clinical decisions in 90% of U.S. hospitals

Even when a medical algorithm is regulated, regulation may occur through relatively permissive de novo pathways and 510(k) pathways. A de novo pathway is used for novel devices determined to be low to moderate risk, and thus subject to a lower burden of proof with respect to safety and equity. A 510(k) pathway can be used to approve a medical device exhibiting “substantial equivalence” to a previously approved device, i.e., it has the same intended use and/or same technological features. Different technical features can be approved so long as there are not questions raised around safety and effectiveness.

Medical devices approved through de novo pathways can be used as predicates for approval of devices through 510(k) pathways. Moreover, a device approved through a 510(k) pathway can remain on the market even if its predicate device was recalled. Widespread use of 510(k) approval pathways has generated a “collapsing building” phenomenon, wherein many technologies currently in use are based on failed predecessors. Indeed, 97% of devices recalled between 2008 to 2017 were approved via 510(k) clearance. 

Even more alarming is evidence showing that machine learning can further entrench medical inequities. Because machine learning medical algorithms are powered by data from past medical decision-making, which is rife with human error, these algorithms can perpetuate racial, gender, and economic bias. Even algorithms demonstrated to be unbiased at the time of approval can evolve in biased ways over time, with little to no oversight from the FDA. As technological innovation progresses, an intentional focus on this problem will be required.

Finally, there is not a list of approved medical algorithms on the market, making it difficult for researchers to assess them for bias.

Medical devices

Currently, the Medical Device User Fee Act requires the FDA to consider the least burdensome appropriate means for manufacturers to demonstrate the effectiveness of a medical device or to demonstrate a device’s substantial equivalence. This requirement was reinforced by the 21st Century Cures Act, which also designated a category for “breakthrough devices” subject to far less-stringent data requirements. Such legislation shifts the burden of clinical data collection to physicians and researchers, who might discover bias years after FDA approval. This legislation also makes it difficult to require assessments on the differential impacts of technology.

Like medical algorithms, many medical devices are approved through 510(k) exemptions or de novopathways. The FDA has taken steps since 2018 to increase requirements for 510(k) approval and ensure that Class III (high-risk) medical devices are subject to rigorous pre-market approval, but problems posed by equivalence and limited diversity requirements remain. 


The 1993 Revitalization Act strictly governs clinical trials for pharmaceuticals and does not make recommendations for adequate sex or genetic diversity in preclinical research. The results are that a disproportionately high number of male animals are used in research and that only 5% of cell lines used for pharmaceutical research are of African descent. Programs like All of Us, an effort to build diverse health databases through data collection, are promising steps towards improving equity and representation in pharmaceutical research and development (R&D). But stronger enforcement is needed to ensure that preclinical data (which informs function in clinical trials) reflects the diversity of our nation. 

Bias in medical innovation exists throughout the commercialization pipeline

Bias occurs not only in multiple medical innovation sectors, but also across the development, testing and regulation, and market-deployment and evaluation phases of the medical innovation pipeline. This can be understood through the example of pulse oximeters.


Pulse oximetry was developed by Biox and given FDA approval in 1980. The technology works by shining a light through the skin and measuring the difference in light absorbance to estimate arterial oxygen saturation. Melanin absorbs visual and infrared light and will interfere at all wavelengths. No algorithm has yet been developed to account for melanin attenuation. Hence pulse oximeter calibration data does not accurately reflect Black patients.

Testing and regulation

The first pulse oximeter was approved by the FDA at a time when clinical trials did not require gender and racial diversity. Thus, the foundational, 1980s-era pulse oximeter technology upon which subsequent 510(k) clearance for pulse oximeters has been granted is one that was tested almost exclusively on white, male patient populations.

With the 510(k) clearance, only 10 people are required in a study of any new pulse oximeter’s efficacy. The FDA states that pulse oximetry study populations should have a range of skin pigmentations and must include at least two darkly pigmented individuals or 15% of the participant pool, whichever is larger. But the FDA does not provide an objective standard for “darkly pigmented”. Moreover, this requirement (i) does not have the statistical power necessary to detect differences between demographic groups, and (i) does not represent the composition of the U.S. population. Finally, FDA guidance is silent on how pulse oximetry technology should be calibrated — it does not, for instance, specifically recommend studies on melanin interference.

Market deployment and evaluation

To clinical practitioners, pulse oximeters are a metaphorical “black box”, with oxygenation calculations hidden by proprietary algorithms. When errors or biases occur in oximeter data (if they are even noticed), the practitioner may blame the patient for their lifestyle rather than the technology used for assessment. This in turn leads to worse clinical outcomes for patients with darker skin tones, as they are at greater risk of becoming sicker before receiving care. The problem is exacerbated by the fact that clinicians who use oximeter technology for the first time (as was the case during COVID-19) generally are not trained to spot factors that cause inaccurate measurements. This leads to underreporting of adverse events to the FDA — which is already a problem due to the voluntary nature of adverse-event reporting. When problems are ultimately identified during market deployment and evaluation of a given technology, government can be slow to respond. The pulse oximeter’s limitations in monitoring oxygenation levels across diverse skin tones was identified as early as the 1990s. 31 years later, despite repeated follow-up studies indicating biases, no manufacturer has incorporated skin-tone-adjusted calibration algorithms into pulse oximeters. It required the large Sjoding study, and the media coverage it garnered, for the FDA to issue a safety communication. Even then, the safety communication has not been followed with any additional regulatory action. 

Inequitable access to medical innovation represents a form of bias

Americans face wildly different levels of access to new medical innovations. As many new innovations have high cost points, these devices exist outside the price range of many smaller healthcare institutions and/or federally funded healthcare services, including Veterans Affairs, health centers, and the Indian Health Service. Emerging care-delivery strategies might not be covered by Medicare and Medicaid, meaning that patients under those systems cannot access the most cutting-edge treatments. Finally, the shift to digital health in response to COVID-19 has compromised access to healthcare in rural communities without reliable broadband access. 

Finally, the Advanced Research Projects Agency for Health (ARPA-H) has a commitment to have all programs and projects consider equity in their design. To fulfill ARPA-H’s commitment, there is a need for action across the federal government to ensure that medical technologies are developed fairly, tested with rigor, deployed safely, and made affordable and accessible to everyone.

Plan of Action

The Biden-Harris Administration should launch “Healthcare Technology for All Americans” (HTAA), a government-wide initiative to address systemic inequities in U.S. healthcare wrought by biased medical technology. Through a comprehensive approach that addresses bias in all medical sectors, at all stages of the commercialization pipeline, and in all geographies, the initiative will strive to ensure unbiased, equitable care delivery across the entire medical-innovation ecosystem. HTAA should be a joint mandate of Health and Human Services (HHS) and the Office of Science Technology and Policy (OSTP) to work with federal agencies on priorities of health equity, and initiative leadership should sit at both HHS and OSTP. 

This initiative will require involvement of multiple federal agencies, as summarized in the table below. Additional detail is provided in the subsequent sections describing how the federal government can mitigate bias in the development phase; testing, regulation, and approval phases; and market deployment and evaluation phases.

Three guiding principles should underlie the initiative:

  1. Equity should drive action. Actions should seek to improve the health of those who have been historically excluded from medical research and development. We should design standards that repair past exclusion and prevent future exclusion. 
  2. Coordination and cooperation are necessary. The executive and legislative branches must collaborate to address the full scope of the problem of bias in medical technology, from federal processes to new regulations. Legislative leadership should task the Government Accountability Office (GAO) to engage in ongoing assessment of progress towards the goal of achieving equity in medical innovation.
  3. Transparent, evidence-based decision making is paramount. There is abundant peer-reviewed literature that examines bias in drugs, devices, and algorithms used in healthcare settings — this literature should form the basis of an equity-driven approach to medical innovation. Gaps in evidence should be focused on through deployed research funding. Moreover, as algorithms become ubiquitous in medicine, every effort should be made to ensure that these algorithms are trained on representative data of those experiencing a given healthcare condition.

Addressing bias at the development phase

The following actions should be taken to address bias in medical technology at the innovation phase:

Addressing bias at the testing, regulation, and approval phases

The following actions should be taken to address bias in medical innovation at the testing, regulation, and approval phases:

Addressing bias at the market deployment and evaluation phases 

A comprehensive road map is needed

In January 2021, Senators Elizabeth Warren, Cory Booker, and Ron Wyden called for an FDA review of pulse oximetry measurements and their skin tone bias, citing the lack of understanding about clinical outcomes of this bias in their call to action. The GAO should go a step beyond this call to action and conduct a comprehensive investigation of “black box” medical technologies utilizing algorithms that are not transparent to end users, medical providers, and patients. The investigation should inform a national strategic plan for equity and inclusion in medical innovation that relies heavily on algorithmic decision-making. The plan should include identification of noteworthy medical algorithms exacerbating inequities, creation of enforceable regulatory standards, development of new sources of research funding to address knowledge gaps, development of enforcement mechanisms for bias reporting, and ongoing assessment of equity goals.

Timeline for action

Realizing HTAA will require mobilization of federal funding, introduction of regulation and legislation, and coordination of stakeholders from federal agencies, industry, healthcare providers, and researchers around a common goal of mitigating bias in medical technology. Such an initiative will be a multi-year undertaking and require funding to enact R&D expenditures, expand data capacity, assess enforcement impacts, create educational materials, and deploy personnel to staff all the above.

Near-term steps that can be taken to launch HTAA include issuing a public request for information, gathering stakeholders, engaging the public and relevant communities in conversation, and preparing a report outlining the roadmap to accomplishing the policies outlined in this memo. 


Medical innovation is central to the delivery of high-quality healthcare in the United States. Ensuring equitable healthcare for all Americans requires ensuring that medical innovation is equitable across all sectors, phases, and geographies. Through a bold and comprehensive initiative, the Biden-Harris Administration can ensure that our nation continues leading the world in medical innovation while crafting a future where healthcare delivery works for all.

Frequently Asked Questions
1. How will the success of HTAA be evaluated?

HTAA will be successful when medical policies, projects, and technologies yield equitable health care access, treatment, and outcomes. For instance, success would yield the following outcomes:

  1. Representation in preclinical and clinical research equivalent to the incidence of a studied condition in the general population.

  2. Research on a disease condition funded equally per affected patient.

  3. Existence of data for all populations facing a given disease condition.

  4. Medical algorithms that have equal efficacy across subgroup populations.

  5. Technologies that work equally well in testing as they do when deployed to the market.

  6. Healthcare technologies made available and affordable to all care facilities.

2. Why does this memo propose an expansive multi-agency effort instead of just targeting the FDA?

Regulation alone cannot close the disparity gap. There are notable gaps in preclinical and clinical research data for women, people of color, and other historically marginalized groups that need to be filled. There are also historical biases encoded in AI/ML decision-making algorithms that need to be studied and rectified. In addition, the FDA’s role is to serve as a safety check on new technologies — the agency has limited oversight over technologies once they are out on the market due to the voluntary nature of adverse reporting mechanisms. This means that agencies like the FTC and CMS need to be mobilized to audit high-risk technologies once they reach the market. Eliminating bias in medical technology is only possible through coordination and cooperation of federal agencies with each other as well as with partners in the medical device industry, the pharmaceutical industry, academic research, and medical care delivery.

3. Why is ARPA-H critical to this effort?

Working together to address the enormous challenge of bias in medical innovation will require communication, coordination, and collaboration. ARPA-H provides the essential platform for these three tasks. As an agency bridging academic research and industry, ARPA-H will focus on developing technologies that address some of the greatest healthcare challenges facing Americans, including inequities existing in healthcare. By committing to consider equity in every project, ARPA-H provides the basis for practice of technological development that is inclusive, responsible, and accountable. ARPA-H’s deep relationships with industry will spur medical device companies to align with ARPA-H’s processes.

4. Why create a new initiative when we have standing offices like the Office of Minority Health (OMH) focused on health equity?

Offices like the OMH do necessary work in identifying disparities in care and pointing out solutions. For example, the call for digital infrastructure improvements to improve care access for vulnerable populations has been echoed by OMH. But these offices lack the ability to operationalize multi-agency collaborations needed to address cross-cutting challenges related to medical bias. A new initiative led by the White House      in close partnership with HHS leadership is needed to ensure that the broad scope of the plan outlined in this memo is actualized.

5. What challenges might the administration encounter from industry in launching this initiative?

A significant focus of the medical device and pharmaceutical industries is reducing      the time to market for new medical devices and drugs. Imposing additional requirements for subgroup analysis and equitable use as part of the approval process could work against this objective. On the other hand, ensuring equitable use during the development and approval stages of commercialization will ultimately be less costly than dealing with a future recall or a loss of Medicare or Medicaid eligibility if inequitable outcomes are discovered. FAR regulation can also be employed to incentivize companies to adhere to equity standards in order to receive federal contracts.

6. How can the Administration build the bipartisan support necessary to secure the funding for this initiative?

Healthcare disparities exist in every state in America and are costing billions a year in economic growth. Some of the most vulnerable people live in rural areas, where they are less likely to receive high-quality care because costs of new medical technologies are too high for the federally qualified health centers that serve one in five rural residents as well as rural hospitals. Furthermore, during continued use, a biased device creates adverse healthcare outcomes that cost taxpayers money. A technology functioning poorly due to bias can be expensive to replace. It is economically imperative to ensure technology works as expected, as it leads to more effective healthcare and thus healthier people.