Skip to main content

Rethinking when and how to use race appropriately in care delivery

Bias in health care delivery can be addressed through a system-level approach to data, algorithms, and technologies

Executive summary

 

As health care systems strive to decrease disparities and improve health equity, there have been increasing calls to address discrimination and bias in health care delivery.1 Lack of data standards makes addressing system-level biases much more challenging. Data issues include collecting data on race and ethnicity, the misuse of racial and ethnic data to inform treatment and diagnosis, and algorithms that inappropriately account for race factors based on biased data. Technologies can further exacerbate these biases. If these systemic issues aren’t sufficiently addressed, inequities will likely continue to widen, possibly at exponential rates as new technologies are applied in health care. 

COVID-19 has highlighted stark health disparities in the United States, particularly along the lines of race and ethnicity. For example, individuals who identify as American Indian, Alaska Native (AIAN), Latinx, or Black, are more than twice as likely to be hospitalized from COVID-19 compared with white people, and almost twice as likely to die.2 While COVID-19 research is ongoing, initial studies show that racially and ethnically diverse individuals aren’t more susceptible to the virus and they don’t face worse health outcomes because of their race. Instead, poor outcomes can be attributed to factors associated with racism such as decreased access to care, living in multigenerational homes and crowded conditions, working in high-exposure environments, and the direct impact of discrimination, among many others.3 

Although it’s often used to describe the prevalence of disease, race is not an underlying cause of health disparities. Racism is. Health disparities are evidence of systemic bias, deep inequities in the nonmedical drivers of health (DOH), and structural flaws in the health system. And these inequities affect both individual and community health and well-being and can be compounded through systemic biases in clinical algorithms and technologies. 

Research methodology: To better understand the steps stakeholders are taking to improve health equity and address bias in their data, diagnostic algorithms, and technologies, the Deloitte Center for Health Solutions (DCHS) interviewed 19 industry experts in health equity. DCHS also conducted an extensive secondary literature review to understand the strategies that health systems are deploying and the barriers they’re overcoming. 

Findings: Interviewed experts expressed that systemic racial bias in medicine stems from the historically incorrect concept of race which was originally developed as a system of hierarchical human categorization. This led to the notion that white people were superior based on race alone. Medical education and clinical guidelines have unintentionally continued to reflect the antiquated notion that race is a biologically valid distinction among individuals, rather than socially constructed. While there are many types of bias, including gender, disability, sexual orientation, and language, to name a few—and the intersections between these groups (e.g., a Black woman whose first language isn’t English) increase the risk for bias—our research focused specifically on race and ethnicity. 

Some health systems have already begun to address these systemic biases in clinical care through implicit bias training and changes in medical education. These hospitals are driven by their missions as well as the opportunity to improve outcomes and quality, increase profit margins, rebuild patient trust, respond to policymakers, and embed this work within their environmental, social, and governance (ESG) initiatives.

Understanding that race has been socially constructed and has no basis in biological differences is another way health systems can activate health equity in their communities. Interviewed experts expressed the need to address this issue holistically at all levels of care delivery. Health systems can take the following approaches to the insertion of biologic race in medicine and biases that stem from it by:

  • Implementing strategies for data granularity and standardization: Developing standards for data collection can help health systems better understand their patient populations and the health challenges that need to be addressed. Health systems should consider expanding the types of data they collect to include race, ethnicity, preferred language, and DOH. In addition to claims data, health systems can use new datasets like employment data, and leverage nontraditional and community partnerships.
  • Developing metrics for proper data collection and use: In addition to establishing data standards, developing measures and metrics for proper data collection and use is vital. Health systems should consider educating and training providers about why this data is important to collect, how to talk to patients about the importance of sharing this data in a culturally humble and empathetic way, and how to develop scorecards and health equity indicators (HEIs). Understanding when to use social race and ethnicity data, and when not to—even when it’s available—will likely be key to understanding the root causes of health and how to determine an appropriate care plan (this also applies to areas outside of direct care delivery).
  • Reevaluating clinical algorithms: Reexamining long standing clinical algorithms, including care pathways and workflows, can help health care systems ensure all patients receive the care they need. Health systems should consider forming designated teams to evaluate algorithms and assess which clinical algorithms are currently being used in their facilities, how race is used in the algorithm or calculation, and whether race is justified. Scrutinizing existing practices in a new light can ensure that  the intention of using said racial adjustment is consistent with the desired impact (i.e. closing racial disparity gaps and improving outcome measures for all); if not, then race should be removed. This approach can help determine what underlying factors are driving differences in health outcomes and which should be included in the algorithm. 
  • Conducting regular audits: As the use of artificial intelligence (AI), medical devices, and other technologies increase, continually testing for bias will be critical to ensuring health disparities aren’t unintentionally exacerbated. Health systems should consider conducting regular audits of their AI systems to check for bias and re-evaluate their current tools and devices by considering if other screening tools should be used, reviewing their vendors, and understanding the diversity of the clinical trials that devices were tested on.

Addressing systemic bias in health care delivery

 

As the COVID-19 pandemic and ongoing social injustices continue to spur health care organizations to address equity, there have been increased calls to address systemic discrimination and bias in health care delivery.4 While many organizations are focused on solving biases that stem from individual clinicians,5 there also is a need to address the systemic biases within the health care ecosystem infrastructure that stem from using race as a factor in medical decision-making. The lack of data standards and accountability around race and ethnicity, the misuse of racial and ethnic data to inform diagnosis and treatment plans, the implementation of algorithms that account for race factors based on biased data, and the use of technologies that exacerbate these issues are examples of systemic biases.

Continuing to ignore and perpetuate these biases could result in increased health care costs, poor quality of care for both individuals and communities, decreased patient trust, and growing disparities in preventable poor health outcomes. Focusing on racism, rather than race, as a determinant of illness is pivotal to activating health equity but will likely require health care systems to rethink when and how to use race appropriately in care delivery.

To learn how organizations are tackling the issue of bias in care delivery and improve health equity in their health care data, algorithms and technologies, the Deloitte Center for Health Solutions (DCHS) interviewed 19 industry experts, including chief diversity, equity, and inclusion officers at health systems, former CEOs of health systems, academics, medical directors, and chief medical information officers. These interviews helped us to better understand the insertion of race in medicine today and the steps stakeholders are taking to improve health equity in their data, diagnostic algorithms, and technologies. DCHS also conducted an extensive secondary literature review to understand the barriers and identify the strategies organizations are deploying in this space. 

Racism has significantly influenced the way the US health care system and nonmedical DOH have been structured, both contributing to negative health outcomes for racially and ethnically diverse individuals. To address racism, health care systems have taken steps to advance diversity in their workforce, prioritize health equity internally, and rebuild consumer trust. One way to rebuild trust and address the DOH is by joining health care ecosystems with community-based organizations.

Show more

For hundreds of years, race has been used to define a specific group of people with genetic and biological differences which has resulted in using race to explain differences in prevalence of disease and health outcomes.6 For example, in the United States, sickle cell disease (SCD) is more prevalent in the Black community compared to other races.7 However, SCD is correlated with an evolutionary adaption to malaria exposure, not the color of one’s skin.8 The Centers for Disease Control and Prevention (CDC) and the American Society of Hematology acknowledge that SCD can affect anyone, regardless of race, including white people, but is more prevalent among certain ancestral lineages like those who came from sub-Saharan Africa; Spanish-speaking regions in the Western Hemisphere (South America, the Caribbean, and Central America); Saudi Arabia; India; and Mediterranean countries such as Turkey, Greece, and Italy.9 The historical underinvestment in research in diseases presumed to only affect members of certain racial groups has contributed to a lack of understanding that diseases like SCD can also affect white people. 

Despite evidence that race isn’t a reliable proxy for genetic differences, race unintentionally is embedded in medical practice and can lead to health disparities.10 The American Medical Association (AMA) cites that that one of the most infamous examples of this is the use of a drug, marketed as BiDil, which is used to treat congestive heart failure in Black people.11 It became the first race-based prescription drug in the United States. However, the study only enrolled men and women who self-identified as Black. BiDil is still marketed as the only medicine specifically targeted for Black patients, although many continue to question this practice. Interviewees noted that differences in comorbidities of Black patients compared to white patients may be one explanation for differences in drug efficacy.

In 2020, the AMA released policies that recognize race as a socially constructed category that differs from ethnicity, genetic ancestry, or biology.12 Furthermore, the AMA states that the practice of accepting race as a biological construct exacerbates health disparities and results in negative health outcomes for racially and ethnically diverse individuals.13 

Show more

There are several examples of bias that stem from the misuse of racial and ethnic data and the insertion of race in algorithms and technologies that inform diagnosis and treatment (see “Appendix,” for more information). Some of the most well-documented examples include:

  • Data bias from the lack of race and ethnicity reporting standards: Health care systems have faced long-standing issues around the collection and use of race and ethnicity data in health care—due to both lack of standards and misconceptions.14 As previously reported by the CDC, race and ethnicity data aren’t available for nearly 40% of people testing positive for COVID-19 or receiving a vaccine.15 Furthermore, our recent report, Addressing the Drivers of Health, also found that only 19% of health system leaders surveyed are measuring outcomes or results from DOH activity, and less than half (38%) are using nomedical data like access to transportation to understand their community’s needs.
  • Algorithmic bias in kidney function equations: Two common kidney function equations (the Modification of Diet in Renal Disease [MDRD] equation and the Chronic Kidney Disease Epidemiology Collaboration [CKD-EPI] equation) use four factors to estimate glomerular filtration rate (GFR): age, gender, race, and levels of creatinine.16 However, the tools can only categorize the race of the patient as Black or not Black. When the calculations were created, Black people were falsely perceived to have higher muscle mass on average, which often resulted in a higher kidney function score and triggered a delay in necessary treatment.17 One study found that removing the race correction from this calculation increased the prevalence of chronic kidney disease from approximately 15% to 18%. Additionally, 29% of those with existing chronic kidney disease were reclassified to more severe stages of the disease.18 The study concluded that leaving the race correction in is more likely to create additional harm to the patient than removing it. 
  • Technology bias in pulse oximeters: There are some medical devices, like pulse oximeters, that don’t work as well on racially and ethnically diverse individuals. During the COVID-19 pandemic, pulse oximeters have been crucial in determining oxygen level in patients. However, these devices measure how much light is transmitted through skin and inaccuracies have been identified when used on racially diverse patients. Studies found that Black patients had nearly three times the frequency of occult hypoxemia, or lower oxygen levels, compared to white patients19 and increased incidence of occult hypoxemia in Black infants when using pulse oximeters.20

“Notions of biological race-based medicine hurts us all, inclusive of historically advantaged social, political, and demographic groups. It hurts us all in the ways in which it perverts our understanding of truth and the associated structural drivers of inequity. It forces providers to racially discriminate in their care and provide separate and unequal treatments based on the flawed notion of biological racial differences. Failing to realize the unintended consequences of propagating racial biology far outweighs whatever supposed benefit the racial biologists in medicine claim but have yet been able to prove. And at its worst, there is the fact that many of these racial adjustments allocate scarce healthcare resources away from diverse patient populations, likely contributing to the very same healthcare inequities we are all trying to overcome.”

- Dr. Louis Hart, medical director of health equity, Yale New Haven Health System

Why does systemic bias exist?

 

Interviewed experts expressed that systemic bias in medicine stems from the historically incorrect concept of race which was originally developed as a system of hierarchical human categorization. While there’s evidence that race has been used to describe differences in humans as early as the 1500s, the Western concept of race that we see today was conceptualized by philosophers and anthropologists in the 1700s and early 1800s who erroneously concluded there are distinct biological differences between humankind that could be classified based on race.21

  • Carl Linnaeus was an 18th-century naturalist who was one of the first scientists to classify humankind into distinct racial categories by skin color and attributed inherently positive traits to lighter-skinned Europeans and negative traits to darker-skinned African Americans.22
  • Johan Blumenbach was a German scientist who further classified humankind based on skull size and race and refuted the notion that Ethiopians were inferior to other races.23 He also coined the term Caucasian in 1795 and claimed it to be the “original race.”24
  • Samuel Morton was an American anthropologist that studied skull size differences based on race. He concluded that Black people had smaller brains than white people, and therefore were less intelligent than white people.25

This intersection of race and biological inferiority was used to justify chattel slavery and spurred medical atrocities such as Henrietta Lacks’ cells being used without consent, the forced sterilization of Black, Latinx, and Indigenous women, the US Public Health Study at Tuskegee, and many others.26 It also fueled the development of race differences in clinical handbooks that are still used today. For example, in the 1800s, Samuel Cartwright quantified a 20% deficiency between lung function of Black and white people, and established race as the prime culprit for these differences.27 This led to the inclusion of race differences in clinical handbooks, without accounting for other social factors such as manual labor. Racial adjustment in lung function measurement is still used ubiquitously today and has normalized lung pathology (lower levels of expected lung function) in non-white patient populations. 

As recently as 1999, a study using the National Health and Nutrition Examination Survey to assess the health and nutrition status of Americans,28 concluded that lung capacity differences between Mexican Americans, Black Americans, and white Americans are potentially related to body build.29 After this study, race-adjustment was embedded into the software of modern spirometers, despite studies showing that removing race correction from pulmonary function tests results in higher prevalence and severity of lung disease for Black individuals.30

Despite medical and scientific consensus that race was socially constructed,31 our interviewees described aspects of medical education that continue to reflect the antiquated notion that race is a biologically valid distinction among individuals. A study revealed that medical professors frequently linked diseases to specific racial groups (e.g., sickle cell disease, hypertension, diabetes, obesity, neonatal jaundice).32 Other examples include the misrepresentation, or inappropriate use of clinical nomenclature that can create bias and exacerbate disparities.

Show more

“It became clear that you really couldn't classify humans into consistent groups using either physical traits or gene frequencies. By the mid-20th century, amongst people who knew better, the idea that humans have biological races was already dismissed. That didn’t become public knowledge at the time and still isn't public knowledge in the 21st century.” 

– Dr. Joseph Graves, professor of biological sciences, North Carolina Agricultural and Technical State University

Health systems are addressing systemic bias

 

Interviewed health executives noted that their organizations were driven to address systemic bias and improve health equity because: 

  • Mission: Organizations realize that as part of their publicly stated mission, it’s their moral obligation to provide standardized care to all patients to improve the health outcomes of individuals and the community. 
  • Outcomes, quality, safety, and cost: Interviewees noted that addressing inequities caused by racism in care delivery reduces overall health care costs and risk to the organization. The goal of every health system is to provide high-quality, safe care that allows people, families, and communities to live their best lives, enhance patient satisfaction, and ensure that the patient experience is positive—while also improving profit margins. 
  • Patient trust: Organizations realize existing mistrust and negative health care experiences among patients related to race and ethnicity exist and are working to rebuild trust and improve patient experiences. 
  • Regulatory push: The Biden administration has acknowledged racial equity as a key priority and the focus can be seen in various initiatives such as the need to improve collection of race and ethnicity data in federal health programs. 
  • Environment, social, and governance (ESG) initiatives: An ESG framework calls for sustainable energy and waste management systems, investments in community health, and demands that leaders embrace diversity and inclusion as essential duties.As more boards of health organizations look to incorporate ESG in their work, reducing systemic bias in health care can support ESG initiatives. 
Show more

Recommendations to address systemic bias

 

Based on interviews and secondary literature reviews, we identified two areas where health systems can reduce systemic bias:

  • Creating strategies for better data collection and accountability to understand racially and ethnically diverse patient populations, and addressing the appropriate use of racial and ethnic data
  • Assessing the use of race in clinical algorithms, AI, and tools

Reduce systemic bias with better data collection and use

 

On a systemic level, data on race and ethnicity is important to understanding who patients are and how to improve their care. It’s also critical in identifying the root causes of health disparities and how they relate to race. However, if data is incorrect, not well-defined or standardized, or isn’t analyzed appropriately, then providers are no better off than if they had no data at all. 

Collecting patient data: Interviewees noted that collecting race, ethnicity, language, and disability (REALD) and sexual orientation and gender identity (SOGI) data can be difficult. Patients may elect to skip the questions on paper forms if they don’t understand the importance of providing the information. They may feel uneasy when asked to answer the question verbally. To avoid this, front office or clinical staff have sometimes made their own best guess at what a patient’s REALD and SOGI are, which often results in incorrect data. Interviewees noted that their staff were sometimes uncomfortable or weren’t trained on how to appropriately ask patients for this information. Some health systems are addressing this systemwide by educating their staff on why this data is important to collect and how to talk to patients about the importance of providing this data in a culturally humble and empathetic way (see sidebar, “Collecting accurate patient information,” for more information). 

"We recognized that we were not collecting race, ethnicity and preferred language accurately… We have spent the last few years with widespread education efforts across all levels on the importance of accurate collection of these data and their link to customized care delivery to improve health outcomes.” 

– Dr. Jennifer Mieres, senior VP, Northwell Health Center for Equity of Care, and chief diversity and inclusion officer

Collecting accurate patient information

 

We Ask Because We Care is an initiative adopted by many health systems (including, NYP Dalio Center for Health Justice, Northwell Health, and Stanford Medicine) that focuses on educating team members and patients about the importance of collecting and using accurate race, ethnicity, preferred language, sexual orientation, and gender identity data.33 From this initiative, one interviewed health system told us they improved informative race by 15%, while it aims to get to 90% data with race and ethnicity identified. 

Show more

Another type of important data that some hospitals have started collecting is patient experience data related to perceptions of bias or discrimination. Interviewees discussed conducting patient surveys or asking patients during the appointment about their experience. One physician uses an everyday discrimination scale adapted to medical settings to understand whether patients felt discriminated against. The responses can help health systems better understand and address areas that are impacting patient experience such as, physical office space, front office waiting area, and toiletries offered to patients. 

“Universally I ask all my patients if they've experienced bias, inequity, or discrimination in our care, or if they have any concerns for the quality and safety in the care they are receiving. It begins to normalize the fact that bias is a key contributor to poorer patient outcomes and that equity must be established as a fundamental pillar of quality and safety systems in health care.” 

– Dr. Louis Hart, medical director of health equity, Yale New Haven Health System 

Setting data standards: Current data categories used by health systems often aren’t defined in a specific enough manner to identify and understand underserved populations.34 For example, only having one umbrella category for Asian or Latinx populations, despite the multiple and distinct differences within these social groups, limits the usefulness of the information. Furthermore, as more data sources are used by health care organizations, ensuring standardization across census, payer, and other data sources, will be important.

Ensuring oversight and accountability of data collection: Currently, interviewees stated that most health systems lack target metrics for collecting data on race and ethnicity, and little oversight is provided to ensure the data inputted into the electronic health record is accurate and valid. Establishing metrics and creating incentives for providers to ensure the data is captured accurately can help address this issue. For example, one hospital set a goal of having 95% complete collection rate of race, ethnicity, language, disability, sexual orientation, and gender identity. Another hospital is working with data scientists to understand “other” or null inputs for race and ethnicity.

Developing standards for data use, collection, and accountability

 

The University of California Los Angeles (UCLA) Health has created equity dashboards to detect variation in patients’ health care quality or outcomes by race, ethnicity, gender identity, age, sexual orientation, and community-level social vulnerability to identify inequities and address them. The dashboards are reviewed every month by UCLA’s equity council, to include the health system’s president and leadership team.35 

Show more

Using racial and ethnic data appropriately: Interviewed experts strongly explained that while racial and ethnic data is necessary to better understand patient populations and address disparities, it’s often unintentionally misrepresented and used inappropriately in clinical care.

A standard part of medical education is learning to quickly make diagnoses and disease stereotyping, or race-based diagnostic bias, is one way to do that.36 And textbooks, curricula, and examinations perpetuate this thinking. Often-cited examples include white people being more likely to have cystic fibrosis and Black people being more likely to have SCD. However, Black people also can have cystic fibrosis and white people can have SCD. But because of disease stereotyping, these patients often are initially misdiagnosed.

Disease stereotyping is harmful.37 As race is a social construct, not a biological one, race can’t be the cause of disease. And while race often is used as a proxy for DOH, it’s not a good proxy.38 Instead of using race, clinicians should use specific DOH and family medical history to determine an individual patient’s health risks. Race, however, does impact health through the racism that patients may or may not experience. DOH often are the biproduct of systemic racism.39

To address this at the systemic level, health systems should consider collecting DOH data and making it available for clinical use. A DCHS survey found that only 19% of health system leaders are measuring outcomes or results from DOH activity, and less than half (38%) are using nonmedical data like access to transportation to understand their community’s needs. One way to address this is to harness real-world and nontraditional data sources like employment or housing data. Additionally, while there are no DOH measures in any federal health care payment or quality programs, the Center for Medicare and Medicaid Services (CMS) is considering measures from the Physicians Foundation focused on food insecurity, housing instability, and transportation.40 CMS also developed a 10-item screening tool41 to identify patient needs across the DOH domains that can be used by health systems to address the root causes of health disparities and see how they relate to race and ethnicity.

“People should stop conflating biological and social conceptions. If what you're interested in is genetic risk factors, the correct question to ask is what is the person’s family medical history.”

- Dr. Joseph Graves, professor of biological sciences, North Carolina Agricultural and Technical State University

“If we put white folks in the exact same situation that they put Black folks in, they would have all the comorbidities that we're witnessing in the research.”

- Edwin Lindo, acting assistant professor in the department of family medicine, the University of Washington School of Medicine, assistant dean for social and health justice, office of health care. Excerpt derived from Clinical Problem Solvers Anti-Racism Podcast Series, Episode 141.42

While race and ethnicity shouldn’t be used as shortcuts in clinical care, it’s important to collect data on race and ethnicity to understand and measure disparities in patient populations and target strategies to increase equitable care (see sidebar, “Race-related health disparities in breast cancer,” for more information). A colorblind or identity-blind approach that ignores the race and ethnicity of patients also ignores the racism that racially and ethnically diverse patients disproportionately experience. 

Race-related health disparities in breast cancer

 

Literature has shown that, on average, white women (13%) and Black women (12%) have comparable breast cancer incidence rates.43 Yet, Black patients' breast cancers have a 41% increased mortality rate compared to white patients.44 They also are more likely to experience delays in diagnostic evaluation and biopsy and have decreased access to standard of care screening technology. Literature also suggests that socioeconomic status and education doesn’t eliminate most of the disparities,45 inferring that racism may have a large role to play. On an individual level, a Black patient has a higher risk for breast cancer if they have a family history of breast cancer, a genetic marker, and/or drivers of health that would increase the risk of breast cancer.46

Show more

Questions for health systems to consider:

  • Where are your gaps in race and ethnicity data?
  • What categories of data are collected as it relates to drivers of health?
  • How is race and ethnicity data being used?

Assessing the use of race in clinical algorithms, AI, and tools

 

Clinical algorithms, often represented as flow diagrams,48 serve to guide clinical decision-making as it relates to prevention, diagnosis, and treatment.49 For example, clinical algorithms determine when someone should receive heart surgery or who is at more risk for kidney stones.50 There are several clinical algorithms in medical practice today that adjust or “correct” their outputs based on the patient’s race.51 This is in part because the foundational data sets for these calculations were based on either a lack of diversity and/or flawed analysis.

As previously discussed, the eGFR calculations are one example of a type of algorithm that lowers an individual’s risk score for kidney disease because of their race, potentially leading to deferred treatment or procedures.52 While the eGFR equations are a well-documented example of a clinical algorithm that race-corrects, there are several others outlined in literature that expand across therapy areas and populations (see sidebar, “Reassessing clinical algorithms that use race to adjust outputs,” for more information). 

Reassessing clinical algorithms that use race to adjust outputs

 

Heart failure risk score: The Society of Thoracic Surgeons develop calculators to estimate mortality risk and other complications during surgery. The calculators use race and ethnicity based on observed differences in outcomes across racial and ethnic groups, despite a lack of understanding as to the specific factors correlated to these observed differences. This insertion of race increases the risk of mortality and complications for Black patients during cardiac surgical interventions, but not for any other race. This could lead to decreased surgical interventions among Black patients due to perceived risk.

Urinary tract infection (UTI) calculator: This diagnostic calculator is used to estimate the risk of a UTI in children aged between 2 and 23 months and guides clinician’s decision about pursuing definitive diagnostic testing. Child’s race is one of the input variables for this calculator and if a child identifies as Black – the calculator assigns a lower probability of a UTI for them. This could hinder clinicians from conducting definitive testing in Black children presenting with UTI symptoms.53

Simple calculated osteoporosis risk estimation (SCORE): This tool is used to determine the risk of low bone density in women and guide decisions related to pursuing screening with DXA Scan. High score indicates high risk of osteoporosis. If a patient identifies as non-Black, they are assigned five additional points. Like the UTI calculator, SCORE puts Black patients at lower risk and can deter clinicians from conducting further diagnostic evaluation in these patients. This may result in delayed diagnosis and treatment.54

Show more

Clinical algorithms that adjust for race work under the assumption that racial disparities exist and are normal and immutable without understanding the specific factors or DOH that are driving them.55 Some algorithm developers provide no rationale for why race differences exist, and others do offer explanations but are rooted in flawed or biased data. For example, the heart failure algorithms included race in the calculation despite a lack of understanding of the underlying factors associated with the risk. Whereas, the eGFR calculation was based in part, under the flawed and biased notion that Black people are more muscular.56 The Agency for Healthcare Research and Quality also states57 that studies used to justify the insertion of race in clinical algorithms may not be representative of diverse races and can reinforce the misconception that race is a contributing cause. For example, the University of California Davis Health found that a study used to develop the MDRD equation used to calculate eGFR was predominantly white, with Black patients comprising only 12% of the study population.58

There have been growing calls to understand and assess the insertion of race in clinical algorithms59 because they can guide clinical decisions in a way that perpetuates health disparities and negatively impact the health outcomes of racially diverse individuals.60

According to our primary and secondary research, industry stakeholders are addressing race in algorithms in some of the following ways: 

  • Standing up teams and partnerships to assess and screen for bias: Many organizations are just beginning to assess the insertion of race in algorithms. According to interviewees, within health systems, many are forming their own teams to examine this issue. The New York City Health Department launched the formation of the Coalition to End Racism in Clinical Algorithms (CERCA) aimed at ending the insertion of race in clinical algorithms. The coalition is comprised of 12 members across various health systems in New York City who have pledged to end race adjustment in at least one clinical algorithm.61 The National Kidney Foundation and the American Society of Nephrology also established a task force aimed at assessing the insertion of race in the eGFR which resulted in the recommended removal of race in this equation.62
  • Removing and replacing race-based calculations:Our interviewees also told us they’re taking steps to remove race from clinical algorithms. One example is the calculator designed to determine the likelihood of having a successful vaginal birth after cesarean (VBAC) (see sidebar, “Removing race from the VBAC calculator,” for more information). The nation’s largest public health system, NYC Health + Hospitals, recently announced63 they’ll no longer use the VBAC calculator, or a similar algorithm for kidney testing, because of their potentially negative impact on racially and ethnically diverse patients. Additionally, the health system intends to assess calculators used to test pulmonary function and diagnose urinary tract infections in children. After assessing these initial four calculations, the health system plans to use an article published in the New England Journal of Medicine64 as a guide for which calculations to evaluate next. 

Removing race from the VBAC calculator

 

Vaginal birth after cesarean (VBAC) refers to the vaginal delivery of a baby after having a caesarean section in a previous pregnancy.65 Since 2007, clinicians have used a VBAC calculator to determine the likelihood of a successful vaginal birth after a prior cesarean section.66 The calculator factors in age, height, weight, and the patient’s past medical history. It also asks if the patient is Black or Latinx. The VBAC score predicts a lower chance of success if the person is identified as Black or Latinx.67

While the study used to create the algorithm showed that marital status and insurance source were important factors correlated with VBAC success, they weren’t included in the algorithm, even though less relevant inputs such as race and ethnicity were. In the United States, Black women are significantly more likely to have cesarean section compared to all other races.68 Use of this algorithm can exacerbate these disparities. After years of work by stakeholders in the industry, the calculator has been replaced by a new version that achieves the same level of accuracy but no longer includes race and ethnicity.

Show more

The University of Chicago, Booth School of Business, developed an Algorithmic Bias Playbook69 to guide health systems leaders and other stakeholders on how to define, measure and mitigate bias in algorithms. They outlined four key steps to mitigate algorithmic bias: inventory algorithms, screen for bias, retrain or remove algorithms, and set up structure to prevent future bias. 

Like algorithmic bias, artificial intelligence (AI) also can exacerbate health disparities through human bias and lack of representation in training data sets used to inform machine learning. One study revealed how AI software that used health care costs to predict and rank which patients need extra care was unintentionally and systematically discriminating against Black individuals. The bias stemmed from using health costs as a proxy for health needs, but didn’t factor in that Black patients often spend less on health care than white patients because of the unique disparities they face. This bias led the algorithm to falsely conclude that Black patients were healthier than white patients and resulted in less money and interventions spent on them.70 Another study that analyzed 62 AI-based COVID-19 diagnosis studies for potential clinical use found that all had methodological flaws and/or underlying biases.71 

“We need to be careful about the risk of machine learning models systematically making inaccurate predictions about certain groups of people (eg. racial, ethnic, socioeconomic) based on historical patterns from the training data sets that come from suboptimal care delivered to those groups.  This may deepen biases and inequities in care delivery.” 

- Ron Li, informatics director for digital health and artificial intelligence clinical integration, Stanford Health Care

Some of the health systems we interviewed are developing their own models and validating them to assess for potential biases. For example, one academic medical center is building its own algorithms and validating them internally and externally to remove any race and ethnicity-based biases. To build models efficiently, the medical center used Jupyter Notebook which can curate a variety of models in a matter of hours.

Similarly, in 2018, the University of California Los Angeles started to develop a machine learning model to predict the risk of hospitalization and/or emergency department visits.72 Instead of using health costs as a proxy for unmet needs due to potential biases, the university used emergency department visits and admissions and incorporated it into the model validation process help detect bias. With the model validation process, the algorithm reduced or even eliminated potential racial bias but acknowledged that the model will inherit bias from other factors and should be reviewed periodically to address any unintended bias. Consistent audits of AI tools will be essential to ensuring that biases are reduced as much as possible.73

In October 2021, the Food and Drug Administration released an action plan74 for AI and machine learning device development. The agency also released guiding principles for Good Machine Learning Practice,75 which highlight the importance of addressing and mitigating bias and lays out ten action steps for promoting safe, effective, and high-quality medical devices that use AI and machine learning. 

“Users of AI models should have an adequate understanding of how those models are trained and whether they are appropriate for the patient populations they are caring for."

- Ron Li, informatics director for digital health and artificial intelligence clinical integration, Stanford Health Care

Racial bias is also a problem with some of the tools and devices used in clinical settings. One example of technology-based bias is a tool used to measure jaundice in newborns that underestimates risk of jaundice in lighter skin and overestimates in darker skin.76 Furthermore, in the past, East Asian race was considered a risk factor when determining which newborns were more likely to develop neonatal jaundice.77 In a recent statement, the American Academy of Pediatrics plans to “revise all practice guidelines and/or policies that include race assignment as a part of clinical decision-making,” as appropriate. 78 

Through our literature analysis and interviews, we learned that stakeholders are addressing biases in the tools and devices that inform treatment and diagnosis by:

  • Using other screening tools: While our interviewees acknowledged that changing the effectiveness of the devices will require collaboration with vendors, as well as extensive time and resources, some have focused on adjusting threshold values or using other screening tools. One health system convened a working group with clinicians to change protocols regarding pulse oximeter readings. Others are utilizing other screening tools. For example, one health system is relying less on pulse oximeter measurements and more on other screening tools such as labored breathing while walking up and down a hall to evaluate patients. 
  • Collaborating with approved vendors: Interviewees also told us they’re establishing processes for selecting vendors to ensure equity is embedded within the tools. One health system uses a deliberate governance process and scoring rubric to choose who they want to work with or what technologies and systems they use. The scoring rubric allowed them to learn from previous problems, identify blind spots, and look for potential sources of bias. Health systems can also specifically ask their device makers about the diversity of clinical trials to enhance representation. 

Moving toward more equitable health outcomes

 

One of the many ways that health systems can activate health equity in their communities is by understanding that race is socially constructed and has no basis in biological differences within the human species. Health systems should consider the following approaches to address the insertion of race in medicine and the biases that stem from it:

  • Implementing strategies for data granularity and standardization: Developing standards for data collection can help health systems better understand their patient populations and the health challenges that should be addressed. Health systems should consider expanding the types of data they collect to include race, ethnicity, preferred language, gender, and sexual orientation. In addition to claims data, health systems can use new datasets like employment data, and leverage nontraditional and community-based partnerships.

    Questions to consider:

  • Where are there gaps in data for race and ethnicity?
  • What type of data categories are collected as it relates to drivers of health? 
  • Developing metrics for proper data collection and use: In addition to data standards, measures and metrics for proper data collection and use is vital. Health systems should consider implementing provider education and training on why this data is important to collect and how to talk to patients about the importance of providing this data in a culturally humble and empathetic way and developing scorecards and HEIs.

    Questions to consider:

  • How is racial and ethnic data being used?
  • How is the workforce being held accountable to properly collect and validate data?
  • Reevaluating clinical algorithms: Reexamining long-standing clinical algorithms provides the opportunity to help ensure all patients receive the care they need. Health systems should consider forming designated teams to evaluate algorithms and assess which clinical algorithms are currently being used in their facility, how race is used in the algorithm or calculation, if race is justified, and determine the underlying factors that are driving differences in health outcomes that should be included in the algorithm. 

    Questions to consider:

  • Is the use of race in clinical algorithms explainable?
  • What data is used to rationale race in clinical algorithms?Are there any groups over-or-underrepresented? 
  • Does the model account for existing structural biases?
  • What processes and procedures are in place to evaluate how insights from analytics are being used to trigger action?
  • How do predictive and data models perform differently across populations?
  • Conducting regular audits: Continuous audits ensure the intention of using racial adjustment is having the desired impact to clinical practice and outcome. As the use of AI, medical devices, and other technologies increase, continually testing for biases will be critical to ensuring that health disparities aren’t unintentionally exacerbated. Health systems should conduct regular audits of their AI systems to check for bias and re-evaluate their current tools and devices by considering if other screening tools should be used, reviewing their vendors, and understanding the diversity of the clinical trials that devices were tested on..

    Questions to consider:

  • Does the technology solution account for existing structural biases within its target population?
  • Does it unintentionally exclude any communities it ends up serving?
  • What processes exist for detecting and mitigating bias in medical devices and other technologies?

The centuries of race-based medical practice can’t be erased overnight. But with system-level approaches to addressing bias, such as having better race and ethnicity data, a more nuanced understanding of the history of clinical algorithms, and more scrutiny in the effectiveness and selection of medical tools, health systems can make strides in improving health equity.

Appendix

Deloitte's Services for the Health Care Industry

 

Amid uncertainty and change, health care stakeholders are looking for new ways to transform the journey of care. By focusing on the differentiated needs of plans and providers, our US health care practice helps clients transform uncertainty into possibility, and rapid change into lasting progress.  

Learn more

 

The Deloitte Health Equity Institute

 

The Deloitte Health Equity Institute, under the guidance of leaders Jay Bhatt and Kulleni Gebreyes, conducts original research and disseminates findings to help drive data-based equitable outcomes, as well as activate interventions that address systemic inequities.

Learn more

  1. Sara Heath, “ACP calls to end health disparities, racial bias in healthcare,” PatientEngagementHIT, January 13, 2021.

    View in Article
  2. Centers for Disease Control and Prevention, Risk for COVID-19 infection, hospitalization, and death by race/ethnicity, March 25, 2022.

    View in Article
  3. William F. Marshall, “Coronavirus infection by race: What's behind the health disparities?,” Mayo Clinic, August 13, 2020.

    View in Article
  4. Heath, “ACP calls to end health disparities, racial bias in healthcare.”

     

    View in Article
  5. American Medical Association, “AMA adopts guidelines that confront systemic racism in medicine,” press release, June 15, 2021.

    View in Article
  6. Joe Sammen et al., Race is a social construct, Center for Health Progress, October 24, 2017.

    View in Article
  7. Centers for Disease Control and Prevention, Data & statistics on sickle cell disease, March 30, 2022.

    View in Article
  8. WGBH Educational Foundation and Clear Blue Sky Productions, “A mutation story,” video, 2001.

    View in Article
  9. Centers for Disease Control and Prevention, Data & statistics on sickle cell disease; ASH Foundation, ASH Sickle Cell Disease Initiative Fund, 2022.

    View in Article
  10. American Academy of Family Physicians, “Race based medicine,” July 2020.

    View in Article
  11. Timothy M. Smith, “Race-based medicine is wrong. How should physicians oppose it?,” American Medical Association, December 14, 2020.

    View in Article
  12. American Medical Association, “New AMA policies recognize race as a social, not biological, construct,” press release, November 16, 2021.

    View in Article
  13. John P. A. Ioannidis, “Recalibrating the use of race in medical research,” Jama Network, January 25, 2021.

    View in Article
  14. Cara V. James et al., “Modernizing race and ethnicity data in our Federal Health Programs,” The Commonwealth Fund, October 26, 2021.

    View in Article
  15. Centers for Disease Control and Prevention, “Demographic characteristics of people receiving COVID-19 vaccinations in the United States,” 2022.

    View in Article
  16. Alexander Ladenheim, “Race and eGFR: Addressing health disparities in chronic kidney disease,” UC Davis Health, April 29, 2021.

    View in Article
  17. Theresa Gaffney, “A yearslong push to remove racist bias from kidney testing gains new ground,” STAT, July 17, 2020.

    View in Article
  18. James A. Diao, Gloria J. Wu and Herman A. Taylor, “Clinical implications of removing race from estimates of kidney function,” no. 2, pp. 184-186.

    View in Article
  19. The New England Journal of Medicine, “Racial bias in pulse oximetry measurement,” December 17, 2020.

    View in Article
  20. Zachary Vesoulis et al., “Racial discrepancy in pulse oximeter accuracy in preterm infants,” Journal of Perinatology, (2022): pp. 79–85.

    View in Article
  21. David R. Roediger, “Historical foundations of race,” National Museum of African American History & Culture, accessed April 25,2022.

    View in Article
  22. Staffan Muller-Wille, "Race and History: Comments from an Epistemological Point of View," Sci Technol Human Values (2015): pp. 597-606.

    View in Article
  23. Raj Bhopal, "The beautiful skull and Blumebach's errors: the birth of the scientific concept of race," BMJ (2007): pp. 1308-9.

    View in Article
  24. Facing History & Ourselves, "The Science of Race: Holocaust and Human Behavior," accessed May 9, 2022.

    View in Article
  25. Paul Wolff Mitchell, "The fault in his seeds: Lost notes to the case of bias in Samuel George Morton's cranial race science," PLoS Biol (2018).

    View in Article
  26. Ayah Nuriddin, Graham Mooney, and Alexandre I R White, “Reckoning with histories of medical racism and violence in the USA,” Elsevier Public Health Emergency Collection, (2020): pp.949–951.

    View in Article
  27. Meredith A Anderson, Atul Malhotra and Amy L Non, “Could routine race-adjustment of spirometers exacerbate racial disparities in COVID-19 recovery?,” The Lancet Respiratory Medicine, no. 2(2021): pp. 124-125.

    View in Article
  28. Centers for Disease Control and Prevention, “About the National Health and Nutrition Examination Survey,” accessed April 25,2022.

    View in Article
  29. Meredith A Anderson, Atul Malhotra and Amy L Non, “Could routine race-adjustment of spirometers exacerbate racial disparities in COVID-19 recovery?

    View in Article
  30. American Thoracic Society, “Removal of “race correction” in pulmonary function tests shows more prevalent and severe lung disease among black patients,” press release, May 16, 2021.

    View in Article
  31. Megan Gannon, “Race is a social construct, Scientists argue,” Scientific American, February 5, 2016.

    View in Article
  32. Christina Amutah et al., “Misrepresenting race — The role of medical schools in propagating physician bias,” The New England Journal of Medicine, March 4, 2021.

    View in Article
  33. NewYork-Presbyterian, “Dalio Center for Health Justice,” accessed April 25, 2022; Northwell Health, “We ask because we care,” accessed April 25, 2022; Stanford Medicine, “Health Equity—Our mission,” accessed April 25, 2022.

    View in Article
  34. Agency for Healthcare Research and Quality, Defining categorization needs for race and ethnicity data, accessed April 25, 2022.

    View in Article
  35. Martha Hostetter, “Confronting racism in health care,” The Commonwealth Fund, October 18, 2021.

    View in Article
  36. Amutah et al., “Misrepresenting race — The role of medical schools in propagating physician bias."

    View in Article
  37. Michelle Tong and Samantha Artiga, “Use of race in clinical diagnosis and decision making: Overview and implications,” Kaiser Family Foundation, December 9, 2021.

    View in Article
  38. NEJM roundtable

    View in Article
  39. Ana Sandoiu, “'Weathering': What are the health effects of stress and discrimination?,” Medical News Today, February 26, 2021.

    View in Article
  40. Rachel Nuzum, Corinne Lewis and Debbie I. Chang, “Measuring what matters: Social drivers of health,” The Commonwealth Fund, November 6, 2021.

    View in Article
  41. Alexander Billioux et al., “Standardized screening for health-related social needs in clinical settings: The accountable health communities screening tool,” National Academy of Medicine, May 30, 2017.

    View in Article
  42. Maani Kamal, “Episode 141: Antiracism in medicine series episode 2 – Dismantling race-based medicine part 1: Historical and ethical perspectives with Edwin Lindo,” November 17, 2020.

    View in Article
  43. Susan G. Komen, “Breast Cancer Risk: Race and ethnicity,” March 15, 2022.

    View in Article
  44. ACS Medical Content and News Staff, “More black women die from breast cancer than any other cancer,” American Cancer Society, February 14, 2022.

    View in Article
  45. Valentina A. Zavala et al., “Cancer health disparities in racial/ethnic minorities in the United States,” British Jornal of Cancer, (2021): pp. 315-332.

    View in Article
  46. Komen, “Breast Cancer Risk.”

    View in Article
  47. Amutah et al., “Misrepresenting race — The role of medical schools in propagating physician bias."

    View in Article
  48. Michael W. Kattan, “Clinical algorithms, encyclopedia of medical decision making,” accessed April 25, 2022.

    View in Article
  49. Carmi Z. Margolis, “Uses of clinical algorithms,” Jama Network, February 4, 1983; Andis Robeznieks, “Feds warned that algorithms can introduce bias to clinical decisions,” American Medical Association, June 23, 2021.

    View in Article
  50. Sharon Begley, “Racial bias skews algorithms widely used to guide care from heart surgery to birth, study finds,” STAT, June 17, 2020.

    View in Article
  51. Darshali A. Vyas, Leo G. Eisenstein, and David S. Jones, “Hidden in plain sight — Reconsidering the use of race correction in clinical algorithms,” The New England Journal of Medicine, August 27, 2020.

    View in Article
  52. Begley, “Racial bias skews algorithms widely used to guide care from heart surgery to birth, study finds.”

    View in Article
  53. Vyas, Eisenstein, and Jones, “Hidden in plain sight,” The New England Journal of Medicine.

    View in Article
  54. Ibid.

    View in Article
  55. Ibid.

    View in Article
  56. Understanding African American and non-African American eGFR laboratory results | National Kidney Foundation

    View in Article
  57. Agency for Healthcare Research and Quality, Impact of healthcare algorithms on racial and ethnic disparities in health and healthcare, January 25, 2022.

    View in Article
  58. Ladenheim, “Race and eGFR: Addressing health disparities in chronic kidney disease,” UC Davis Health; A S Levey et al., “A more accurate method to estimate glomerular filtration rate from serum creatinine: a new prediction equation. Modification of diet in renal disease study group,” National Library of Medicine, March 16, 1999.

    View in Article
  59. Agency for Healthcare Research and Quality, Impact of healthcare algorithms on racial and ethnic disparities in health and healthcare.

    View in Article
  60. Vyas, Eisenstein, and Jones, “Hidden in plain sight,” The New England Journal of Medicine.

    View in Article
  61. Jonathan Greig, “NYC creates coalition to end bias and 'race norming' in medical algorithms,” ZDNet, December 13, 2021.

    View in Article
  62. Cynthia Delgado et al., “A Unifying Approach for GFR Estimation: Recommendations of the NKF-ASN Task Force on Reassessing the Inclusion of Race in Diagnosing Kidney Disease,” American Journal of Kidney Disease, no.2 (2022): pp. 268-288

    View in Article
  63. Anuja Vaidya, “NYC Health + Hospitals drops use of two race-based clinical assessments,” MedCityNews, May 18, 2021.

    View in Article
  64. Elise G. Valdes, “Examining cesarean delivery rates by race: a population-based analysis using the robson ten-group classification system,” Journal of Racial and Ethnic Health Disparities, (2021): PP. 844-851.

    View in Article
  65. Mayo Clinic, “Vaginal birth after cesarean (VBAC),” June 09, 2020.

    View in Article
  66. Katie Palmer, “Changing the equation: Researchers remove race from a calculator for childbirth,” STAT, June 3, 2021.

    View in Article
  67. Vyas, Eisenstein, and Jones, “Hidden in plain sight,” The New England Journal of Medicine.

    View in Article
  68. Elise G. Valdes, “Examining cesarean delivery rates by race: a population-based analysis using the robson ten-group classification system."

    View in Article
  69. Center for Applied AI at Chicago Booth, Algorithmic bias playbook, 2021.

    View in Article
  70. Ziad Obermeyer, “Dissecting racial bias in an algorithm used to manage the health of populations,” Science, October 25, 2019.

    View in Article
  71. Michael Roberts et al., “Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans,” Nature Machine Intelligence, March 15, 2021.

    View in Article
  72. UC AI Working Group Final Report (ucop.edu)---------------------- Link not working. Please ask the author to send the correct/alternative link

     

    View in Article
  73. Ibid.

    View in Article
  74. US Food & Drugs Administration, Artificial Intelligence/Machine Learning (AI/ML)-Based software as a Medical Device (SaMD) Action Plan, January 2021.

    View in Article
  75. US Food & Drugs Administration, “Good machine learning practice for medical device development: Guiding principles,” October 27, 2021.

    View in Article
  76. Bolajoko O. Olusanya et al., "Transcutaneous bilirubin nomograms in African neonates," PLoS One (2016).

    View in Article
  77. Subcommittee on Hyperbilirubinemia, "Management of Hyperbilirubinemia in the Newborn Infant 35 or More Weeks of Gestation," Pediatrics (2004): 297-316.

    View in Article
  78. American Academy of Pediatrics, "American Academy of Pediatrics Calls for Elimination of Race-Based Medicine," May 2, 2022.

    View in Article
  79. Michelle Tong and Samantha Artiga, “Use of race in clinical diagnosis and decision making: Overview and implications,” KFF, December 9, 2021. 

    View in Article
  80. Whitney J. Palmer, “Black women twice as likely to experience breast cancer diagnostic delay,” Diagnostic Imaging, July 20, 2021. 

    View in Article
  81. Linda Goler Blount, “Black women are receiving inferior breast screening,” Imaging Technology News, July 6, 2021; Maria Aspan, “‘We can’t ever go to the doctor with our guard down’: Why Black women are 40% more likely to die of breast cancer,” Fortune, June 30, 2020. 

    View in Article
  82. Erin Brodwin and Nicholas Fleur, “‘These numbers are incredibly important’: Doctors and lawmakers call on FDA to address racial disparities in pulse oximeters,” STAT news, February 10, 2021; Tori Rodriguez, “Pulse oximetry: Examining racial bias in a valuable but flawed tool,” Pulmonology Advisor, April 2, 2021. 

    View in Article
  83. Tiffany L. Green et al., “Rethinking bias to achieve maternal health equity: Changing organizations, not just individuals,” Obstetrics and Gynecology 137, no. 5 (2021): pp. 935–40. 

    View in Article
  84. Jamila Taylor et al., Eliminating racial disparities in maternal and infant mortality, Center for American Progress, May 2019. 

    View in Article
  85. Rutgers University, “African-Americans more likely to be misdiagnosed with schizophrenia,” ScienceDaily, March 21, 2019. 

    View in Article
  86. June Liang, Brittany E. Matheson, and Jennifer M. Douglas, “Mental health diagnostic considerations in racial/ ethnic minority youth,” Journal of Child and Family Studies 25, no. 6 (2016): pp. 1, 926–40; Mental Health America, “Black and African American communities and mental health,” accessed March 16, 2022; Tahmi Perzichilli, “The historical roots of racial disparities in the mental health system,” Counseling Today, May 7, 2020. 

     

    View in Article
  87. Christina Caron, “Why are more Black kids suicidal? A search for answers,” New York Times, November 18, 2021. 

    View in Article
  88. Monika K. Goyal et al., “Racial disparities in pain management of children with appendicitis in emergency departments,” JAMA Pediatrics 169, no. 11 (2015): pp. 996–1002. 

    View in Article
  89. Darshali A Vyas, Leo G. Eisenstein, and David S. Jones, "Hidden in Plain Sight--Reconsidering the Use of Race Correction in Clinical Algorithms," New England Journal of Medicine 383(2020): 874-882.

    View in Article
  90. Enrique Rivero, “Black patients’ Lyme disease often diagnosed late, possibly due to missed signs,” UCLA Newsroom, September 30, 2021.

    View in Article
  91. Janice A. Sabin, “How we fail black patients in pain,” AAMC, January 6, 2020.

    View in Article

Project team: Apoorva Singh and Richa Malhotra contributed significantly to the research and analysis of the paper. Wendy Gerhardt provided invaluable guidance and support throughout the project.

The authors would also like to thank Kulleni Gebreyes, Asif Dhar, Matthew Piltch, Nicole Kelm, Jennifer Radin, and Kenneth Abrams for their insights, expertise and critical feedback on the research. The authors would also like to thank Rebecca Knutsen, Laura DeSimio, Meghna Patel, and Zion Bereket, and the many others who contributed to the success of this project.

The study would not have been possible without our research participants who graciously agreed to participate in the interviews. They were generous with their time and insights.

Cover image by: Jim Slatton

Did you find this useful?

Thanks for your feedback

If you would like to help improve Deloitte.com further, please complete a 3-minute survey