Deprogramming Health Bias in AI Platforms to Achieve Equity

The following is a guest article by Kenji Taylor, MD, MSc, Clinical Consultant, Supervising Physician at Ubie

Artificial intelligence holds immense potential to revolutionize healthcare, offering precision medicine, accelerated drug discovery, and improved patient outcomes. AI is often touted as a tool for both patients and physicians, having the ability to take a deep dive into large swaths of data and provide analysis much quicker than humans ever could. However, this same data that AI is so proficient with is also a limiting factor for AI.

The current line of thinking is that a platform trained on medical data will be more accurate and trustworthy, and therefore more successful in advancing healthcare. However, this is where bias may creep in. Historically, medical research and data collection have excluded or underrepresented marginalized communities. By taking data at face value, you end up with platforms that neglect or oversimplify the specific health needs of these patient populations. As a result, these algorithms can inadvertently reinforce existing health disparities.

For instance, an AI system trained primarily on data from Caucasian patients may struggle to accurately diagnose diseases that disproportionately affect African Americans, such as sickle cell disease or cardiovascular disease. This can lead to delayed or incorrect diagnoses, resulting in poorer health outcomes for these patients. Similarly, AI-driven treatment recommendations might prioritize interventions that have been primarily tested on Caucasian populations, potentially overlooking effective treatments for other ethnic groups.

In addition to biases based on medical training and data, others can be inadvertently programmed into an AI by designing a system to prioritize certain outcomes or variables over others. For example, an algorithm focused on minimizing healthcare costs might prioritize cheaper treatments that are less effective for certain populations.

The consequences of not addressing these biases can have a negative impact on patients from marginalized communities who are already not receiving equitable care. Presenting AI platforms to these communities as a solution to long-standing inequity while not addressing the potential of bias in developing these systems will only continue to perpetuate disparities in access to care, diagnosis, treatment, and overall health outcomes.

As a medical provider passionate about eliminating health disparities, and a Clinical Consultant and Supervising Physician for Ubie, a company that has developed an AI symptom diagnostic solution, I have unique insights to both sides.

We need to not only be aware of health disparities but also more critically actively work at every step to ensure AI systems are promoting health equity. First and foremost, AI systems must be exposed to data that accurately reflects the health conditions prevalent in a wide variety of populations, including underrepresented populations. Additionally, patient-facing tools should incorporate race, ethnicity, and social determinants of health as key factors.

We know, for example, how critical a zip code may be than personal genetics in determining health outcomes. We also know that race/ethnicity in medicine is also not a standalone data point and intersects with an individual’s surroundings, social relationships, and socioeconomic background. By keeping health equity at the forefront of our work in building AI systems for health care, we can work toward eliminating health disparities by providing personalized insights tailored to individual needs and communities.

More specifically, developers should:

• Expand Data Representation: Include data from diverse populations to ensure AI systems are trained on a representative sample
• Develop Bias-Aware Algorithms: Create algorithms that can identify and correct potential biases in the data
• Implement Rigorous and Ongoing Testing: Thoroughly evaluate AI systems for fairness and equity before deployment that continues throughout deployment
• Foster Collaboration: Engage with diverse stakeholders, including patients, clinicians, and ethicists, in the development and deployment of AI
• Promote Transparency: Clearly communicate the limitations and potential biases of AI systems to end-users

With Ubie, we’ve taken the step of programming our AI at the disease level to ensure broad coverage within each body system. Programming is always done with physician supervision, based on the best available peer-reviewed evidence and real-world user feedback to continually improve upon the products. Additionally, we are incorporating race/ethnicity data census data to ensure equitable representation within the AI platform. This approach allows the AI system to better identify diseases prevalent in specific ethnic and racial groups, leading to more accurate diagnoses and personalized recommendations. Ubie’s commitment to incorporating race, ethnicity, and social determinants of health into its AI models is a crucial step toward reducing health disparities.

By taking the steps above we can move towards a future where AI is a tool for improving health equity rather than exacerbating disparities. It is crucial to recognize that eliminating bias is an ongoing process that requires continuous evaluation and refinement.

About Dr. Kenji Taylor

Dr. Taylor is a Japanese-African American physician focused on equitable access to high-quality health care through patient care, health care training, technology, and health systems development. With his medical degree from UPenn and Masters in Health Policy from Stanford, Dr. Taylor specialized in Family and Community Medicine at UCSF, where he served as chief resident prior to holding the faculty position at Stanford. He is currently practicing family medicine in Tokyo, teaching Japanese medical residents, and remotely serving as a medical director for Roots Community Health Center. Dr. Taylor serves as Clinical Consultant and Supervising Physician for Ubie.

The following is a guest article by Kenji Taylor, MD, MSc, Clinical Consultant, Supervising Physician at Ubie

Artificial intelligence holds immense potential to revolutionize healthcare, offering precision medicine, accelerated drug discovery, and improved patient outcomes. AI is often touted as a tool for both patients and physicians, having the ability to take a deep dive into large swaths of data and provide analysis much quicker than humans ever could. However, this same data that AI is so proficient with is also a limiting factor for AI.

The current line of thinking is that a platform trained on medical data will be more accurate and trustworthy, and therefore more successful in advancing healthcare. However, this is where bias may creep in. Historically, medical research and data collection have excluded or underrepresented marginalized communities. By taking data at face value, you end up with platforms that neglect or oversimplify the specific health needs of these patient populations. As a result, these algorithms can inadvertently reinforce existing health disparities.

For instance, an AI system trained primarily on data from Caucasian patients may struggle to accurately diagnose diseases that disproportionately affect African Americans, such as sickle cell disease or cardiovascular disease. This can lead to delayed or incorrect diagnoses, resulting in poorer health outcomes for these patients. Similarly, AI-driven treatment recommendations might prioritize interventions that have been primarily tested on Caucasian populations, potentially overlooking effective treatments for other ethnic groups.

In addition to biases based on medical training and data, others can be inadvertently programmed into an AI by designing a system to prioritize certain outcomes or variables over others. For example, an algorithm focused on minimizing healthcare costs might prioritize cheaper treatments that are less effective for certain populations.

The consequences of not addressing these biases can have a negative impact on patients from marginalized communities who are already not receiving equitable care. Presenting AI platforms to these communities as a solution to long-standing inequity while not addressing the potential of bias in developing these systems will only continue to perpetuate disparities in access to care, diagnosis, treatment, and overall health outcomes.

As a medical provider passionate about eliminating health disparities, and a Clinical Consultant and Supervising Physician for Ubie, a company that has developed an AI symptom diagnostic solution, I have unique insights to both sides.

We need to not only be aware of health disparities but also more critically actively work at every step to ensure AI systems are promoting health equity. First and foremost, AI systems must be exposed to data that accurately reflects the health conditions prevalent in a wide variety of populations, including underrepresented populations. Additionally, patient-facing tools should incorporate race, ethnicity, and social determinants of health as key factors.

We know, for example, how critical a zip code may be than personal genetics in determining health outcomes. We also know that race/ethnicity in medicine is also not a standalone data point and intersects with an individual’s surroundings, social relationships, and socioeconomic background. By keeping health equity at the forefront of our work in building AI systems for health care, we can work toward eliminating health disparities by providing personalized insights tailored to individual needs and communities.

More specifically, developers should:

• Expand Data Representation: Include data from diverse populations to ensure AI systems are trained on a representative sample
• Develop Bias-Aware Algorithms: Create algorithms that can identify and correct potential biases in the data
• Implement Rigorous and Ongoing Testing: Thoroughly evaluate AI systems for fairness and equity before deployment that continues throughout deployment
• Foster Collaboration: Engage with diverse stakeholders, including patients, clinicians, and ethicists, in the development and deployment of AI
• Promote Transparency: Clearly communicate the limitations and potential biases of AI systems to end-users

With Ubie, we’ve taken the step of programming our AI at the disease level to ensure broad coverage within each body system. Programming is always done with physician supervision, based on the best available peer-reviewed evidence and real-world user feedback to continually improve upon the products. Additionally, we are incorporating race/ethnicity data census data to ensure equitable representation within the AI platform. This approach allows the AI system to better identify diseases prevalent in specific ethnic and racial groups, leading to more accurate diagnoses and personalized recommendations. Ubie’s commitment to incorporating race, ethnicity, and social determinants of health into its AI models is a crucial step toward reducing health disparities.

By taking the steps above we can move towards a future where AI is a tool for improving health equity rather than exacerbating disparities. It is crucial to recognize that eliminating bias is an ongoing process that requires continuous evaluation and refinement.

About Dr. Kenji Taylor

Dr. Taylor is a Japanese-African American physician focused on equitable access to high-quality health care through patient care, health care training, technology, and health systems development. With his medical degree from UPenn and Masters in Health Policy from Stanford, Dr. Taylor specialized in Family and Community Medicine at UCSF, where he served as chief resident prior to holding the faculty position at Stanford. He is currently practicing family medicine in Tokyo, teaching Japanese medical residents, and remotely serving as a medical director for Roots Community Health Center. Dr. Taylor serves as Clinical Consultant and Supervising Physician for Ubie.