Ensuring Equity and Fairness When Deploying Artificial Intelligence

Laura M. Cascella, MA, CPHRM

Artificial intelligence (AI) is quickly trailblazing a path in healthcare with applications addressing administrative and logistical support, disease detection, precision medicine, predictive analytics, patient engagement, and more. AI is perhaps the most consequential advancement in modern medicine, and it offers hope to accelerate research, propel the discovery of new drugs and therapies, and vastly improve patient care. Yet, to deliver on these promises, the potential pitfalls of AI must be addressed.

Bias sits atop the list of risks associated with AI, and the ways in which it can occur are complex and multifaceted. Bias can arise from training data, algorithmic design or function, and how people (e.g., clinicians and patients) interact with the technology.¹ Unfortunately, identifying bias is not always straightforward, particularly when it occurs in sophisticated AI models that have opaque functionality and continue to evolve over time.

Although AI is relatively new, bias in healthcare is not — and history has shown its detrimental consequences. Health inequities, lack of representation, poor communication, patient harm, lack of trust, reduced access to care, and other suboptimal outcomes can occur when bias is left unchecked.

For AI to deliver on its promises — particularly its promise to improve patient outcomes — it must help resolve inequities, not perpetuate them. Thus, it is incumbent on healthcare organizations to take steps to ensure equity and fairness when deploying AI. Doing so will require a prudent and discerning approach during implementation and throughout the technology’s lifecycle.

It’s important to note that tackling bias in AI is not merely an issue for healthcare organizations. Much of the work needs to take place upstream, with interventions for AI developers and coders as well as laws and standards to create a framework for the ethical use of AI. However, AI continues to forge ahead in the absence of many of these guardrails, making it vital that healthcare organizations develop their own strategies and safeguards.

The following recommendations can help healthcare organizations prioritize equity and fairness as they develop AI initiatives:

• Form a panel or committee to oversee AI governance and evaluate potential AI vendors and applications. Ensure the panel or committee includes diverse representatives and stakeholders, such as health information technology and AI specialists, data analysts, clinicians, nonclinical staff, individuals with legal and ethical expertise, risk managers, and health equity experts.
• Develop criteria and a standardized process for evaluating AI vendors and products. Considerations should include:
  • The diversity of programmers and coders and the vendor’s efforts and commitment in relation to diversity and equity.
  • The heterogeneity of training data and whether it aligns with the target patient population. A recent review notes that “a diverse and representative dataset ensures that AI algorithms are rigorously tested across different scenarios, thereby enhancing their overall performance and utility.”²
  • How developers are trained to discern biased training data and identify potential bias in their algorithms.
  • Transparency in how the AI application works and produces results.
  • Data that validates the quality and performance of the AI application (not just in silico but also in clinico).
• Include patients in the evaluation and testing of AI solutions, particularly those with which they may have direct interaction (e.g., virtual health assistants, chatbots, remote patient monitoring devices, and diagnostic aids). The Advisory Board notes that “Patients can also identify biases and pain points that impact their use of the AI solution.”³
• Determine how to account for nonquantifiable factors that should be factored into processes and decision-making but that fall outside the parameters of algorithms (e.g., subtleties in context, cultural differences, and underlying motivations).
• Routinely audit AI performance and conduct validation studies to identify potential bias and ensure the systems remain fair, accurate, and effective in the settings in which they are deployed. Consider establishing quality assurance teams to facilitate auditing and validation as well as to devise quality improvement strategies.
• Monitor AI applications over time to ensure they continue to perform as expected and do not introduce bias because of shifts in health trends, the environment, or the patient population. An article in NPJ Digital Medicine explains that “these highly complex systems are sensitive to changes in the environment and liable to performance decay.”⁴
• Educate providers and staff members about the risk of potential bias in AI applications and the strategies and best practices that can help ensure that AI is equitable and fair.
• Encourage clinicians to think critically about AI results and recommendations, collaborate with other providers and specialists when questions arise, and use alternative sources to verify information or support decision-making. Stress the importance of using a human-in-the-loop approach to AI.
• Clearly communicate with patients/families about AI’s benefits and risks as well as its limitations. Discuss how the organization is using AI and implementing safeguards to prevent issues associated with bias. Well-informed patients can better engage with their providers and actively participate in their care.
• Create opportunities for collaboration and discussion about equitable and fair approaches to care in an AI-enabled healthcare environment. Consider hosting workshops, discussion groups, and online forums to support continuous learning and improvement.
• Stay abreast of evolving guidance and best practices from experts and professional organizations about identifying and addressing bias in health AI and ensuring the technology is impartial.⁵

The concepts of equity and fairness in healthcare are closely tied to three of the main pillars of medical ethics: beneficence (acting in patients’ best interests), nonmaleficence (doing no harm), and justice (ensuring equality in care and treatment). AI offers transformative potential for healthcare but also creates ethical challenges because it is vulnerable to bias. As a result, it may reinforce health disparities and further marginalize certain populations.

Although bias is not always easy to eradicate, healthcare organizations can take proactive measures to address it as part of AI initiatives. Establishing multidisciplinary and diverse governance and oversight teams; carefully assessing vendors and products; auditing, validating, and monitoring AI applications; providing comprehensive education; and engaging patients/families will support an AI strategy focused on equity and fairness.

Endnotes

¹ Ueda, D., Kakinuma, T., Fujita, S., Kamagata, K., Fushimi, Y., Ito, R., . . . Naganawa, S. (2024). Fairness of artificial intelligence in healthcare: Review and recommendations. Japanese Journal of Radiology, 42(1), 3–15. doi: https://doi.org/10.1007/s11604-023-01474-3

² Ibid.

³ Aderhold, T. & League, J. (2024, October 7 [last updated]). Market insights: 3 principles for the equitable use of AI in healthcare. The Advisory Board. Retrieved from www.advisory.com/topics/health-equity/2024/04/3-principles-equitable-use-of-ai-in-healthcare

⁴ Feng, J., Phillips, R. V., Malenica, I., Bishara, A., Hubbard, A. E., Celi, L. A., & Pirracchio, R. (2022). Clinical artificial intelligence quality improvement: Towards continual monitoring and updating of AI algorithms in healthcare. NPJ Digital Medicine, 5(1), 66. doi: https://doi.org/10.1038/s41746-022-00611-y

⁵ Ueda, et al., Fairness of artificial intelligence in healthcare: Review and recommendations; Aderhold, et al., Market insights: 3 principles for the equitable use of AI in healthcare; Feng, et al., Clinical artificial intelligence quality improvement: Towards continual monitoring and updating of AI algorithms in healthcare; Dankwa-Mullan, I. (2024). Health equity and ethical considerations in using artificial intelligence in public health and medicine. Preventing Chronic Disease, 21, 240–245. doi: http://dx.doi.org/10.5888/pcd21.240245; Northeastern University. (2025, March 11). Expert insights on responsible AI solutions for healthcare: Best practices for implementation. Retrieved from https://ai.northeastern.edu/news/expert-insights-on-responsible-ai-solutions-for-healthcare-best-practices-for-implementation

---

Prevention & Education Center home