Mitigating Bias: Why Regulating AI in Healthcare Must Include Algorithms

Regulating AI in healthcare is becoming increasingly crucial as artificial intelligence continues to revolutionize the medical field. From transforming medical diagnosis to optimizing patient care, AI technologies offer unprecedented opportunities for improving healthcare services. However, with these advancements come responsibilities, prompting institutions like the Massachusetts Institute of Technology (MIT) and its counterparts to advocate for comprehensive regulatory frameworks.

Recent discussions among researchers from the MIT Department of Electrical Engineering and Computer Science (EECS), Equality AI, and Boston University highlight critical gaps in the oversight of both AI and non-AI algorithms within the healthcare sector. In a commentary published in the New England Journal of Medicine AI (NEJM AI), these researchers call for enhanced regulatory measures following the U.S. Office for Civil Rights (OCR) under the Department of Health and Human Services (HHS) issuing a new rule under the Affordable Care Act (ACA). This new rule is aimed at preventing discrimination in patient care decision support tools by prohibiting discrimination on various grounds, including race, color, and sex.

AI Technologies and Their Impact on Healthcare

AI technologies are transforming healthcare with innovations like Predictive Analytics, which forecast healthcare needs and personalize treatment plans. Medical Imaging techniques have enhanced the efficiency of identifying medical conditions, while Virtual Health Assistants provide real-time health advice and facilitate remote consultations. Additionally, AI is streamlining administrative functions, optimizing workflows, and improving patient engagement through personalized content.

However, this excitement must be tempered with caution. According to Marzyeh Ghassemi, associate professor of EECS and senior author of the commentary, the newly instituted ACA rule is pivotal. It encourages equity-driven improvements to existing non-AI algorithms and decision-support tools. This stance is shared by Isaac Kohane, chair of the Department of Biomedical Informatics at Harvard Medical School and editor-in-chief of NEJM AI, who underscores the importance of the quality of datasets and expert-selected variables in determining the efficacy of clinical risk scores.

Challenges in Regulation and Oversight

Despite the increasing approval of AI-enabled medical devices by the U.S. Food and Drug Administration (FDA)—nearly a thousand to date—the oversight of clinical risk scores remains insufficient. Researchers emphasize the necessity for a regulatory body dedicated to overseeing these scores, as they play a significant role in clinical decision-making.

The Jameel Clinic at MIT plans to address these regulatory shortcomings by hosting another regulatory conference in March 2025. Past conferences have sparked meaningful discussions among academics, industry leaders, and regulatory agencies regarding AI’s role in healthcare.

Maia Hightower, CEO of Equality AI and co-author of the commentary, notes that regulating these clinical risk scores is challenging due to their embedded nature in electronic medical records. Yet, she insists that this regulation is essential for ensuring transparency and non-discrimination, even under political climates favoring deregulation.

The Global Perspective and Ethical Considerations

The World Health Organization (WHO) has echoed the calls for responsible AI adoption. Dr. Tedros Adhanom Ghebreyesus, WHO Director-General, stresses the dual nature of AI: its potential to enhance global health and its capability to cause harm if misused. The WHO’s guiding principles advocate for AI systems that respect human autonomy, ensure safety, and promote wellbeing.

Looking to the future, AI in healthcare is expected to expand beyond pilot projects, integrating more comprehensively into all areas of healthcare delivery. This expansion necessitates stringent regulatory frameworks that prioritize patient safety and uphold ethical standards.

Striking a Balance Between Innovation and Responsibility

For executives like Alex Smith, the complexities of integrating AI into healthcare systems may seem daunting. However, understanding these regulatory landscapes and recognizing the potential for AI to drive productivity, enhance patient experiences, and enable data-driven decisions are crucial. While Alex may grapple with a lack of expertise or concerns about integration costs, the evolving regulatory measures provide a roadmap for navigating these challenges.

It becomes imperative for leaders and policymakers to establish clear standards for AI application within healthcare, embracing both innovative potential and ethical obligations. This balance will not only safeguard patient health but also ensure that AI’s transformative power is harnessed responsibly and effectively.

The dialogue around regulating AI in healthcare is far from concluded. As AI continues to evolve, so too must the frameworks that govern its use—ensuring that the pursuit of innovation never overshadows the commitment to equitable and safe medical care.

For more information, visit the original article on MIT News.