Dear Editor,
In the dynamic healthcare landscape, the adoption of artificial intelligence (AI) technologies promises a more digitized and personalized system of care that can substantially enhance patient outcomes. AI can facilitate accurate predictions, improve diagnostic precision, and support tailored treatment plans that address individual patient needs. Furthermore, AI supports evidence-based practice, reduces costs, and accelerates medical research, improving the experiences of both healthcare providers and patients (1).
AI is now widely used in clinical settings, as demonstrated by its application in analyzing electronic health records and by its use of machine learning (ML) to predict HIV risk and inform pre-exposure prophylaxis recommendations (2). AI-driven clinical decision support systems (AI-CDSSs) are increasingly applied, particularly in complex domains such as oncology, to support clinical decision-making. Nonetheless, transferring AI algorithms from research environments to real-world clinical use has profound implications for healthcare delivery and raises critical ethical concerns about patient rights, including the risk of human rights infringements. Therefore, the ethical ramifications of implementing AI in healthcare must be carefully considered to ensure that patient rights are protected and preserved (3). Core safeguards include full patient information and informed consent, the right to opt out of AI-driven interventions, meaningful human oversight, data privacy, transparency, and clear accountability for AI-related outcomes.
Transparency and informed consent are central to these concerns. Patients are often unaware of the role of AI in diagnosis and treatment, although they regard disclosure of this role as essential. Promoting transparency and ensuring informed consent respect patient autonomy and foster trust in technology-enabled medicine (2). Park notes that patients are often unaware of the role of AI in their treatment process; compared with conventional information about short-term effects, details about AI use are perceived by patients as particularly important. Therefore, disclosing AI involvement in diagnosis and treatment during the informed consent process is essential and should be considered a fundamental patient right (4).
Equally important is the patient’s right to refuse AI-driven interventions, which preserves human agency and aligns care with personal values. The European Union General Data Protection Regulation mandates human oversight of AI decisions to mitigate challenges posed by AI complexity and opacity. Optimal outcomes arise when human clinical judgment is integrated with AI-generated data, reflecting a synergistic approach rather than full automation (5).
Data privacy and security are also major challenges because AI systems handle sensitive personal health information that may be vulnerable to breaches and unauthorized access, particularly when multiple data sources are integrated. Strengthening data anonymization, consent mechanisms, and cybersecurity protocols is imperative to preserve confidentiality and trust.
Accountability in AI-based healthcare presents multifaceted legal, ethical, and social challenges. Comprehensive frameworks that delineate responsibilities among AI developers, healthcare providers, and institutions are urgently needed to ensure transparency, address errors, and uphold ethical standards. Despite the growing role of AI, ultimate responsibility for patient care rests with human clinicians, underscoring the need for vigilant oversight (5).
Responsible implementation also requires proactive and preventive ethical and regulatory strategies. These strategies should maximize the benefits of AI while preventing adverse outcomes through ethical audits, continuous monitoring, and technological safeguards administered by ethically aware humans (6). A robust risk-assessment model that categorizes AI autonomy, from data presentation to full automation, is also critical for assigning appropriate confidence levels and transferring responsibilities accordingly. Bitterman et al. emphasize the importance of clearly defining and classifying AI autonomy for effective risk assessment of AI algorithms. They propose 5 levels of automation applicable to healthcare: Data presentation, clinical decision support, conditional automation, high automation, and full automation. Their framework suggests that increased automation requires greater confidence in AI systems; this shift, in turn, transfers responsibility from medical professionals to the developers and programmers of these technologies (7).
Finally, establishing a regulatory framework that categorizes AI systems according to risk levels will improve accountability among those involved in their design and use (6). The Artificial Intelligence Act uses a risk-based approach that classifies AI systems into 3 risk levels: unacceptable, high, such as medical devices, and low or minimal. This categorization facilitates the application of specific risk assessment standards and ensures that individuals who create and use higher-risk AI systems are subject to more stringent accountability requirements. This approach fosters safer and more responsible implementation of AI technologies in healthcare.
Protecting patient rights in AI-driven healthcare demands a multidisciplinary and multilayered approach that encompasses informed consent, the right to refuse AI, strong privacy safeguards, human oversight, and legally grounded accountability frameworks. Risk-based regulation combined with preventive ethics will optimize the transformative potential of AI while minimizing harm, fostering trust, and centering care on patients’ rights and dignity.
