FolioStart free

Computer Science · Literature

Research papers on Explainable AI in healthcare

Recent and highly-cited academic work on explainable ai in healthcare, gathered from Semantic Scholar, CrossRef and OpenAlex.

Search all 200M+ papers on this topic, free →Or track new explainable ai in healthcare papers automatically as they publish

  3. A survey of methods for explaining black box models

    Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, et al. · 2019 · ISTI Open Portal · 4,914 citations

    In recent years, many accurate decision support systems have been constructed as black boxes, that is as systems that hide their internal logic to the user. This lack of explanation constitutes both a practical and an ethical issue. The literature reports many approaches aimed at overcoming this crucial weakness, sometimes at the cost of sacrificing accuracy for interpretability. The applications in which black box decision systems can be used are various, and each approach is typically developed to provide a solution for a specific problem and, as a consequence, it explicitly or implicitly delineates its own definition of interpretability and explanation. The aim of this article is to provi

  4. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models

    Tiffany H. Kung, Morgan Cheatham, Arielle Medenilla, et al. · 2023 · PLOS Digital Health · 3,724 citations

    We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.

  5. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods

    Professor Gary S. Collins, Karel G.M. Moons, Paula Dhiman, et al. · 2024 · BMJ · 2,839 citations

    The TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) statement was published in 2015 to provide the minimum reporting recommendations for studies developing or evaluating the performance of a prediction model. Methodological advances in the field of prediction have since included the widespread use of artificial intelligence (AI) powered by machine learning methods to develop prediction models. An update to the TRIPOD statement is thus needed. TRIPOD+AI provides harmonised guidance for reporting prediction model studies, irrespective of whether regression modelling or machine learning methods have been used. The new checklist supersedes

  6. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective

    Julia Amann, Alessandro Blasimme, Effy Vayena, et al. · 2020 · BMC Medical Informatics and Decision Making · 2,025 citations

    BACKGROUND: Explainability is one of the most heavily debated topics when it comes to the application of artificial intelligence (AI) in healthcare. Even though AI-driven systems have been shown to outperform humans in certain analytical tasks, the lack of explainability continues to spark criticism. Yet, explainability is not a purely technological issue, instead it invokes a host of medical, legal, ethical, and societal questions that require thorough exploration. This paper provides a comprehensive assessment of the role of explainability in medical AI and makes an ethical evaluation of what explainability means for the adoption of AI-driven tools into clinical practice. METHODS: Taking A

  7. The false hope of current approaches to explainable artificial intelligence in health care

    Marzyeh Ghassemi, Luke Oakden‐Rayner, Andrew L. Beam · 2021 · The Lancet Digital Health · 1,496 citations

    The black-box nature of current artificial intelligence (AI) has caused some to question whether AI must be explainable to be used in high-stakes scenarios such as medicine. It has been argued that explainable AI will engender trust with the health-care workforce, provide transparency into the AI decision making process, and potentially mitigate various kinds of bias. In this Viewpoint, we argue that this argument represents a false hope for explainable AI and that current explainability methods are unlikely to achieve these goals for patient-level decision support. We provide an overview of current explainability techniques and highlight how various failure cases can cause problems for deci

  8. Designing Theory-Driven User-Centric Explainable AI

    Danding Wang, Qian Yang, Ashraf Abdul, et al. · 2019 · 863 citations

    From healthcare to criminal justice, artificial intelligence (AI) is increasingly supporting high-consequence human decisions. This has spurred the field of explainable AI (XAI). This paper seeks to strengthen empirical application-specific investigations of XAI by exploring theoretical underpinnings of human decision making, drawing from the fields of philosophy and psychology. In this paper, we propose a conceptual framework for building human-centered, decision-theory-driven XAI based on an extensive review across these fields. Drawing on this framework, we identify pathways along which human cognitive patterns drives needs for building XAI and how XAI can mitigate common cognitive biases

  9. Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review

    Anna Markella Antoniadi, Yuhan Du, Yasmine Guendouz, et al. · 2021 · Applied Sciences · 606 citations

    Machine Learning and Artificial Intelligence (AI) more broadly have great immediate and future potential for transforming almost all aspects of medicine. However, in many applications, even outside medicine, a lack of transparency in AI applications has become increasingly problematic. This is particularly pronounced where users need to interpret the output of AI systems. Explainable AI (XAI) provides a rationale that allows users to understand why a system has produced a given output. The output can then be interpreted within a given context. One area that is in great need of XAI is that of Clinical Decision Support Systems (CDSSs). These systems support medical practitioners in their clini

  10. Explainable AI in Clinical Decision Support Systems: A Meta-Analysis of Methods, Applications, and Usability Challenges

    Qaiser Abbas, Woo-Min Jeong, Seung Won Lee · 2025 · Healthcare · 172 citations

    Background: Theintegration of artificial intelligence (AI) into clinical decision support systems (CDSSs) has significantly enhanced diagnostic precision, risk stratification, and treatment planning. AI models remain a barrier to clinical adoption, emphasizing the critical role of explainable AI (XAI). Methods: This systematic meta-analysis synthesizes findings from 62 peer-reviewed studies published between 2018 and 2025, examining the use of XAI methods within CDSSs across various clinical domains, including radiology, oncology, neurology, and critical care. Model-agnostic techniques such as visualization models like Gradient-weighted Class Activation Mapping (Grad-CAM) and attention mecha

  11. Co-design of Human-centered, Explainable AI for Clinical Decision Support

    Cecilia Panigutti, Andrea Beretta, Daniele Fadda, et al. · 2023 · ACM Transactions on Interactive Intelligent Systems · 113 citations

    eXplainable AI (XAI) involves two intertwined but separate challenges: the development of techniques to extract explanations from black-box AI models and the way such explanations are presented to users, i.e., the explanation user interface. Despite its importance, the second aspect has received limited attention so far in the literature. Effective AI explanation interfaces are fundamental for allowing human decision-makers to take advantage and oversee high-risk AI systems effectively. Following an iterative design approach, we present the first cycle of prototyping-testing-redesigning of an explainable AI technique and its explanation user interface for clinical Decision Support Systems (D

  12. Explainability in medicine in an era of AI-based clinical decision support systems

    Robin Pierce, Wim Van Biesen, Daan Van Cauwenberge, et al. · 2022 · Frontiers in Genetics · 88 citations

    The combination of "Big Data" and Artificial Intelligence (AI) is frequently promoted as having the potential to deliver valuable health benefits when applied to medical decision-making. However, the responsible adoption of AI-based clinical decision support systems faces several challenges at both the individual and societal level. One of the features that has given rise to particular concern is the issue of explainability, since, if the way an algorithm arrived at a particular output is not known (or knowable) to a physician, this may lead to multiple challenges, including an inability to evaluate the merits of the output. This "opacity" problem has led to questions about whether physician

  17. EXPLAINABLE AI FOR THERAPEUTIC DECISION-MAKING AND PRESCRIPTION SAFETY: A LONGITUDINAL FRAMEWORK FOR CLINICAL DECISION SUPPORT

    Albert Bacelar · 2026 · International Journal of Advanced Research

    Background: Clinical artificial intelligence has been evaluated mainly through diagnosis, triage, image interpretation, and isolated question answering. Therapeutic decision-making has a different structure: it converts clinical reasoning into action through drug choice, dose, route, timing, contraindication screening, monitoring, reassessment, escalation, de-escalation, and discontinuation. An explainable system that names a diagnosis but does not account for this action chain remains incomplete as clinical decision support.

  18. Explainable AI for Clinical Decision Support: A Study on Interpretable Models for Disease Diagnosis

    Ronak Goyal · 2026 · International Journal on Science and Technology

    This study explores the impact of explainable artificial intelligence (XAI) components on enhancing Diagnostic Accuracy (DA) within clinical decision support systems (CDSS). Focusing on three key predictors—Model Interpretability (MI), Clinician Trust (CT), and Case Complexity (CM)—the study utilizes primary data collected from 250 healthcare professionals in New York. A structured questionnaire measured all constructs on a 5-point Likert scale. Data analysis was conducted using R Studio, applying multiple regression to assess the relationships among variables. Results indicate that MI, CT, and CM each have a significant and positive effect on DA, with the model explaining approximately 65%

  19. Explainable Artificial Intelligence for Trustworthy Clinical Decision Support Systems

    Dr. Aarav Sharma · 2026 · European International Journal of Multidisciplinary Research and Management Studies

    The increasing integration of Artificial Intelligence (AI) into healthcare has transformed Clinical Decision Support Systems (CDSS) from rule-based advisory platforms into intelligent systems capable of diagnosing diseases, predicting clinical outcomes, recommending treatments, and optimizing healthcare workflows. Despite remarkable improvements in predictive performance through deep learning, ensemble learning, and generative artificial intelligence, the widespread clinical adoption of AI remains constrained by the limited transparency of complex machine learning models. Most high-performing AI algorithms function as "black-box" systems, providing highly accurate predictions while offering

  20. Development and Evaluation of an Explainable AI-Driven Decision Support System for Clinical and Operational Healthcare Management

    Preethi B · 2026 · Journal of Intelligent Decision Making and Information Science

    Healthcare organizations need to have an integrated decision-support system that can enhance patient risk assessment and optimize their use of resources. But the lack of transparency and separation between the use of AI in clinical use and administrative applications puts adoption challenges in the way.Conceive and evaluate a healthcare management explainable AI-based decision support system integrating clinical-risk prediction, resource-demand prediction and operational healthcare management. A synthetic multicentre dataset of 1,800 anonymized patient encounters was divided into a development set (n=1,260), a validation set (n=270) and an independent test set (n=270). The following models w

Write your paper with these sources

Folio is the integrity-first research workspace: search 200M+ papers, save sources, and write with citations that format themselves. Free for students and researchers.

Start writing free →