EXPLAINABLE MEDICAL IMAGING AI FOR CANCER DIAGNOSIS: LINKING SALIENCY MAPS WITH RADIOLOGIST DECISION-MAKING AND CLINICAL OUTCOMES

Abstract

This study proposes an explainable medical imaging AI framework for cancer diagnosis that embeds visualization of saliency maps, analysis of the steps made by radiologists during the diagnosis and clinical outcome analysis. The objective is to increase the transparency of the model, the confidence in the diagnosis, the clinical interpretability and the predictive value of the model. A multimodal dataset consisting of CT, MRI and digital pathology images of patients with different cancer types was used. Images were pre-processed prior to training of the model.The images were preprocessed, augmented and normalized before being given as input to the model. A deep learning framework based on convolutional neural networks (CNNs), attention mechanisms, and gradient-based explainability tools like Grad-CAM and saliency mapping was designed. Model explanations were assessed based on two criteria: interpretability and localization accuracy (compared to annotations by expert radiologists). The accuracy, sensitivity, specificity, precision, F1-score and AUC-ROC were used to assess the performance of the diagnosis. Correlation analysis between the associations and clinical outcome (disease progression and response to therapy) was also performed.