TAMI
Transparent Artificial Medical Intelligence Artificial IntelligenceBioengineering
Description
The current rise of Artificial Intelligence (AI) has brought exciting advances in machine learning, which have raised its capabilities across a suite of applications. The advances have been facilitated by the availability of large and diverse data sets, improved algorithms that find patterns in torrents of data, and new levels of mathematical computing power. As a result of these advances, systems which only a few years ago performed at noticeably below-human levels can now outperform humans at some specific tasks. In health care, AI is challenging doctors on their home turf. Despite the spectacular successes of deep learning during the current rising wave of AI, there remain huge challenges, and progress is needed in areas of public concern that constraint its wider use. The European Commission Report on a comprehensive European industrial policy on artificial intelligence and robotics states that technology needs to be ethical-by-design and characteristics such as transparency and explainability need to be embedded in the development of AI.
The TAMI project brought explainability to AI methods supporting clinical professionals from screening to daily clinical practice, with a special focus on decision support systems relying on imagiological data. Thus, the aim of the project was to create a new platform for commercial, scientific and academic use that will provide "consumers" access to results and explanations of registered diagnostic orders, filtered data sets access for investigators or scientists and a knowledge base for academic purposes.
Scientific Advances
In order to achieve such objective, the project was based on the following specific objectives involving the development of research in the following areas:
a) quantitative methods to objectively assess and compare different explanations of the automatic decisions;
b) methods to generate better explanations, providing variety in the explanations, adapting the explanations to who will consume them and explaining multimodal decisions;
c) novel visualization solutions for interpretations of decisions based on imagiological data.
In order to accomplish that, TAMI used clinical data, from structured to image data, in order to design and validate interpretable machine learning models. During the project, different multimodal settings were tested to enable a better understanding of the AI-based decisions. Moreover, the algorithms were designed to generate self-explanatory AI-based decisions, minimise bias, and act ethically in their context.
Proof-of-concepts and demonstrators of how to integrate the researched explainable AI into workflows of cervical cancer treatment, pathology detection in chest X-Ray images in a screening environment, and glaucoma detection in retinal fundus images were developed to validate the algorithmic solutions.
Results
In the area of CXR, several state-of-the-art approaches for detecting pathology/normality were developed and then validated in public databases. It was shown that an approach based on object detection can achieve higher performance with a greater degree of explainability regarding regions of the image that present abnormalities. Complementary, an automatic method for lung segmentation using a U-Net was also developed, allowing the use of anatomical regions to filter abnormality predictions. Subsequently, a new solution was implemented based on spatial attention mechanisms, which facilitate, in an unsupervised way, the pathology classification algorithms to focus on the most relevant regions of the image.
To improve the explanations provided by the algorithms, methods were developed that use information on the location of radiological findings of the different pathologies present in the annotated images to select, in the image under evaluation, the regions of the image that had the greatest contribution to the algorithm's decision. It was also explored how complementary information through an eye-tracking system can improve pathology detection by automatic systems and their explainability. A multi-resolution approach was also explored, consisting of using versions of the input image with different resolutions, and it was verified that there is an obvious relationship between the size of the lesion and image resolution, where the regions that most contributed to the decision are selected. The proposed approach's innovation consists of using a multi-resolution architecture but with several parameters equivalent to a mono-resolution model.
Several efforts were made to evaluate the explanations of the classification models proposed for detecting various thoracic pathologies to identify which strategies are the most successful. To this goal, two lines of evaluation were established: assessing the model's reliability and selecting the best visual explanation technique. Finally, a new explanation technique proposed in this project, called Grid-CAM, was also evaluated. Finally, it was evaluated how the performance of these explainability methods changed when there was a change in the dataset used, having been demonstrated that for several networks and explainability methods, validation in an external dataset led to a decrease in performance and, in future work, it would be necessary to study how to make these methods more robust to this dataset bias.
Regarding the generation of explanations based on real examples while maintaining privacy, the study of the generation of artificial CXR images through generative artificial networks (GANs) was explored. It was shown that a few-shot GAN, designed to learn from a limited number of examples, can generate realistic chest CXR images, thus enhancing the generation of images of rare pathologies. An approach was also designed to retrieve images from a database that are similar to the image under analysis, based on information obtained from lesions detected in that image and not the complete image. The use of artificial contrastive explanations was also explored, through the development of artificial generative networks for inpainting, used to generate a normal version of a pathological x-ray after identifying the lesion of interest, allowing the visualization of the radiological characteristics that determine that lesion. Heatmap explanation techniques, one of the methods most often found in literature, are often variations of Grad-CAM. One of the disadvantages of this method is the fact that there is no guarantee of correspondence between an activation zone and the corresponding zone of the original image. To try to overcome this difficulty, well known to the scientific community, a method was proposed called Grid-CAM, which is inspired by Grad-CAM but takes this correspondence failure into account during image rescaling. A new explanation method based on heatmaps was also proposed, which attempts to represent the maps more intuitively, adding an extra validation step that indicates whether the user should trust the initial explanation. This method was called Confident-CAM.
In the area of cervical cytology, observational inquiry sessions (contextual inquiry) were carried out with specialists from IPO-Porto to elicit the mental flows of specialists in cervical-vaginal cytology in the context of analysis, annotation, and classification of images of visual fields of cytological samples, duly anonymized. Automated and interpretable suitability assessment of cervical cytology samples was also explored, and a new approach was proposed based on an SSD REsNet50 model for detecting and counting different types of nuclei present in LBC samples. Automated cell counting was merged with the suitability criteria proposed by The Bethesda System, with the automatic decision for the adequacy of the sample accompanied by the respective explanation.
In terms of cervical cytology and glaucoma, state-of-the-art surveys were carried out at various levels, namely: i) approaches for synthetic data generation; ii) methods that ensure privacy in medical applications (eg differential privacy, homomorphic or federated encryption learning) ; iii) methodologies that seek to relate visual and textual explanations; and iv) methodologies that allow generating similar and contradictory visual examples of classified cases. We also proposed a new model-agnostic approach to generating example-based explanations for post-hoc evaluation of cervical cytology models. The proposed pipeline is based on feature extraction maps predictions from pre-trained models, expanding the capabilities of these models by additionally providing example-based explanations (similar examples) that justify each prediction.
The TAMI project brought explainability to AI methods supporting clinical professionals from screening to daily clinical practice, with a special focus on decision support systems relying on imagiological data. Thus, the aim of the project was to create a new platform for commercial, scientific and academic use that will provide "consumers" access to results and explanations of registered diagnostic orders, filtered data sets access for investigators or scientists and a knowledge base for academic purposes.
Scientific Advances
In order to achieve such objective, the project was based on the following specific objectives involving the development of research in the following areas:
a) quantitative methods to objectively assess and compare different explanations of the automatic decisions;
b) methods to generate better explanations, providing variety in the explanations, adapting the explanations to who will consume them and explaining multimodal decisions;
c) novel visualization solutions for interpretations of decisions based on imagiological data.
In order to accomplish that, TAMI used clinical data, from structured to image data, in order to design and validate interpretable machine learning models. During the project, different multimodal settings were tested to enable a better understanding of the AI-based decisions. Moreover, the algorithms were designed to generate self-explanatory AI-based decisions, minimise bias, and act ethically in their context.
Proof-of-concepts and demonstrators of how to integrate the researched explainable AI into workflows of cervical cancer treatment, pathology detection in chest X-Ray images in a screening environment, and glaucoma detection in retinal fundus images were developed to validate the algorithmic solutions.
Results
In the area of CXR, several state-of-the-art approaches for detecting pathology/normality were developed and then validated in public databases. It was shown that an approach based on object detection can achieve higher performance with a greater degree of explainability regarding regions of the image that present abnormalities. Complementary, an automatic method for lung segmentation using a U-Net was also developed, allowing the use of anatomical regions to filter abnormality predictions. Subsequently, a new solution was implemented based on spatial attention mechanisms, which facilitate, in an unsupervised way, the pathology classification algorithms to focus on the most relevant regions of the image.
To improve the explanations provided by the algorithms, methods were developed that use information on the location of radiological findings of the different pathologies present in the annotated images to select, in the image under evaluation, the regions of the image that had the greatest contribution to the algorithm's decision. It was also explored how complementary information through an eye-tracking system can improve pathology detection by automatic systems and their explainability. A multi-resolution approach was also explored, consisting of using versions of the input image with different resolutions, and it was verified that there is an obvious relationship between the size of the lesion and image resolution, where the regions that most contributed to the decision are selected. The proposed approach's innovation consists of using a multi-resolution architecture but with several parameters equivalent to a mono-resolution model.
Several efforts were made to evaluate the explanations of the classification models proposed for detecting various thoracic pathologies to identify which strategies are the most successful. To this goal, two lines of evaluation were established: assessing the model's reliability and selecting the best visual explanation technique. Finally, a new explanation technique proposed in this project, called Grid-CAM, was also evaluated. Finally, it was evaluated how the performance of these explainability methods changed when there was a change in the dataset used, having been demonstrated that for several networks and explainability methods, validation in an external dataset led to a decrease in performance and, in future work, it would be necessary to study how to make these methods more robust to this dataset bias.
Regarding the generation of explanations based on real examples while maintaining privacy, the study of the generation of artificial CXR images through generative artificial networks (GANs) was explored. It was shown that a few-shot GAN, designed to learn from a limited number of examples, can generate realistic chest CXR images, thus enhancing the generation of images of rare pathologies. An approach was also designed to retrieve images from a database that are similar to the image under analysis, based on information obtained from lesions detected in that image and not the complete image. The use of artificial contrastive explanations was also explored, through the development of artificial generative networks for inpainting, used to generate a normal version of a pathological x-ray after identifying the lesion of interest, allowing the visualization of the radiological characteristics that determine that lesion. Heatmap explanation techniques, one of the methods most often found in literature, are often variations of Grad-CAM. One of the disadvantages of this method is the fact that there is no guarantee of correspondence between an activation zone and the corresponding zone of the original image. To try to overcome this difficulty, well known to the scientific community, a method was proposed called Grid-CAM, which is inspired by Grad-CAM but takes this correspondence failure into account during image rescaling. A new explanation method based on heatmaps was also proposed, which attempts to represent the maps more intuitively, adding an extra validation step that indicates whether the user should trust the initial explanation. This method was called Confident-CAM.
In the area of cervical cytology, observational inquiry sessions (contextual inquiry) were carried out with specialists from IPO-Porto to elicit the mental flows of specialists in cervical-vaginal cytology in the context of analysis, annotation, and classification of images of visual fields of cytological samples, duly anonymized. Automated and interpretable suitability assessment of cervical cytology samples was also explored, and a new approach was proposed based on an SSD REsNet50 model for detecting and counting different types of nuclei present in LBC samples. Automated cell counting was merged with the suitability criteria proposed by The Bethesda System, with the automatic decision for the adequacy of the sample accompanied by the respective explanation.
In terms of cervical cytology and glaucoma, state-of-the-art surveys were carried out at various levels, namely: i) approaches for synthetic data generation; ii) methods that ensure privacy in medical applications (eg differential privacy, homomorphic or federated encryption learning) ; iii) methodologies that seek to relate visual and textual explanations; and iv) methodologies that allow generating similar and contradictory visual examples of classified cases. We also proposed a new model-agnostic approach to generating example-based explanations for post-hoc evaluation of cervical cytology models. The proposed pipeline is based on feature extraction maps predictions from pre-trained models, expanding the capabilities of these models by additionally providing example-based explanations (similar examples) that justify each prediction.