Presentations

Possibilities of Enhancing Medical Image Segmentation with Modified Visual Transformers and Attention-based Explanations

Volkov E.N., Averkin A.N.

Dubna State University, 141982, Dubna, Moscow region, Universitetskaya 19

One of the most popular tasks of automated medical image analysis using artificial neural networks is the task of image segmentation. There is a large number of convolutional neural network (CNN) architectures used for this purpose. However, the use of CNN has a number of problems concerning the structure of the analyzed images (scaling, instability to geometric transformations, etc.). One of the solutions to such problems is to build a neural network architecture based on modified visual transformer blocks [1]. This architecture allows both to generalize texture features of different scales by effectively dividing image patches and to model spatial relations between them by using attention mechanisms. In addition, the attention mechanism can also be used to generate visual explanations as a method of explainable artificial intelligence to improve the interpretability of the model and increase the trust of the clinicians in the technology as a whole [2, 3]. Proposed architecture was tested on the task of segmentation of diabetic retinopathy biomarkers on fundus images. The proposed approach allowed to outperform the results of fundus image segmentation quality by standard U-Net type architectures.

The research was carried out within the state assignment of Ministry of Science and Higher Education of the Russian Federation (theme no. 124092700007-4).

References

1. Volkov E. N., Averkin A. N. Possibilities of Using Visual Transformers in the Classification of Ophthalmologic Diseases. URL: https://www.mce.biophys.msu.ru/rus/archive/abstracts/sect452302/doc460933/.

2. Volkov E. N., Averkin A. N. Explainable artificial intelligence in medical image analysis: State of the art and prospects //XXVI International Conference on Soft Computing and Measurements (SCM). IEEE, 2023. Pp. 134-137. DOI: 10.1109/SCM58628.2023.10159033.

3. Averkin A. N., Volkov E. N., Yarushev S. A. Explainable artificial intelligence in deep learning neural nets-based digital images analysis //Journal of Computer and Systems Sciences International Vol. 63, No. 1, 2024. Pp. 175-203. DOI: 10.1134/S1064230724700138.

• abstract in English (PDF)