Research Article: A vessel-guided multi-task deep learning framework with visual interpretability for simultaneous retinal vessel segmentation and multi-disease classification from fundus images

Abstract:
Retinal diseases represent a leading cause of visual impairment and blindness worldwide, with early and accurate diagnosis being crucial for preventing irreversible vision loss. Although deep learning techniques have achieved significant advances in fundus image analysis, existing methods predominantly focus on single tasks, treating vessel segmentation and disease diagnosis as independent problems without fully leveraging their intrinsic relationships. Furthermore, the lack of transparency in deep model predictions limits clinical adoption; while full interpretability remains an open challenge, post-hoc techniques such as Grad-CAM can provide partial transparency by highlighting influential image regions. This study presents V-MNet, a vessel-guided multi-task deep learning framework that simultaneously achieves retinal vessel segmentation and multi-disease classification, and provides visual transparency through Grad-CAM-based class activation mapping to support clinical decision-making. The framework comprises three core modules: a shared encoder extracts multi-scale feature representations; a segmentation decoder employs a U-Net-style architecture to generate vessel masks; and a classification decoder incorporates an innovative vessel-guided mechanism that explicitly transfers structural priors from the segmentation branch to the classification task, enabling the model to precisely localize pathological regions. Concurrently, an integrated Grad-CAM module generates post-hoc class activation maps for each disease category, highlighting spatially relevant lesion regions for clinician review. Comprehensive experiments were conducted on four public datasets-RFMiD, ODIR-5K, DRIVE, and EyePACS-light-v2. Experiments demonstrate that V-MNet achieves a Dice coefficient of 0.831 and AUC of 0.985 for vessel segmentation tasks, and an average AUC of 0.978 with F1-score of 0.935 for multi-disease classification tasks, significantly outperforming single-task baseline models and existing state-of-the-art methods. Ablation studies systematically quantify the performance contributions of multi-task learning and the vessel-guided mechanism, confirming the effectiveness of the framework's core innovations. V-MNet demonstrates broad application potential as a computer-aided diagnostic tool by jointly leveraging vascular structure and disease pathology for superior performance and visual transparency. The vessel-guided multi-task design effectively exploits the intrinsic relationship between vessel segmentation and disease classification, while the integrated Grad-CAM module addresses the lack of model transparency, facilitating clinical adoption and supporting clinical decision-making.

Introduction:
Retinal diseases represent a leading cause of visual impairment and blindness worldwide, with early and accurate diagnosis being crucial for preventing irreversible vision loss. Although deep learning techniques have achieved significant advances in fundus image analysis, existing methods predominantly focus on single tasks, treating vessel segmentation and disease diagnosis as independent problems without fully leveraging their intrinsic relationships. Furthermore, the lack of transparency in deep model…

Read more