Multimodal Bronchoscopic Video Analysis System for Early Lung Cancer Detection

Early detection of lung cancer is crucial as it significantly improves survival rates by facilitating timely and effective treatment. Lung cancer often begins as bronchial lesions developing along the airway walls. Bronchoscopy is the minimally invasive method of choice for detecting such lesions. Currently, three complementary bronchoscopic video modalities have been utilized for this purpose: white-light bronchoscopy (WLB), narrow-band imaging (NBI), and autofluorescence bronchoscopy (AFB). Unfortunately, current practice forces the clinician to manually examine each video source and later interactively correlate the results of these exams to make final lesion decisions. Because of the lack of effective tools for multimodal endoscopic video analysis, this proves to be an extremely time-consuming, error-prone process, making it impractical for common clinical use. To address this problem, we propose a multimodal video analysis and synchronization system that enables efficient analysis of multimodal bronchoscopic videos for early cancer lesion detection. The system provides methods for planning and guiding a straightforward multimodal airway exam through the major airways. Subsequent video processing methods then draw on deep-learning-based techniques to identify candidate single-mode bronchial lesions. Next, a synchronization/registration pipeline registers all bronchoscopic video data to a reference 3D airway tree model derived from a patient's X-ray computed tomography (CT) scan. This finally facilitates interactive graphical visualization and interaction with all processed multimodal data. Results with lung cancer patient studies indicate the system's promise for efficient, effective video analysis and lesion detection.