Abstract
Background:
Artificial intelligence (AI) technology has been used for finding lesions via gastrointestinal endoscopy. However, there were few AI-associated studies that discuss bronchoscopy.
Objectives:
To use convolutional neural network (CNN) to recognize the observed anatomical positions of the airway under bronchoscopy.
Design:
We designed the study by comparing the imaging data of patients undergoing bronchoscopy from March 2022 to October 2022 by using EfficientNet (one of the CNNs) and U-Net.
Methods:
Based on the inclusion and exclusion criteria, 1527 clear images of normal anatomical positions of the airways from 200 patients were used for training, and 475 clear images from 72 patients were utilized for validation. Further, 20 bronchoscopic videos of examination procedures in another 20 patients with normal airway structures were used to extract the bronchoscopic images of normal anatomical positions to evaluate the accuracy for the model. Finally, 21 respiratory doctors were enrolled for the test of recognizing corrected anatomical positions using the validating datasets.
Results:
In all, 1527 bronchoscopic images of 200 patients with nine anatomical positions of the airway, including carina, right main bronchus, right upper lobe bronchus, right intermediate bronchus, right middle lobe bronchus, right lower lobe bronchus, left main bronchus, left upper lobe bronchus, and left lower lobe bronchus, were used for supervised machine learning and training, and 475 clear bronchoscopic images of 72 patients were used for validation. The mean accuracy of recognizing these 9 positions was 91% (carina: 98%, right main bronchus: 98%, right intermediate bronchus: 90%, right upper lobe bronchus: 91%, right middle lobe bronchus 92%, right lower lobe bronchus: 83%, left main bronchus: 89%, left upper bronchus: 91%, left lower bronchus: 76%). The area under the curves for these nine positions were >0.98. In addition, the accuracy of extracting the images via the video by the trained model was 94.7%. We also conducted a deep learning study to segment 10 segment bronchi in right lung, and 8 segment bronchi in Left lung. Because of the problem of radial depth, only segment bronchi distributions below right upper bronchus and right middle bronchus could be correctly recognized. The accuracy of recognizing was 84.33 ± 7.52% by doctors receiving interventional pulmonology education in our hospital over 6 months.
Conclusion:
Our study proved that AI technology can be used to distinguish the normal anatomical positions of the airway, and the model we trained could extract the corrected images via the video to help standardize data collection and control quality.
Introduction
The bronchoscope was invented by Killian, 1 and with the technological advancements over the past century, many novel technologies, such as endobronchial ultrasound (EBUS), 2 electromagnetic navigation bronchoscopy (ENB),3–5 radial EBUS (r-EBUS),6,7 ultrathin bronchoscopy, 8 virtual bronchoscopic navigation (VBN),9,10 trans-parenchymal nodule access (TPNA), 11 and robotic bronchoscopy12–14 have been developed to assist clinicians in the diagnosis and treatment of various diseases.
Nowadays, images and videos under gastrointestinal endoscopy are widely used for deep learning studies to find the lesions of tumor and infection.15–20 This implies that artificial intelligence (AI) technology can help young doctors discover these lesions. However, there are few AI-associated studies discussing bronchoscopy. Previously, Matava et al. 21 conducted an AI-based bronchoscopic study to distinguish between vocal cord and tracheal, while another study conducted by anesthesiologists, aiming to help define the various intubation positions, showed that utilizing AI could distinguish between carina and main bronchi. 22
There exist huge development prospects accompanied with tremendous challenges in AI-associated studies about bronchoscopy. In our views, the characteristics of the images under bronchoscopy are different from those under gastrointestinal endoscopy. On the one hand, the background images under bronchoscopy are more difficult than those in gastrointestinal endoscopy because of diverse lobes and bronchi. 23 However, the background images under gastrointestinal endoscopy only contained one pathway because of the physiological characteristics of the human body. On the other hand, distinguishing the anatomical positions is also important for the study of bronchoscopy.24–26 Our study attempted to use convolutional neural network (CNN) to automatically differentiate the anatomical positions of the airway under bronchoscopy, which could help extract the bronchoscopic images of trained anatomical positions as well as increase the learning efficiency of anatomical positions under bronchoscopy, to shorten the training period of doctors in respiratory intervention.
Methods
This was an AI-associated study about classification and segmentation by bronchoscopic images. We collected clinical and images data from 272 patients (141 male and 131 female) who undergoing bronchoscopy from March 2022 to October 2022 in the First Affiliated Hospital of Guangzhou Medical University. Their clinical reports, computed tomography, and bronchoscopic images showed that their airway structure was normal. We retrospectively collected their bronchoscopic images of the following nine anatomical positions of the airway: carina, right main bronchus, right upper lobe bronchus, right intermediate bronchus, right middle lobe bronchus, right lower lobe bronchus, left main bronchus, left upper lobe bronchus, left lower lobe bronchus, and using these images for classification learning by CNN. Furthermore, 20 bronchoscopic videos of examination procedures from 20 other patients with normal airway structures were used to extract the images of normal anatomical positions to evaluate the accuracy of the AI model.
We conducted a segment learning by U-Net for recognizing 10 segment bronchi – anterior segment, posterior segment, apical segment, medial segment, lateral segment, dorsal segment ( ‘dorsal segment’ is the same as ‘superior segment’), medial basal segment, anterior basal segment, lateral basal segment, and posterior basal segment – in right lung, and eight segment bronchi – superior lingular segment, inferior lingular segment, anterior segment, apical posterior segment, dorsal segment (‘dorsal segment’ is the same as ‘superior segment’), anterior medial basal segment, lateral basal segment, and posterior basal segment in left lung.
Finally, we also conducted a small test to recognize the corrected anatomical positions under bronchoscopy by using a validation dataset for the respiratory doctors. We randomly chose 50 images from the validation dataset for the test; the randomized method was computer-generated randomized number. This study was a retrospective study, and the study protocol was approved by the Ethics Committee of the First Affiliated Hospital of Guangzhou Medical University (approval number: ES-2023-028-01).
Inclusion and exclusion criteria
The inclusion criteria were as follows: (1) patients aged >14 years; (2) patients who underwent bronchoscopy with clear image data (at least containing three anatomic airway positions of the following: carina, right main bronchus, right upper lobe bronchus, right intermediate bronchus, right middle lobe bronchus, right lower lobe bronchus, left main bronchus, left upper lobe bronchus, and left lower lobe bronchus; (3) patients who did not undergo lung operations previously;(4) patients who did not have any lesion in the airway; (5) patients who did not have stenosis or atresia in these nine airway anatomical positions. (6) patients who did not undergo therapies (such as stents) during the examinations.
The inclusion criteria of clear images were: (1) The bronchoscopic pictures of these nine airway positions should contain complete anatomical structures (e.g. the images of carina should contain cartilage rings and membrane, the images of right upper lobe bronchus should contain the openings of three segments). We excluded the images with tracheobronchial variations. Figure 1 shows the sample figures of these nine anatomical positions. (2) The images should hardly contain secretions that make the anatomical characteristics of the airway difficult to be identified by AI. (3) if the images described the same locations in a single person, we only retained one chosen image.

The sample images of the used images of normal anatomical positions of the airway.
The inclusion criteria of the enrolled doctors for the test to recognize the corrected anatomical positions by using the validation datasets were as follows: (1) they should have worked as respiratory doctors for over 5 years, (2) they should have worked for high-level hospitals in China (Grade-A tertiary hospital), (3) they joined our hospital for advanced studies about interventional pulmonology.
Statistical analysis
Categorical variables were expressed as number and percentage, and continuous variables were expressed as mean ± standard deviation. Python 3.7, EfficientNet 27 (one of the CNNs), and U-Net were performed. The EfficientNet could ignore the influence of the pixel. 27 The bronchoscopic images of all the airway anatomical positions were all pre-processed by Gaussian filter (which could smooth the image and filter the noise), 20 graphic lightening, and normalizing. The confusion matrix and receiver operating characteristic curves were calculating to show the accuracies.
Results
Baseline characteristics of these patients for training and validation
Table 1 describes the baseline characteristics of the 200 patients with clear bronchoscopic pictures in the training dataset. In this dataset, the bronchoscopic images from 104 male and 96 female patients were used. Table 1 shows the baseline characteristics of the 72 patients with clear bronchoscopic images in validation dataset, wherein the bronchoscopic images of 37 male and 35 female patients were utilized for validation.
The baseline characteristics of the included patients.
In the training dataset, these normal bronchoscopic images of airway anatomical structures contained carina (n = 195), right main bronchus (n = 189), right upper lobe bronchus (n = 145), right intermediate bronchus (n = 190), right middle lobe bronchus (n = 171), right lower lobe bronchus (n = 164), left main bronchus (n = 170), left upper lobe bronchus (n = 160), and left lower lobe bronchus (n = 143) (Table 2).
The characteristics of the included images.
In the validation dataset, the included bronchoscopic pictures of normal airway anatomical structures were the carina (n = 65), right main bronchus (n = 62), right upper lobe bronchus (n = 43), right intermediate bronchus (n = 65), right middle lobe bronchus (n = 53), right lower lobe bronchus (n = 50), left main bronchus (n = 56), left upper lobe bronchus (n = 47), and left lower lobe bronchus (n = 34).
The validations using images
The mean accuracy of identifying these nine anatomical positions of the airway was 91%, (carina: 98%, right main bronchus: 98%, right intermediate bronchus: 90%, right upper lobe bronchus:91%, right middle lobe bronchus 92%, right lower lobe bronchus: 83%, left main bronchus: 89%, left upper bronchus: 91%, left lower bronchus: 76%). Figure 2 shows the confusion matrix of the accuracies of these nine anatomical positions of the airway.

The accuracies of recognizing the nine anatomical positions of the airway by AI.
The area under the curves of these nine positions were all over 0.98 (carina: 0.999, right main bronchus: 1.000, right intermediate bronchus: 0.996, right upper lobe bronchus:0.999, right middle lobe bronchus:0.996, right lower lobe bronchus: 0.994, left main bronchus: 0.990, left upper bronchus: 0.997, left lower bronchus: 0.986) (Figure 3). The training loss and validation loss are shown in Figure 4.

The receiver operating characteristic curves (ROCs) of recognizing the nine anatomical positions of the airway by artificial intelligence (AI).

The training loss and validation loss curves.
Recognizing the segment bronchi by using segmenting algorithm
Only segment bronchi distributions below right upper bronchus [anterior segment (90.7%), posterior segment (97.7%), apical segment (93%)]; right middle bronchus [medial segment (98.1%), lateral segment (86.8%)], and two dorsal segments below right lower lobe and left lower lobe bronchi (100% and 94.1%) could be correctly recognized]. Other segments could not be evaluated because of the radial depth, and the accuracies were difficult to calculate and assess (Table 3)
The accuracies of recognizing the segment bronchi.
NA: not available; because of the radial depth of images, and the accuracies were difficult to calculate; ‘dorsal segment’ is the same as ‘superior segment’.
The validations using video data
We also collected the bronchoscopic video data of another 20 patients and used the training model to extract the images of the specific anatomical positions. The results showed that the accuracy of the model was 94.7%. Figure 5 shows the sample images of the anatomical positions extracted by the AI model from the bronchoscopic videos. The segmenting model was used for the identification of 18 segment bronchi (Supplemental video data).

The sample images of the trained model to extract via video data.
Doctors’ test with the validation dataset
We used the validating dataset (total, 475 pictures) as a test for the doctors to recognize the corrected anatomical positions under the bronchoscopy. Their backgrounds of interventional pulmonary were various, and the prior bronchoscopies performance were also too subjective. Hence, we did not obtain this information. These doctors all received high school education, and have been working as respiratory doctors for over 5 years. Therefore, we only divided the enrolled doctors into three groups based on the duration of receipt of interventional pulmonology education in our hospital (group A ⩽ 3 months, group B: 3–6 months, group C: > 6 months). The baseline characteristics of these doctors are shown in Table 4. Group A showed significantly lower scores than Group B (79.33 ± 9.50% vs 53.33 ± 7.94%, P < 0.001), and group C (84.33 ± 7.52% vs 53.33 ± 7.94%, P < 0.001) (Figure 6); However, Group C did not get significantly higher scores than Group B.
The baseline characteristics of the enrolled doctors.

The accuracies of recognizing the corrected airway positions by the doctors.
Discussion
Our study showed that the total accuracy for discerning all nine anatomical positions of the airway, namely, the carina, right main bronchus, right upper lobe bronchus, right intermediate bronchus, right middle lobe bronchus, right lower lobe bronchus, left main bronchus, left upper lobe bronchus, and left lower lobe bronchus was >90%. Furthermore, we could use AI technology to detect the correct images of airway anatomical positions via video data (accuracy was >90%), which could help doctors automatically extract the needed bronchoscopic images for clinical practice. The accuracy of the trained AI model (91%) was similar to the reached the levels of accuracy of senior physicians with advanced training (accuracy: 84.33%). We used the static images of these nine anatomical positions that were relatively challenging to manually identify. However, in some hospitals, only high-level doctors have the ability/authorization to use bronchoscopes to diagnose or treat respiratory illnesses. Hence, young doctors should pass long-time primary clinical practice.
The tracheobronchial tree plays an important role in delivering air from the trachea down to the pulmonary alveolus, which facilitates gas exchange. 23 A previous study demonstrated an incidence of 1 to 12% of the tracheobronchial variations. 23 Three embryogenic theories have been proposed about tracheobronchial variations, which contained the reduction, migration, and selection. 23 The variations of tracheal bronchi could be divided into four groups according to their morphologic pattern: displaced, rudimentary, supernumerary, and anomalous right upper lobe bronchus. 23 However, because of the rare incidence of tracheobronchial variation, we only used the bronchoscopic images of normal airway structures.
After the supervised machine learning, the training model could identify the correct anatomical positions of the airway and classify the images that could promptly and automatically collect image data during examination. The algorithm research of CNN went through ALexNet, 28 VGG, 29 ResNet, 30 and EfficientNet. 27 Now, the EfficientNet added the attention mechanism with high efficient, and could ignore the influence of pixel. 27
The mechanisms of the machine recognized the anatomical positions of airway, but the learned anatomical characteristics from the images remain unknown. However, experienced professors could recognize the immediate anatomical positions regardless of where the bronchoscope went during the examination. Experienced professors recognized the anatomical positions by cartilage rings, membrane, segmentations, and somethings can’t able to express. As an AI technology with feedback function similar to humans, in our view, deep learning could also identify the corrected positions.
In the future, we should further utilize high-performance computers for real-time teaching. We could use this AI model to show the names of every lobe bronchus, as well as some segmental bronchi when they conduct bronchoscopy, to train doctors.
The study conducted by Li et al. 31 also recognizes the anatomical positions under bronchoscopy. However, there were several differences between our studies. The AI model we used was EfficientNet, but they used VGG16, ResNet 50, and they used more training pictures than us (28,441 vs 1527). However, the accuracies were not superior to ours, perhaps because the model they used did not fit the pictures of bronchoscopy. Moreover, as a scientific study, they did not collect the data of the enrolled patients, and the inclusion criteria of patients were lack. Thus, in our view, the segment bronchi could not be better recognized which could influence the efficacy of the AI model (in their study, the accuracies for the basal segment bronchi were <85%). Accordingly, we used segment learning by U-Net to recognize the segment bronchi in the images of lobe bronchi.
The limitations of our study were the following. First, we only included the bronchoscopic images of normal airway anatomical positions, so the model could not identify the positions in case of existing lesions (such as neoplasm). Second, it is only a basic study for AI study of bronchoscopy, thus, we subsequently plan to use the bronchoscopic images of lesions to distinguish diseases. However, this study displayed an applicable prospect for grading diagnosis via bronchoscopy, which could let the computer know the locations of the lesions by bronchoscopic images. Third, the number of the included subjects was quite small, and only patients belonging to Chinese ethnicity were included. Beder et al. 32 observed over 40% tracheobronchial variations in Turkish persons, which imply that more imaging and varied population data should be included into supervised machine learning for differentiating these variations of anatomical positions. Fourth, because of the problem of radial depth, some segmental bronchi (such as basal segments) could not be better recognized. In the future investigations, these aspects should be considered. Finally, we did not calculate the sample size selected in this study.
Conclusions
Our study showed that we can use the AI technology to differentiate normal anatomical positions of the airway and extracting the corrected images via the video, which can help to standardize data collection and ensure quality control.
Footnotes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
