Convolutional neural network classifier for distinguishing Barrett's esophagus and neoplasia endomicroscopy images.
Jisu Hong ,Bo-Yong Park ,Hyunjin Park
Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference
Barrett's esophagus is a diseased condition with abnormal changes of the cells in the esophagus. Intestinal metaplasia (IM) and gastric metaplasia (GM) are two sub-classes of Barrett's esophagus. As IM can progress to the esophageal cancer, the neoplasia (NPL), developing methods for classifying between IM and GM are important issues in clinical practice. We adopted a deep learning (DL) algorithm to classify three conditions of IM, GM, and NPL based on endimicroscopy images. We constructed a convolutional neural network (CNN) architecture to distinguish among three classes. A total of 262 endomicroscopy imaging data of Barrett's esophagus were obtained from the international symposium on biomedical imaging (ISBI) 2016 challenge. 155 IM, 26 GM and 55 NPL cases were used to train the architecture. We implemented image distortion to augment the sample size of the training data. We tested our proposed architecture using the 26 test images that include 17 IM, 4 GM and 5 NPL cases. The classification accuracy was 80.77%. Our results suggest that CNN architecture could be used as a good classifier for distinguishing endomicroscopy imaging data of Barrett's esophagus.
Computer-aided detection of early neoplastic lesions in Barrett's esophagus.
van der Sommen Fons,Zinger Svitlana,Curvers Wouter L,Bisschops Raf,Pech Oliver,Weusten Bas L A M,Bergman Jacques J G H M,de With Peter H N,Schoon Erik J
BACKGROUND AND STUDY AIMS:Early neoplasia in Barrett's esophagus is difficult to detect and often overlooked during Barrett's surveillance. An automatic detection system could be beneficial, by assisting endoscopists with detection of early neoplastic lesions. The aim of this study was to assess the feasibility of a computer system to detect early neoplasia in Barrett's esophagus. PATIENTS AND METHODS:Based on 100 images from 44 patients with Barrett's esophagus, a computer algorithm, which employed specific texture, color filters, and machine learning, was developed for the detection of early neoplastic lesions in Barrett's esophagus. The evaluation by one endoscopist, who extensively imaged and endoscopically removed all early neoplastic lesions and was not blinded to the histological outcome, was considered the gold standard. For external validation, four international experts in Barrett's neoplasia, who were blinded to the pathology results, reviewed all images. RESULTS:The system identified early neoplastic lesions on a per-image analysis with a sensitivity and specificity of 0.83. At the patient level, the system achieved a sensitivity and specificity of 0.86 and 0.87, respectively. A trade-off between the two performance metrics could be made by varying the percentage of training samples that showed neoplastic tissue. CONCLUSION:The automated computer algorithm developed in this study was able to identify early neoplastic lesions with reasonable accuracy, suggesting that automated detection of early neoplasia in Barrett's esophagus is feasible. Further research is required to improve the accuracy of the system and prepare it for real-time operation, before it can be applied in clinical practice.
Deep-Learning System Detects Neoplasia in Patients With Barrett's Esophagus With Higher Accuracy Than Endoscopists in a Multistep Training and Validation Study With Benchmarking.
de Groof Albert J,Struyvenberg Maarten R,van der Putten Joost,van der Sommen Fons,Fockens Kiki N,Curvers Wouter L,Zinger Sveta,Pouw Roos E,Coron Emmanuel,Baldaque-Silva Francisco,Pech Oliver,Weusten Bas,Meining Alexander,Neuhaus Horst,Bisschops Raf,Dent John,Schoon Erik J,de With Peter H,Bergman Jacques J
BACKGROUND & AIMS:We aimed to develop and validate a deep-learning computer-aided detection (CAD) system, suitable for use in real time in clinical practice, to improve endoscopic detection of early neoplasia in patients with Barrett's esophagus (BE). METHODS:We developed a hybrid ResNet-UNet model CAD system using 5 independent endoscopy data sets. We performed pretraining using 494,364 labeled endoscopic images collected from all intestinal segments. Then, we used 1704 unique esophageal high-resolution images of rigorously confirmed early-stage neoplasia in BE and nondysplastic BE, derived from 669 patients. System performance was assessed by using data sets 4 and 5. Data set 5 was also scored by 53 general endoscopists with a wide range of experience from 4 countries to benchmark CAD system performance. Coupled with histopathology findings, scoring of images that contained early-stage neoplasia in data sets 2-5 were delineated in detail for neoplasm position and extent by multiple experts whose evaluations served as the ground truth for segmentation. RESULTS:The CAD system classified images as containing neoplasms or nondysplastic BE with 89% accuracy, 90% sensitivity, and 88% specificity (data set 4, 80 patients and images). In data set 5 (80 patients and images) values for the CAD system vs those of the general endoscopists were 88% vs 73% accuracy, 93% vs 72% sensitivity, and 83% vs 74% specificity. The CAD system achieved higher accuracy than any of the individual 53 nonexpert endoscopists, with comparable delineation performance. CAD delineations of the area of neoplasm overlapped with those from the BE experts in all detected neoplasia in data sets 4 and 5. The CAD system identified the optimal site for biopsy of detected neoplasia in 97% and 92% of cases (data sets 4 and 5, respectively). CONCLUSIONS:We developed, validated, and benchmarked a deep-learning computer-aided system for primary detection of neoplasia in patients with BE. The system detected neoplasia with high accuracy and near-perfect delineation performance. The Netherlands National Trials Registry, Number: NTR7072.
The Argos project: The development of a computer-aided detection system to improve detection of Barrett's neoplasia on white light endoscopy.
de Groof Jeroen,van der Sommen Fons,van der Putten Joost,Struyvenberg Maarten R,Zinger Sveta,Curvers Wouter L,Pech Oliver,Meining Alexander,Neuhaus Horst,Bisschops Raf,Schoon Erik J,de With Peter H,Bergman Jacques J
United European gastroenterology journal
Background:Computer-aided detection (CAD) systems might assist endoscopists in the recognition of Barrett's neoplasia. Aim:To develop a CAD system using endoscopic images of Barrett's neoplasia. Methods:White light endoscopy (WLE) overview images of 40 neoplastic Barrett's lesions and 20 non-dysplastic Barret's oesophagus (NDBO) patients were prospectively collected. Experts delineated all neoplastic images.The overlap area of at least four delineations was labelled as the 'sweet spot'. The area with at least one delineation was labelled as the 'soft spot'. The CAD system was trained on colour and texture features. Positive features were taken from the sweet spot and negative features from NDBO images. Performance was evaluated using leave-one-out cross-validation. Outcome parameters were diagnostic accuracy of the CAD system per image, and localization of the expert soft spot by CAD delineation (localization score) and its indication of preferred biopsy location (red-flag indication score). Results:Accuracy, sensitivity and specificity for detection were 92, 95 and 85%, respectively. The system localized and red-flagged the soft spot in 100 and 90%, respectively. Conclusion:This uniquely trained and validated CAD system detected and localized early Barrett's neoplasia on WLE images with high accuracy. This is an important step towards real-time automated detection of Barrett's neoplasia.
Artificial intelligence using convolutional neural networks for real-time detection of early esophageal neoplasia in Barrett's esophagus (with video).
Hashimoto Rintaro,Requa James,Dao Tyler,Ninh Andrew,Tran Elise,Mai Daniel,Lugo Michael,El-Hage Chehade Nabil,Chang Kenneth J,Karnes Williams E,Samarasena Jason B
BACKGROUND AND AIMS:The visual detection of early esophageal neoplasia (high-grade dysplasia and T1 cancer) in Barrett's esophagus (BE) with white-light and virtual chromoendoscopy still remains challenging. The aim of this study was to assess whether a convolutional neural artificial intelligence network can aid in the recognition of early esophageal neoplasia in BE. METHODS:Nine hundred sixteen images from 65 patients of histology-proven early esophageal neoplasia in BE containing high-grade dysplasia or T1 cancer were collected. The area of neoplasia was masked using image annotation software. Nine hundred nineteen control images were collected of BE without high-grade dysplasia. A convolutional neural network (CNN) algorithm was pretrained on ImageNet and then fine-tuned with the goal of providing the correct binary classification of "dysplastic" or "nondysplastic." We developed an object detection algorithm that drew localization boxes around regions classified as dysplasia. RESULTS:The CNN analyzed 458 test images (225 dysplasia and 233 nondysplasia) and correctly detected early neoplasia with sensitivity of 96.4%, specificity of 94.2%, and accuracy of 95.4%. With regard to the object detection algorithm for all images in the validation set, the system was able to achieve a mean average precision of .7533 at an intersection over union of .3 CONCLUSIONS: In this pilot study, our artificial intelligence model was able to detect early esophageal neoplasia in BE images with high accuracy. In addition, the object detection algorithm was able to draw a localization box around the areas of dysplasia with high precision and at a speed that allows for real-time implementation.