logo logo
Development and Validation of a Deep Neural Network for Accurate Evaluation of Endoscopic Images From Patients With Ulcerative Colitis. Takenaka Kento,Ohtsuka Kazuo,Fujii Toshimitsu,Negi Mariko,Suzuki Kohei,Shimizu Hiromichi,Oshima Shiori,Akiyama Shintaro,Motobayashi Maiko,Nagahori Masakazu,Saito Eiko,Matsuoka Katsuyoshi,Watanabe Mamoru Gastroenterology BACKGROUND & AIMS:There are intra- and interobserver variations in endoscopic assessment of ulcerative colitis (UC) and biopsies are often collected for histologic evaluation. We sought to develop a deep neural network system for consistent, objective, and real-time analysis of endoscopic images from patients with UC. METHODS:We constructed the deep neural network for evaluation of UC (DNUC) algorithm using 40,758 images of colonoscopies and 6885 biopsy results from 2012 patients with UC who underwent colonoscopy from January 2014 through March 2018 at a single center in Japan (the training set). We validated the accuracy of the DNUC algorithm in a prospective study of 875 patients with UC who underwent colonoscopy from April 2018 through April 2019, with 4187 endoscopic images and 4104 biopsy specimens. Endoscopic remission was defined as a UC endoscopic index of severity score of 0; histologic remission was defined as a Geboes score of 3 points or less. RESULTS:In the prospective study, the DNUC identified patients with endoscopic remission with 90.1% accuracy (95% confidence interval [CI] 89.2%-90.9%) and a kappa coefficient of 0.798 (95% CI 0.780-0.814), using findings reported by endoscopists as the reference standard. The intraclass correlation coefficient between the DNUC and the endoscopists for UC endoscopic index of severity scoring was 0.917 (95% CI 0.911-0.921). The DNUC identified patients in histologic remission with 92.9% accuracy (95% CI 92.1%-93.7%); the kappa coefficient between the DNUC and the biopsy result was 0.859 (95% CI 0.841-0.875). CONCLUSIONS:We developed a deep neural network for evaluation of endoscopic images from patients with UC that identified those in endoscopic remission with 90.1% accuracy and histologic remission with 92.9% accuracy. The DNUC can therefore identify patients in remission without the need for mucosal biopsy collection and analysis. Trial number: UMIN000031430. 10.1053/j.gastro.2020.02.012
Artificial Intelligence-assisted System Improves Endoscopic Identification of Colorectal Neoplasms. Kudo Shin-Ei,Misawa Masashi,Mori Yuichi,Hotta Kinichi,Ohtsuka Kazuo,Ikematsu Hiroaki,Saito Yutaka,Takeda Kenichi,Nakamura Hiroki,Ichimasa Katsuro,Ishigaki Tomoyuki,Toyoshima Naoya,Kudo Toyoki,Hayashi Takemasa,Wakamura Kunihiko,Baba Toshiyuki,Ishida Fumio,Inoue Haruhiro,Itoh Hayato,Oda Masahiro,Mori Kensaku Clinical gastroenterology and hepatology : the official clinical practice journal of the American Gastroenterological Association BACKGROUND & AIMS:Precise optical diagnosis of colorectal polyps could improve the cost-effectiveness of colonoscopy and reduce polypectomy-related complications. However, it is difficult for community-based non-experts to obtain sufficient diagnostic performance. Artificial intelligence-based systems have been developed to analyze endoscopic images; they identify neoplasms with high accuracy and low interobserver variation. We performed a multi-center study to determine the diagnostic accuracy of EndoBRAIN, an artificial intelligence-based system that analyzes cell nuclei, crypt structure, and microvessels in endoscopic images, in identification of colon neoplasms. METHODS:The EndoBRAIN system was initially trained using 69,142 endocytoscopic images, taken at 520-fold magnification, from patients with colorectal polyps who underwent endoscopy at 5 academic centers in Japan from October 2017 through March 2018. We performed a retrospective comparative analysis of the diagnostic performance of EndoBRAIN vs that of 30 endoscopists (20 trainees and 10 experts); the endoscopists assessed images from 100 cases produced via white-light microscopy, endocytoscopy with methylene blue staining, and endocytoscopy with narrow-band imaging. EndoBRAIN was used to assess endocytoscopic, but not white-light, images. The primary outcome was the accuracy of EndoBrain in distinguishing neoplasms from non-neoplasms, compared with that of endoscopists, using findings from pathology analysis as the reference standard. RESULTS:In analysis of stained endocytoscopic images, EndoBRAIN identified colon lesions with 96.9% sensitivity (95% CI, 95.8%-97.8%), 100% specificity (95% CI, 99.6%-100%), 98% accuracy (95% CI, 97.3%-98.6%), a 100% positive-predictive value (95% CI, 99.8%-100%), and a 94.6% negative-predictive (95% CI, 92.7%-96.1%); these values were all significantly greater than those of the endoscopy trainees and experts. In analysis of narrow-band images, EndoBRAIN distinguished neoplastic from non-neoplastic lesions with 96.9% sensitivity (95% CI, 95.8-97.8), 94.3% specificity (95% CI, 92.3-95.9), 96.0% accuracy (95% CI, 95.1-96.8), a 96.9% positive-predictive value, (95% CI, 95.8-97.8), and a 94.3% negative-predictive value (95% CI, 92.3-95.9); these values were all significantly higher than those of the endoscopy trainees, sensitivity and negative-predictive value were significantly higher but the other values are comparable to those of the experts. CONCLUSIONS:EndoBRAIN accurately differentiated neoplastic from non-neoplastic lesions in stained endocytoscopic images and endocytoscopic narrow-band images, when pathology findings were used as the standard. This technology has been authorized for clinical use by the Japanese regulatory agency and should be used in endoscopic evaluation of small polyps more widespread clinical settings. UMIN clinical trial no: UMIN000028843. 10.1016/j.cgh.2019.09.009
Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Wang Pu,Berzin Tyler M,Glissen Brown Jeremy Romek,Bharadwaj Shishira,Becq Aymeric,Xiao Xun,Liu Peixi,Li Liangping,Song Yan,Zhang Di,Li Yi,Xu Guangre,Tu Mengtian,Liu Xiaogang Gut OBJECTIVE:The effect of colonoscopy on colorectal cancer mortality is limited by several factors, among them a certain miss rate, leading to limited adenoma detection rates (ADRs). We investigated the effect of an automatic polyp detection system based on deep learning on polyp detection rate and ADR. DESIGN:In an open, non-blinded trial, consecutive patients were prospectively randomised to undergo diagnostic colonoscopy with or without assistance of a real-time automatic polyp detection system providing a simultaneous visual notice and sound alarm on polyp detection. The primary outcome was ADR. RESULTS:Of 1058 patients included, 536 were randomised to standard colonoscopy, and 522 were randomised to colonoscopy with computer-aided diagnosis. The artificial intelligence (AI) system significantly increased ADR (29.1%vs20.3%, p<0.001) and the mean number of adenomas per patient (0.53vs0.31, p<0.001). This was due to a higher number of diminutive adenomas found (185vs102; p<0.001), while there was no statistical difference in larger adenomas (77vs58, p=0.075). In addition, the number of hyperplastic polyps was also significantly increased (114vs52, p<0.001). CONCLUSIONS:In a low prevalent ADR population, an automatic polyp detection system during colonoscopy resulted in a significant increase in the number of diminutive adenomas detected, as well as an increase in the rate of hyperplastic polyps. The cost-benefit ratio of such effects has to be determined further. TRIAL REGISTRATION NUMBER:ChiCTR-DDD-17012221; Results. 10.1136/gutjnl-2018-317500
Improved Accuracy in Optical Diagnosis of Colorectal Polyps Using Convolutional Neural Networks with Visual Explanations. Jin Eun Hyo,Lee Dongheon,Bae Jung Ho,Kang Hae Yeon,Kwak Min-Sun,Seo Ji Yeon,Yang Jong In,Yang Sun Young,Lim Seon Hee,Yim Jeong Yoon,Lim Joo Hyun,Chung Goh Eun,Chung Su Jin,Choi Ji Min,Han Yoo Min,Kang Seung Joo,Lee Jooyoung,Chan Kim Hee,Kim Joo Sung Gastroenterology BACKGROUND & AIMS:Narrow-band imaging (NBI) can be used to determine whether colorectal polyps are adenomatous or hyperplastic. We investigated whether an artificial intelligence (AI) system can increase the accuracy of characterizations of polyps by endoscopists of different skill levels. METHODS:We developed convolutional neural networks (CNNs) for evaluation of diminutive colorectal polyps, based on efficient neural architecture searches via parameter sharing with augmentation using NBIs of diminutive (≤5 mm) polyps, collected from October 2015 through October 2017 at the Seoul National University Hospital, Healthcare System Gangnam Center (training set). We trained the CNN using images from 1100 adenomatous polyps and 1050 hyperplastic polyps from 1379 patients. We then tested the system using 300 images of 180 adenomatous polyps and 120 hyperplastic polyps, obtained from January 2018 to May 2019. We compared the accuracy of 22 endoscopists of different skill levels (7 novices, 4 experts, and 11 NBI-trained experts) vs the CNN in evaluation of images (adenomatous vs hyperplastic) from 180 adenomatous and 120 hyperplastic polyps. The endoscopists then evaluated the polyp images with knowledge of the CNN-processed results. We conducted mixed-effect logistic and linear regression analyses to determine the effects of AI assistance on the accuracy of analysis of diminutive colorectal polyps by endoscopists (primary outcome). RESULTS:The CNN distinguished adenomatous vs hyperplastic diminutive polyps with 86.7% accuracy, based on histologic analysis as the reference standard. Endoscopists distinguished adenomatous vs hyperplastic diminutive polyps with 82.5% overall accuracy (novices, 73.8% accuracy; experts, 83.8% accuracy; and NBI-trained experts, 87.6% accuracy). With knowledge of the CNN-processed results, the overall accuracy of the endoscopists increased to 88.5% (P < .05). With knowledge of the CNN-processed results, the accuracy of novice endoscopists increased to 85.6% (P < .05). The CNN-processed results significantly reduced endoscopist time of diagnosis (from 3.92 to 3.37 seconds per polyp, P = .042). CONCLUSIONS:We developed a CNN that significantly increases the accuracy of evaluation of diminutive colorectal polyps (as adenomatous vs hyperplastic) and reduces the time of diagnosis by endoscopists. This AI assistance system significantly increased the accuracy of analysis by novice endoscopists, who achieved near-expert levels of accuracy without extra training. The CNN assistance system can reduce the skill-level dependence of endoscopists and costs. 10.1053/j.gastro.2020.02.036
Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Byrne Michael F,Chapados Nicolas,Soudan Florian,Oertel Clemens,Linares Pérez Milagros,Kelly Raymond,Iqbal Nadeem,Chandelier Florent,Rex Douglas K Gut BACKGROUND:In general, academic but not community endoscopists have demonstrated adequate endoscopic differentiation accuracy to make the 'resect and discard' paradigm for diminutive colorectal polyps workable. Computer analysis of video could potentially eliminate the obstacle of interobserver variability in endoscopic polyp interpretation and enable widespread acceptance of 'resect and discard'. STUDY DESIGN AND METHODS:We developed an artificial intelligence (AI) model for real-time assessment of endoscopic video images of colorectal polyps. A deep convolutional neural network model was used. Only narrow band imaging video frames were used, split equally between relevant multiclasses. Unaltered videos from routine exams not specifically designed or adapted for AI classification were used to train and validate the model. The model was tested on a separate series of 125 videos of consecutively encountered diminutive polyps that were proven to be adenomas or hyperplastic polyps. RESULTS:The AI model works with a confidence mechanism and did not generate sufficient confidence to predict the histology of 19 polyps in the test set, representing 15% of the polyps. For the remaining 106 diminutive polyps, the accuracy of the model was 94% (95% CI 86% to 97%), the sensitivity for identification of adenomas was 98% (95% CI 92% to 100%), specificity was 83% (95% CI 67% to 93%), negative predictive value 97% and positive predictive value 90%. CONCLUSIONS:An AI model trained on endoscopic video can differentiate diminutive adenomas from hyperplastic polyps with high accuracy. Additional study of this programme in a live patient clinical trial setting to address resect and discard is planned. 10.1136/gutjnl-2017-314547
New artificial intelligence system: first validation study versus experienced endoscopists for colorectal polyp detection. Hassan Cesare,Wallace Michael B,Sharma Prateek,Maselli Roberta,Craviotto Vincenzo,Spadaccini Marco,Repici Alessandro Gut 10.1136/gutjnl-2019-319914
Real-Time Use of Artificial Intelligence in Identification of Diminutive Polyps During Colonoscopy: A Prospective Study. Mori Yuichi,Kudo Shin-Ei,Misawa Masashi,Saito Yutaka,Ikematsu Hiroaki,Hotta Kinichi,Ohtsuka Kazuo,Urushibara Fumihiko,Kataoka Shinichi,Ogawa Yushi,Maeda Yasuharu,Takeda Kenichi,Nakamura Hiroki,Ichimasa Katsuro,Kudo Toyoki,Hayashi Takemasa,Wakamura Kunihiko,Ishida Fumio,Inoue Haruhiro,Itoh Hayato,Oda Masahiro,Mori Kensaku Annals of internal medicine Background:Computer-aided diagnosis (CAD) for colonoscopy may help endoscopists distinguish neoplastic polyps (adenomas) requiring resection from nonneoplastic polyps not requiring resection, potentially reducing cost. Objective:To evaluate the performance of real-time CAD with endocytoscopes (×520 ultramagnifying colonoscopes providing microvascular and cellular visualization of colorectal polyps after application of the narrow-band imaging [NBI] and methylene blue staining modes, respectively). Design:Single-group, open-label, prospective study. (UMIN [University hospital Medical Information Network] Clinical Trial Registry: UMIN000027360). Setting:University hospital. Participants:791 consecutive patients undergoing colonoscopy and 23 endoscopists. Intervention:Real-time use of CAD during colonoscopy. Measurements:CAD-predicted pathology (neoplastic or nonneoplastic) of detected diminutive polyps (≤5 mm) on the basis of real-time outputs compared with pathologic diagnosis of the resected specimen (gold standard). The primary end point was whether CAD with the stained mode produced a negative predictive value (NPV) of 90% or greater for identifying diminutive rectosigmoid adenomas, the threshold required to "diagnose-and-leave" nonneoplastic polyps. Best- and worst-case scenarios assumed that polyps lacking either CAD diagnosis or pathology were true- or false-positive or true- or false-negative, respectively. Results:Overall, 466 diminutive (including 250 rectosigmoid) polyps from 325 patients were assessed by CAD, with a pathologic prediction rate of 98.1% (457 of 466). The NPVs of CAD for diminutive rectosigmoid adenomas were 96.4% (95% CI, 91.8% to 98.8%) (best-case scenario) and 93.7% (CI, 88.3% to 97.1%) (worst-case scenario) with stained mode and 96.5% (CI, 92.1% to 98.9%) (best-case scenario) and 95.2% (CI, 90.3% to 98.0%) (worst-case scenario) with NBI. Limitation:Two thirds of the colonoscopies were conducted by experts who had each experienced more than 200 endocytoscopies; 186 polyps not assessed by CAD were excluded. Conclusion:Real-time CAD can achieve the performance level required for a diagnose-and-leave strategy for diminutive, nonneoplastic rectosigmoid polyps. Primary Funding Source:Japan Society for the Promotion of Science. 10.7326/M18-0249
Accurate Classification of Diminutive Colorectal Polyps Using Computer-Aided Analysis. Chen Peng-Jen,Lin Meng-Chiung,Lai Mei-Ju,Lin Jung-Chun,Lu Henry Horng-Shing,Tseng Vincent S Gastroenterology BACKGROUND & AIMS:Narrow-band imaging is an image-enhanced form of endoscopy used to observed microstructures and capillaries of the mucosal epithelium which allows for real-time prediction of histologic features of colorectal polyps. However, narrow-band imaging expertise is required to differentiate hyperplastic from neoplastic polyps with high levels of accuracy. We developed and tested a system of computer-aided diagnosis with a deep neural network (DNN-CAD) to analyze narrow-band images of diminutive colorectal polyps. METHODS:We collected 1476 images of neoplastic polyps and 681 images of hyperplastic polyps, obtained from the picture archiving and communications system database in a tertiary hospital in Taiwan. Histologic findings from the polyps were also collected and used as the reference standard. The images and data were used to train the DNN. A test set of images (96 hyperplastic and 188 neoplastic polyps, smaller than 5 mm), obtained from patients who underwent colonoscopies from March 2017 through August 2017, was then used to test the diagnostic ability of the DNN-CAD vs endoscopists (2 expert and 4 novice), who were asked to classify the images of the test set as neoplastic or hyperplastic. Their classifications were compared with findings from histologic analysis. The primary outcome measures were diagnostic accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and diagnostic time. The accuracy, sensitivity, specificity, PPV, NPV, and diagnostic time were compared among DNN-CAD, the novice endoscopists, and the expert endoscopists. The study was designed to detect a difference of 10% in accuracy by a 2-sided McNemar test. RESULTS:In the test set, the DNN-CAD identified neoplastic or hyperplastic polyps with 96.3% sensitivity, 78.1% specificity, a PPV of 89.6%, and a NPV of 91.5%. Fewer than half of the novice endoscopists classified polyps with a NPV of 90% (their NPVs ranged from 73.9% to 84.0%). DNN-CAD classified polyps as neoplastic or hyperplastic in 0.45 ± 0.07 seconds-shorter than the time required by experts (1.54 ± 1.30 seconds) and nonexperts (1.77 ± 1.37 seconds) (both P < .001). DNN-CAD classified polyps with perfect intra-observer agreement (kappa score of 1). There was a low level of intra-observer and inter-observer agreement in classification among endoscopists. CONCLUSIONS:We developed a system called DNN-CAD to identify neoplastic or hyperplastic colorectal polyps less than 5 mm. The system classified polyps with a PPV of 89.6%, and a NPV of 91.5%, and in a shorter time than endoscopists. This deep-learning model has potential for not only endoscopic image recognition but for other forms of medical image analysis, including sonography, computed tomography, and magnetic resonance images. 10.1053/j.gastro.2017.10.010
Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Wang Pu,Xiao Xiao,Glissen Brown Jeremy R,Berzin Tyler M,Tu Mengtian,Xiong Fei,Hu Xiao,Liu Peixi,Song Yan,Zhang Di,Yang Xue,Li Liangping,He Jiong,Yi Xin,Liu Jingjia,Liu Xiaogang Nature biomedical engineering The detection and removal of precancerous polyps via colonoscopy is the gold standard for the prevention of colon cancer. However, the detection rate of adenomatous polyps can vary significantly among endoscopists. Here, we show that a machine-learning algorithm can detect polyps in clinical colonoscopies, in real time and with high sensitivity and specificity. We developed the deep-learning algorithm by using data from 1,290 patients, and validated it on newly collected 27,113 colonoscopy images from 1,138 patients with at least one detected polyp (per-image-sensitivity, 94.38%; per-image-specificity, 95.92%; area under the receiver operating characteristic curve, 0.984), on a public database of 612 polyp-containing images (per-image-sensitivity, 88.24%), on 138 colonoscopy videos with histologically confirmed polyps (per-image-sensitivity of 91.64%; per-polyp-sensitivity, 100%), and on 54 unaltered full-range colonoscopy videos without polyps (per-image-specificity, 95.40%). By using a multi-threaded processing system, the algorithm can process at least 25 frames per second with a latency of 76.80 ± 5.60 ms in real-time video analysis. The software may aid endoscopists while performing colonoscopies, and help assess differences in polyp and adenoma detection performance among endoscopists. 10.1038/s41551-018-0301-3
Artificial Intelligence-Assisted Polyp Detection for Colonoscopy: Initial Experience. Misawa Masashi,Kudo Shin-Ei,Mori Yuichi,Cho Tomonari,Kataoka Shinichi,Yamauchi Akihiro,Ogawa Yushi,Maeda Yasuharu,Takeda Kenichi,Ichimasa Katsuro,Nakamura Hiroki,Yagawa Yusuke,Toyoshima Naoya,Ogata Noriyuki,Kudo Toyoki,Hisayuki Tomokazu,Hayashi Takemasa,Wakamura Kunihiko,Baba Toshiyuki,Ishida Fumio,Itoh Hayato,Roth Holger,Oda Masahiro,Mori Kensaku Gastroenterology 10.1053/j.gastro.2018.04.003