Endoscopy

A machine learning-based choledocholithiasis prediction tool improves ERCP decision making -a proof of concept study
基于机器学习的胆总管结石预测工具改善ERCP决策-概念验证研究

Steven NSteinway,Bohao Tang,Jeremy Telezing,Aditya Ashok,Ayesha Kamal,Chung Yao Yu,Nitin Jagtap,James Buxbaum,B. Joseph EImunzer,Sachin B Wani,Mouen AKhashab,Brian SCaffo,Venkata S Akshintala.

Affiliations below.

DOl:10.1055/a-2174-0534

Please cite this article as: Steinway SN,Tang B,TelezingJet al.A machine learning-based choledocholithiasis prediction tool improves ERCP decision making-a proof of concept study.Endoscopy 2023.doi:10.1055/a-2174-0534

Conflict of Interest: Venkata Akshintala:Co-founder and Chief Medical Officer,Origin Endoscopy Inc.
利益冲突：Venkata Akshintala：Origin Endoscopy Inc.联合创始人兼首席医疗官。

Mouen Khashab:Advisory board member and consultant for Boston Scientific,Olympus,Medtronic.
Mouen Khashab：Boston Scientific、Olympus、Medtronic顾问委员会成员和顾问。

All the other authors have no disclosures
所有其他作者都没有披露

Abstract:
摘要：

leading to overutilization of ERCP.Improved stratification may allow for appropriate patient selection for ERCP and the useof
改进的分层可以允许适当的患者选择ERCP和使用

lower-risk modalities (i.e.EUS and MRCP).

Methods:Amachine learning model was developed using patient information from two published cohort studiesoriginally
方法：使用两项已发表的队列研究中的患者信息开发了一个机器学习模型，

used to evaluate performance of published guidelines in predicting choledocholithiasis.Prediction models were developed
用于评价已发表的指南在预测胆总管结石方面的性能。

usingthe gradientboosting(GBM)machine learning method.GBM performance was evaluated using 10-fold cross-validation and area under the receiver operating curve(AUC).Important predictors of choledocholithiasis were identified based on relati- ve importance in GBM.
使用梯度增强（GBM）机器学习方法。使用10倍交叉验证和受试者工作曲线下面积（AUC）评估GBM性能。基于GBM中的相对重要性识别胆总管结石的重要预测因子。

Results:1,378 patients (mean age 43.3 years;55.5%females)were included in the GBM model and 59.4%had choledocholit-
结果：1，378例患者（平均年龄43.3岁; 55.5%为女性）被纳入GBM模型，59.4%患有胆总管结石。

hiasis.Eight variables were identified as predictors of choledocholithiasis.The GBM model was evaluated with 10-fold cross-
8个变量被确定为胆总管结石的预测因子。GBM模型用10倍交叉检验进行评估。

validation andhad an accuracy of71.5±2.5%(AUC0.79±0.06)and performed better than the 2019 ASGE guidelines(accuracy
验证，准确度为71.5 ±2.5%（AUC 0.79 ±0.06），优于2019年ASGE指南（准确度

62.4±2.6%,AUC0.63±0.03)and the ESGE guidelines (accuracy 62.8±2.6%,AUC0.67±0.02).The GBM model correctly categorized 22%of patientsdirected tounnecessary ERCPby the ASGE guidelines and appropriately recommended 48%of ERCPs incorrectly rejected by the ESGE guidelines as the next step in the management.
GBM模型对ASGE指南不必要的ERCP的正确分类率为22%，对ESGE指南不正确的ERCP的正确分类率为48%，GBM模型对ASGE指南不必要的ERCP的正确分类率为62.8± 2.6%，AUC0.63 ± 0.03）和ESGE指南的正确分类率为62.8± 2.6%，AUC0.67 ±0.02。

Conclusions:A machine learning-basedtool was created that provides a real-time,personalized,objective probability of chole-
结论：创建了一个基于机器学习的工具，提供了一个实时的，个性化的，客观的胆固醇概率，

docholithiasis and ERCPrecommendations.This more accurately directs ERCP use than the existingASGE and ESGE guidelines
这比现有的ASGE和ESGE指南更准确地指导ERCP的使用

and has the potential to reduce morbidity and healthcare costs associatedwith ERCPor missedcholedocholithiasis.
并有可能降低与ERCP或漏诊胆总管结石相关的发病率和医疗费用。

Corresponding Author:

Dr.Venkata S Akshintala,Johns Hopkins Medical Institutions,Division of Gastroenterology and Hepatology,600N.Wolfe St,21287 Baltimore,United States,vakshin1@jhmi.edu

Affiliations:

This is a PDF file of an unedited manuscript that has been accepted for publication.As a service
这是一份未经编辑的手稿的PDF文件，已被接受出版。作为一项服务

to our customers we are providing this early version of the manuscript.The manuscriptwill
我们向我们的客户提供这份手稿的早期版本。

undergo copyediting,typesetting,and review of the resulting proofbefore it is published in its
在出版之前，要经过编辑、排版和校对。

final form.Please note that during the production process errors may be discovered which could
请注意，在生产过程中可能会发现错误，

affect the content,and all legal disclaimers that apply to the journal pertain.
影响内容，所有适用于该杂志的法律的声明都适用。

Steven N Steinway,Johns Hopkins Medical Institutions,Division of Gastroenterology and Hepatology,Baltimore,United States
Steven N Steinway，美国巴尔的摩约翰霍普金斯医疗机构胃肠病学和肝病学分部

Bohao Tang,Johns Hopkins Bloomberg School of Public Health Centerfor Teaching and Learning,Department of Biostatistics,Balti- more,United States
唐伯豪，约翰霍普金斯彭博公共卫生学院生物统计系教学中心，巴尔蒂-莫尔，美国

Jeremy Telezing,Johns Hopkins Bloomberg School of Public Health Center for Teachingand Learning,Department of Biostatistics,Balti- more,United States
Jeremy Telezing，约翰霍普金斯彭博公共卫生学院教学中心，生物统计学系，巴尔蒂-莫尔，美国

[.]

Venkata SAkshintala,Johns Hopkins Medical Institutions,Division of Gastroenterology andHepatology,Baltimore,United States

This is a PDF file of an unedited manuscript that has been accepted for publication.As a service
这是一份未经编辑的手稿的PDF文件，已被接受出版。作为一项服务

to our customers we are providing this early version of the manuscript.The manuscriptwill
我们向我们的客户提供这份手稿的早期版本。

undergo copyediting,typesetting,and review of the resulting proof before it is published in its
经过编辑，排版，并审查所产生的证明，然后才发表在其

final form.Please note that duringthe production process errors may be discovered which could
请注意，在生产过程中可能会发现错误，

affectthe content,and all legal discaimers that apply to the journal pertain.
影响内容，以及所有适用于该杂志的法律的争议。

Title: A machine learning-based choledocholithiasis prediction tool improves ERCP decision making – a
标题：一种基于机器学习的胆总管结石预测工具改善了ERCP决策--

proof of concept study
概念验证研究

Short Title: Choledocholithiasis decision-making tool
短标题：胆总管结石决策工具

Authors:

Steven N. Steinway M.D. Ph.D.*1 Bohao Tang Ph.D.*2 Jeremy Telezing 2 Aditya Ashok M.D.1

Ayesha Kamal Ph.D.1 Chung Yao Yu M.D.3 Nitin Jagtap M.D. D.M. 4 James L. Buxbaum M.D.3

Joseph Elmunzer M.D. M.Sc5 Sachin B. Wani M.D.6 Mouen A. Khashab M.D.1 Brian S. Caffo Ph.D.2 #

Venkata S. Akshintala M.D. #

* Authors contributed equally
* 作者贡献相同

# Authors contributed equally
#作者贡献相等

1. Division of Gastroenterology Johns Hopkins Medical Institutions Baltimore MD United States. 2. Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Baltimore MD
1.美国巴尔的摩约翰·霍普金斯医疗机构胃肠病学科Division of Gastroenterology Johns Hopkins Medical Institutions Baltimore MD United States 2.约翰·霍普金斯大学布隆伯格公共卫生学院巴尔的摩医学博士生物统计系Johns Hopkins Bloomberg School of Public Health Baltimore MD

3. Division of Gastroenterology Keck School of Medicine University of Southern California Los Angeles CA
3.美国南加州大学洛杉矶分校（The Division of Gastroenterology Keck School of Medicine）

4. Department of Gastroenterology Asian Institute of Gastroenterology India.
4.主办单位：Asian Institute of Gastroenterology India

5. Division of Gastroenterology and Hepatology Medical University of South Carolina Charleston SC
5.美国南卡罗来纳州查尔斯顿医科大学（Division of Gastroenterology and Hepatology Medical University of South Carolina Charleston SC）

6. Division of Gastroenterology University of Colorado Anschutz Medical Campus Aurora CO

Address correspondence to:

Venkata Akshintala MD

Assistant Professor of Medicine

Johns Hopkins Hospital Division of Gastroenterology and Hepatology
约翰霍普金斯大学胃肠病学和肝病科Johns Hopkins Hospital Division of Gastroenterology and Hepatology

600 N. Wolfe St Blalock 411 Baltimore MD 21287

Email: vakshin1@jhmi.edu

Phone: 410-624 6955

Author

Steven Steinway - interpretation and analysis of data; drafting of the manuscript; critical revision of the
史蒂文·施坦威-数据解释和分析;手稿起草;批判性修订

manuscript for important intellectual content

Bohao Tang - interpretation and analysis of data; critical revision of the manuscript for important
唐伯豪-数据的解释和分析;重要手稿的批判性修订

intellectual content
知识性内容

Jeremy Telezing - interpretation and analysis of data; critical revision of the manuscript for important
Jeremy Telezing -数据的解释和分析;重要手稿的批判性修订

intellectual content
知识性内容

Aditya Ashok - drafting of the manuscript; critical revision of the manuscript for important intellectual
Aditya Ashok -手稿的起草;重要知识分子手稿的批判性修订

content

Ayesha Kamal - acquisition of data; drafting of the manuscript; critical revision of the manuscript for
Ayesha Kamal -数据采集;手稿起草;手稿的关键修订，

important intellectual content
重要智力内容

Chung Yao Yu - acquisition of data; critical revision of the manuscript for important intellectual content
Chung Yao Yu -数据采集;重要知识内容手稿的批判性修订

Nitin Jagtap - critical revision of the manuscript for important intellectual content
Nitin Jagtap -对重要知识内容的手稿进行批判性修订

James L. Buxbaum - critical revision of the manuscript for important intellectual content

B. Joseph Elmunzer - critical revision of the manuscript for important intellectual content

Sachin B. Wani - critical revision of the manuscript for important intellectual content
萨钦B。瓦尼-对重要知识内容的手稿进行批判性修订

Brian S. Caffo - study concept and design; interpretation of data; critical revision of the manuscript for
布莱恩·S Caffo -研究概念和设计;数据解释;手稿的关键修订，

important intellectual content
重要智力内容

Mouen A. Khashab - study concept and design; interpretation of data; drafting of the manuscript; critical

revision of the manuscript for important intellectual content; study supervision
修改重要知识内容的手稿;研究监督

Venkata S. Akshintala - study concept and design; interpretation and analysis of data; drafting of the

manuscript; critical revision of the manuscript for important intellectual content
原稿;对原稿重要知识内容的批判性修改

Abstract
摘要

Background: Prior studies have demonstrated that existing guidelines to predict choledocholithiasis have limited accuracy leading to overutilization of ERCP. Improved stratification may allow for
背景：先前的研究表明，现有的指南预测胆总管结石的准确性有限，导致过度使用ERCP。改进的分层可以允许

appropriate patient selection for ERCP and the use of lower-risk modalities (i.e. EUS and MRCP).
选择适当的患者进行ERCP和使用低风险模式（即EUS和MRCP）。

Methods: A machine learning model was developed using patient information from two published cohort studies originally used to evaluate performance of published guidelines in predicting choledocholithiasis. Prediction models were developed using the gradient boosting (GBM) machine learning method. GBM performance was evaluated using 10-fold cross-validation and area under the receiver operating curve (AUC). Important predictors of choledocholithiasis were identified based on
方法：使用来自两项已发表队列研究的患者信息开发了一种机器学习模型，这些研究最初用于评估已发表指南在预测胆总管结石方面的性能。预测模型是使用梯度提升（GBM）机器学习方法开发的。使用10倍交叉验证和受试者工作曲线下面积（AUC）评价GBM性能。胆总管结石的重要预测因素是基于

relative importance in GBM.
GBM的相对重要性。

Results: 1 378 patients (mean age 43.3 years; 55.5% females) were included in the GBM model and 59.4% had choledocholithiasis. Eight variables were identified as predictors of choledocholithiasis. The GBM model was evaluated with 10-fold cross-validation and had an accuracy of 71.5±2.5% (AUC 0.79±0.06) and performed better than the 2019 ASGE guidelines (accuracy 62.4±2.6% AUC 0.63±0.03) and the ESGE guidelines (accuracy 62.8±2.6% AUC 0.67±0.02). The GBM model correctly categorized 22% of patients directed to unnecessary ERCP by the ASGE guidelines and appropriately recommended
结果：1378例患者（平均年龄43.3岁，55.5%为女性）被纳入GBM模型，59.4%患有胆总管结石。八个变量被确定为胆总管结石的预测因子。GBM模型采用10倍交叉验证进行评估，准确度为71.5±2.5%（AUC 0.79±0.06），优于2019年ASGE指南（准确度62.4±2.6% AUC 0.63±0.03）和ESGE指南（准确度62.8±2.6% AUC 0.67±0.02）。GBM模型正确分类了22%的患者，这些患者根据ASGE指南进行了不必要的ERCP，

48% of ERCPs incorrectly rejected by the ESGE guidelines as the next step in the management .
48%的ERCP被ESGE指南错误地拒绝作为管理的下一步。

Conclusions: A machine learning-based tool was created that provides a real-time personalized objective probability of choledocholithiasis and ERCP recommendations. This more accurately directs ERCP use than the existing ASGE and ESGE guidelines and has the potential to reduce morbidity and
结论：创建了一个基于机器学习的工具，可以提供胆总管结石的实时个性化客观概率和ERCP建议。这比现有的ASGE和ESGE指南更准确地指导ERCP的使用，并有可能降低发病率，

healthcare costs associated with ERCP or missed choledocholithiasis.
与ERCP或漏诊胆总管结石相关的医疗费用。

Keywords: choledocholithiasis; ERCP; calculator; clinical prediction tool; machine learning
关键词：胆总管结石; ERCP;计算器;临床预测工具;机器学习

Abbreviations: (in the order that they appear)
缩略语：（按出现顺序排列）

1. Endoscopic retrograde cholangio-pancreatography (ERCP)

2. Endoscopic ultrasound (EUS)

3. American Society for Gastrointestinal Endoscopy (ASGE)
3.美国胃肠内镜学会（ASGE）

4. European Society for Gastrointestinal Endoscopy (ESGE)

5. Gradient boosting model (GBM)

6. Area under the curve (AUC)

7. Magnetic resonance cholangiopancreatography (MRCP)
7.磁共振胰胆管成像（MRCP）

8. Liver function tests (LFTs)

9. Common bile duct (CBD)

10. Positive predictive value (PPV)

11. Confidence interval (CI)

12. Ultrasound (US)

Introduction
介绍

Gallbladder disease affects an estimated 20.5 million persons in the United States with gallstone disease itself costing about $6.2 billion annually[1]. More specifically choledocholithiasis the presence of gallstones in the common bile duct (CBD) affects 10-20% of patients with symptomatic cholelithiasis 18-33% of patients with acute biliary pancreatitis and 7-14% of patients who underwent cholecystectomy. ERCP with bile duct stone clearance is needed to avoid complications including cholangitis and gallstone pancreatitis. Despite being the favored and least morbid procedure to treat choledocholithiasis ERCP is still associated with high cost radiation exposure to the patient and staff as
胆囊疾病影响美国估计2050万人，胆结石疾病本身每年花费约62亿美元[1]。更具体地，胆总管结石，胆总管（CBD）中胆结石的存在影响10-20%的有症状的胆石症患者、18-33%的急性胆源性胰腺炎患者和7-14%的接受胆囊切除术的患者。ERCP结合胆管结石清除是必要的，以避免并发症，包括胆管炎和胆源性胰腺炎。尽管ERCP是治疗胆总管结石的最受欢迎且发病率最低的手术，但它仍然与患者和工作人员的高成本辐射暴露相关，

well as major adverse events in 6-15% of patients[2]. The major adverse events include ERCP-related
以及6-15%患者的主要不良事件[2]。主要不良事件包括ERCP相关

pancreatitis infections perforation and bleeding[3]. Therefore it is critical to accurately identify the
胰腺炎感染穿孔和出血[3]。因此，准确识别

probability of choledocholithiasis to appropriately select patients for ERCP.
胆总管结石的可能性，以适当选择患者进行ERCP。

The 2010 American Society for Gastrointestinal Endoscopy (ASGE) guidelines categorized patients as high risk (probability>50%) intermediate (10-50%) and low risk (<10%) for
2010年美国胃肠内镜学会（ASGE）指南将患者分为高风险（概率>50%）、中等风险（10-50%）和低风险（<10%）。

choledocholithiasis based on clinical features including bilirubin level and bile duct diameter[4].
胆总管结石基于临床特征，包括胆红素水平和胆管直径[4]。

Nevertheless the 2010 ASGE guidelines directed patients unnecessarily to ERCP in 30-50% of cases. To improve the specificity in 2019 the ASGE revised their criteria to define more stringently which patients should proceed directly to ERCP[5]. The European Society for Gastrointestinal Endoscopy (ESGE) also came out with guidelines in 2019 which stratify patients to low-risk (normal LFTs and ultrasound) intermediate-risk (abnormal LFTs and/or CBD dilation) and high-risk (clinical cholangitis or BDS on US)
然而，2010年ASGE指南在30-50%的病例中不必要地指导患者进行ERCP。为了提高2019年的特异性，ASGE修订了他们的标准，以更严格地定义哪些患者应该直接进行ERCP[5]。欧洲胃肠内镜学会（ESGE）也在2019年发布了指南，将患者分为低风险（正常LFT和超声）、中风险（异常LFT和/或CBD扩张）和高风险（临床胆管炎或US上的BDS）

[2].

Improved stratification allows for more effective triage to lower risk modalities such as endoscopic ultrasound (EUS) and magnetic resonance cholangiopancreatography (MRCP) and thus for more appropriate patient selection for ERCP ultimately minimizing adverse events and improving patient outcomes. Yet conventional models of statistical inference have struggled in this regard. Machine learning on the other hand shows promise in the application to clinical data sets to reformulate patient classes and establish more accurate risk models [6]. To our knowledge there has not been a study that compares machine learning to the ASGE or ESGE guidelines. The goal of this study was to develop a machine learning-based risk estimation tool that can provide a real-time personalized objective
改进的分层可以更有效地将患者分类到低风险模式，如内镜超声（EUS）和磁共振胰胆管造影（MRCP），从而更适当地选择患者进行ERCP，最终最大限度地减少不良事件并改善患者结局。然而，传统的统计推断模型在这方面一直在努力。另一方面，机器学习在应用于临床数据集以重新制定患者类别并建立更准确的风险模型方面显示出希望[6]。据我们所知，还没有一项研究将机器学习与ASGE或ESGE指南进行比较。这项研究的目标是开发一种基于机器学习的风险估计工具，可以提供实时的个性化目标

probability of the presence of a CBD stone.
CBD结石存在的可能性。

Methods

The primary goal was to develop a computer-based prediction and decision-making tool for the
主要目标是开发一个基于计算机的预测和决策工具，

prediction of choledocholithiasis based on risk factors identified from the two cohort studies.
根据两项队列研究确定的风险因素预测胆总管结石。

Study Population

We integrated patient-level data from two large cohorts of patients admitted with suspected choledocholithiasis from the Medical University of South Carolina between January 1 2009 and December 31 2014 and the Los Angeles County Hospital between January 2010 to November 2016. For both cohorts biliary sludge was considered equivalent to choledocholithiasis since both have comparable clinical sequelae[7 8]. These study cohorts were selected since they have the essential patient-level data variables considered important for predicting choledocholithiasis. A total of 1378
我们整合了2009年1月1日至2014年12月31日期间来自南卡罗来纳州医科大学和2010年1月至2016年11月期间来自洛杉矶县医院的两个大型疑似胆总管结石患者队列的患者水平数据。对于两个队列，胆泥被认为等同于胆总管结石，因为两者具有相当的临床后遗症[7 8]。选择这些研究队列是因为它们具有被认为对预测胆总管结石很重要的基本患者水平数据变量。共1378个

patients were included in this study.
患者被纳入本研究。

Study Variables
研究变量

Data pertaining to patient demographics laboratory parameters at the time of presentation and 24-48 hours after imaging and procedural findings were obtained. If available a second set of laboratory tests were incorporated into the machine learning algorithm. For comparison of the effectiveness of the machine learning algorithm the same patients were classified as high risk for the presence of choledocholithiasis requiring ERCP using 2010 ASGE guidelines revised 2019 ASGE guidelines and the ESGE guidelines based on their first or second set of bloodwork. Choledocholithiasis presence was confirmed by ERCP intra-operative cholangiography or clinical follow-up. We excluded
获得了就诊时和成像和手术结果后24-48小时的患者人口统计学实验室参数相关数据。如果可用，则将第二组实验室测试并入机器学习算法中。为了比较机器学习算法的有效性，使用2010年ASGE指南修订的2019年ASGE指南和ESGE指南，根据他们的第一组或第二组血液检查，将相同的患者分类为需要ERCP的胆总管结石的高风险。胆总管结石的存在通过ERCP术中胆管造影或临床随访证实。我们排除

patients who had acute cholangitis as ERCP or biliary drainage was required in such patients.
急性胆管炎患者行ERCP或胆道引流术。

Model Construction and Validation
模型构建和验证

Model derivation and validation were developed on patient-level data obtained from participants of the two previously published cohort studies described above [7 8]. Prediction models
模型推导和验证是基于从上述两项先前发表的队列研究的参与者中获得的患者水平数据开发的[7 8]。预测模型

were developed using a gradient-boosting machine learning algorithm (GBM).
使用梯度提升机器学习算法（GBM）开发。

Gradient boosting is a machine learning algorithm that uses a series of decision trees to make predictions. Each decision tree is trained to correct the errors made by the previous trees. This helps to improve the accuracy of the model by making it more robust to noise in the data. GBMs are often used for classification tasks. They are particularly well-suited for problems where the relationship between the features and the target variable is complex. In this case we are using various laboratory imaging and patient information to iteratively fit new models to provide an increasingly accurate estimate of the response variable the presence or absence of choledocholithiasis. Specifically we used gradient
梯度提升是一种机器学习算法，它使用一系列决策树来进行预测。每个决策树都经过训练，以纠正前一棵树所犯的错误。这有助于提高模型的准确性，使其对数据中的噪声更具鲁棒性。GBM通常用于分类任务。它们特别适合于特征和目标变量之间关系复杂的问题。在这种情况下，我们使用各种实验室成像和患者信息来迭代拟合新模型，以提供对胆总管结石存在或不存在的响应变量的日益准确的估计。具体来说，我们使用梯度

tree boosting as implemented in the R package GBM. [9 10].
在R包GBM中实现的树提升。[9 10]。

Two models were explored. One model with initial liver enzyme testing (AST ALT alkaline phosphatase total bilirubin) and a second model with both initial liver enzyme levels and a follow-up set of labs in order to determine whether a second set of labs would be improving the diagnostic yield
探讨了两种模式。一个模型包含初始肝酶检测（AST ALT碱性磷酸酶总胆红素），第二个模型包含初始肝酶水平和一组随访实验室，以确定第二组实验室是否会提高诊断率

of the model.
的模型。

The Gini impurity index was used to determine the importance of predictors in modeling classification. Gini impurity index measures the probability of a particular variable being wrongly classified when it is randomly chosen. The decrease in the Gini impurity index after the inclusion of each of the predictors was used to calculate an importance score indicating improvement in
基尼杂质指数用于确定预测因子在建模分类中的重要性。基尼杂质指数衡量随机选择特定变量时被错误分类的概率。将每个预测因素纳入后，基尼不洁指数的下降用于计算重要性评分，表明

classification when included in the model.
分类时包含在模型中。

Performance of GBM in estimating choledocholithiasis presence was then evaluated using 10- fold cross-validation. 10-fold cross-validation is a methodology in machine learning in which the dataset is split into a learning or model fitting set and a test set to determine model performance. The model fitting procedure was performed a total of ten times with each fit being performed on a training set of 90% of the total dataset selected at random with the remaining 10% used for validation. Model
然后使用10倍交叉验证评价GBM在估计胆总管结石存在方面的性能。10-折叠交叉验证是机器学习中的一种方法，其中数据集被分成学习或模型拟合集和测试集以确定模型性能。模型拟合程序总共进行了10次，每次拟合都是在随机选择的总数据集的90%的训练集上进行的，剩余的10%用于验证。模型

performance was also determined using the mean area under the receiver operating curve (AUC).
还使用受试者工作曲线下平均面积（AUC）确定性能。

Statistical significance for the AUC were calculated in R with the roc() function which is based on the DeLong method [11]. Influential predictors of choledocholithiasis were further identified based on the relative variable importance metric which is a measure based on the number of times a variable is selected for splitting weighted by the squared improvement to the model as a result of each split and averaged over all trees (Figure 1) [9] [12]. Our fit GBM model was implemented in a computer-based
使用基于DeLong方法的roc（）函数计算R中AUC的统计学显著性[11]。基于相对变量重要性度量进一步识别胆总管结石的影响预测因子，相对变量重要性度量是一种基于选择变量进行拆分的次数的度量，通过每次拆分的结果对模型的平方改善进行加权，并在所有树上取平均值（图1）[9] [12]。我们的适合GBM模型是在一个基于计算机的

application risk-calculator and decision-making tool (Figure 2).
应用程序风险计算器和决策工具（图2）。

Results
结果

Baseline Characteristics

A total of 1 378 patients were included in the model. Patients with clinical cholangitis were removed from the study (143 9.4% of patients). The average age of participants was 43.3 years 844 (55.5%) were female 247 (17.9%) had acute pancreatitis. 800 patients (58.1%) had a dilated common bile duct (CBD >6mm) and 461 (33.5%) patients had a bile duct stone on ultrasound. Total bilirubin was elevated in 712 patients (51.7%) based on their initial laboratories. AST was elevated in 940 (68.2%) ALT in 903 (65.5%) and alkaline phosphatase in 883 (64.1%) patients on follow-up. The baseline
共有1378例患者被纳入该模型。临床胆管炎患者从研究中排除（143 9.4%的患者）。参与者的平均年龄为43.3岁，其中844人（55.5%）为女性，247人（17.9%）患有急性胰腺炎。800例患者（58.1%）有胆总管扩张（CBD >6mm），461例患者（33.5%）超声显示胆管结石。根据初始实验室结果，712例患者（51.7%）的总胆红素升高。随访时，940例（68.2%）患者AST升高，903例（65.5%）患者ALT升高，883例（64.1%）患者碱性磷酸酶升高。基线

characteristics of our patient population are summarized in Table 1.
我们的患者群体的特征总结在表1中。

Study Outcomes

The Gini impurity index was calculated after the inclusion of each of the predictors was added to the model to calculate an importance score for all variables included in the study. In the model with one set of laboratory values eight variables were identified as important independent predictors of choledocholithiasis (Figure 1). In the model with two sets of laboratory values twelve variables were identified as important independent predictors of choledocholithiasis (Supplemental Figure 1A). The finding of a bile duct stone on ultrasound was the single strongest predictor of choledocholithiasis in
在将每个预测因子加入模型后计算基尼杂质指数，以计算研究中包括的所有变量的重要性评分。在具有一组实验室值的模型中，将8个变量确定为胆总管结石的重要独立预测因子（图1）。在具有两组实验室值的模型中，12个变量被确定为胆总管结石的重要独立预测因子（补充图1A）。超声发现胆管结石是胆总管结石的最强预测因素，

both models. Interestingly follow-up total bilirubin and alkaline phosphatase levels were the next

strongest predictors in the two-lab model (Supplemental Figure 1A) whereas bilirubin and alkaline phosphatase levels were the next strongest predictors of BDS in the one lab value model (Figure 1).
在双实验室模型中，胆红素和碱性磷酸酶水平是最强的预测因子（补充图1A），而在单实验室值模型中，胆红素和碱性磷酸酶水平是BDS的次强预测因子（图1）。

Other important predictors include the presence of acute pancreatitis age and CBD greater than 6mm.
其他重要的预测因素包括急性胰腺炎、年龄和CBD大于6mm。

Performance of the GBM machine learning model was only slightly improved with the addition of a second set of lab tests based on the receiver operating characteristic. On a 10-fold cross-validation the GBM model with two sets of labs had an AUC 0.792 (Supplemental Figure 1B) whereas the GBM model with one set of labs had an AUC 0.786 (Figure 3). We additionally tested a version of the machine learning model where we incorporated the difference (i.e. the delta) in lab values to determine whether the change in lab values for AST ALT alkaline phosphatase and total bilirubin during the hospitalization improved model prediction of BDS (i.e. the delta). Interestingly inclusion of the delta in labs produced worse model performance with an AUC 0.768 (Supplemental Figure 1B). Because of the only slight improvement in model performance and the increased complexity for clinical implementation that a second set of lab values requires we chose to use the GBM model that required a single set of labs for further evaluation and we thus call this the “GBM” machine learning model in
GBM机器学习模型的性能仅在添加基于接收器操作特性的第二组实验室测试后略有改善。在10倍交叉验证中，具有两组实验室的GBM模型的AUC 0.792（补充图1B），而具有一组实验室的GBM模型的AUC 0.786（图3）。我们还测试了一个版本的机器学习模型，其中我们将实验室值的差异（即Δ）纳入其中，以确定住院期间AST ALT碱性磷酸酶和总胆红素的实验室值变化是否改善了BDS的模型预测（即Δ）。有趣的是，在实验室中纳入Δ产生了更差的模型性能，AUC 0.768（补充图1B）。 由于模型性能仅有轻微改善，并且第二组实验室值要求临床实施的复杂性增加，因此我们选择使用需要单组实验室进行进一步评估的GBM模型，因此我们将其称为“GBM”机器学习模型。

the rest of the manuscript.
手稿的其余部分。

The GBM machine learning model performed better than the original (2010) revised (2019) ASGE guidelines and the ESGE guidelines. On a 10-fold cross-validation the GBM model (AUC 0.786± 0.06) performed better than the ASGE 2010 guidelines (AUC 0.626±0.03) the updated 2019 ASGE guidelines (AUC 0.623±0.03) and the ESGE guidelines (AUC 0.666±0.02). There was a statistically significant difference for AUC for the GBM model compared to all other models tested (p<0.01). (Figure 3). Specifically the sensitivity (70.3±3.2%) and specificity (72.3±3.9%) of the GBM performed better than the 2010 ASGE (sensitivity 57.6±3.25% specificity 67.6±4.15%) and the 2019 updated ASGE
GBM机器学习模型的表现优于原始（2010）修订（2019）ASGE指南和ESGE指南。在10倍交叉验证中，GBM模型（AUC 0.786± 0.06）优于ASGE 2010指南（AUC 0.626±0.03）、更新的2019年ASGE指南（AUC 0.623±0.03）和ESGE指南（AUC 0.666±0.02）。GBM模型的AUC与测试的所有其他模型相比存在统计学显著差异（p<0.01）。（图3）。具体而言，GBM的灵敏度（70.3±3.2%）和特异性（72.3±3.9%）优于2010年ASGE（灵敏度57.6±3.25%，特异性67.6±4.15%）和2019年更新的ASGE

guidelines (sensitivity 61.9±3.4% and specificity 62.8±4.1%). The ESGE guidelines notably had the
敏感性61.9±3.4%，特异性62.8±4.1%。ESGE指南特别指出，

highest sensitivity 86.2±3.5% with the lowest specificity 46.9±3.0%. The positive predictive value (PPV) for the GBM was higher (78.1± 3.0%) than the 2010 ASGE guidelines (70.0±3.3%) or the 2019 updated ASGE guidelines (70.7% ±3.4). The negative predictive value (NPV) was also better with the GBM (63.4% ±3.9) than the 2010 ASGE guidelines (54.9% ±4.2) or the 2019 updated ASGE guidelines (53.1%±3.9). The ESGE guidelines had the highest PPV (83.3±3.5%) though the lowest NPV (52.6±3.3%) (Figure 4). The GBM model correctly categorized 22% of patients directed to unnecessary ERCP by the 2019 ASGE guidelines and appropriately recommended 48% of ERCPs incorrectly rejected by the ESGE guidelines as the next step in the management. An intuitive computer-based risk-calculator and decision-making tool was created to aid in clinician use of the GBM model. This tool allows the user to enter each of the eight important variables included in the model. After these values are entered the probability of
最高灵敏度为86.2±3.5%，最低特异度为46.9± 3.0%。GBM的阳性预测值（PPV）（78.1± 3.0%）高于2010年ASGE指南（70.0±3.3%）或2019年更新的ASGE指南（70.7% ±3.4）。GBM的阴性预测值（NPV）（63.4% ±3.9）也优于2010年ASGE指南（54.9% ±4.2）或2019年更新的ASGE指南（53.1%±3.9）。ESGE指南的PPV最高（83.3±3.5%），但NPV最低（52.6±3.3%）（图4）。根据2019年ASGE指南，GBM模型正确分类了22%的不必要ERCP患者，并适当推荐了ESGE指南错误拒绝的48%的ERCP作为下一步管理。一个直观的基于计算机的风险计算器和决策工具，以帮助临床医生使用GBM模型。该工具允许用户输入模型中包含的八个重要变量中的每一个。 输入这些值后，

choledocholithiasis presence and a recommendation regarding ERCP are reported (Figure 2).
报告了胆总管结石的存在和关于ERCP的建议（图2）。

Discussion
讨论

We developed a machine learning model which was trained using data from 1 378 patients from two previously published retrospective studies to develop a tool that predicts probability of choledocholithiasis presence and provides a recommendation regarding ERCP. We tested three versions of our model one with lab tests from initial presentation (Figure 3) a second model that incorporated initial lab tests and follow-up labs to see whether a second set of labs improved prediction and a third model that incorporated a “delta” in labs (the difference between the first and second set of labs) (Supplemental Figure 1). The two-lab test model (Supplemental Figure 1) only slightly improves model performance and given several extra inputs; we felt the single lab test model was equivalent (Figure 3). Additionally incorporation of the “delta” lab values actually worsened model performance (Supplemental Figure 1B). These findings are consistent with other studies which did not
我们开发了一种机器学习模型，该模型使用来自两项先前发表的回顾性研究的1378例患者的数据进行训练，以开发一种预测胆总管结石存在概率的工具，并提供关于ERCP的建议。我们测试了我们的模型的三个版本，一个是从最初的演示开始进行实验室测试（图3），第二个模型结合了最初的实验室测试和后续实验室，以查看第二组实验室是否改善了预测，第三个模型在实验室中加入了“delta”（第一组和第二组实验室之间的差异）（补充图1）。双实验室测试模型（补充图1）仅略微提高了模型性能，并提供了几个额外的输入;我们认为单实验室测试模型是等效的（图3）。此外，纳入“delta”实验室值实际上使模型性能恶化（补充图1B）。这些发现与其他研究一致，

find improved prediction of the presence of stones on ERCP with dynamic lab testing[7].
通过动态实验室测试，发现ERCP上结石存在的预测改善[7]。

Our machine learning model has the highest diagnostic performance based on AUC compared to 2010 and 2019 ASGE as well as the ESGE guidelines (Figure 3). Our machine learning model has the highest sensitivity accuracy PPV and NPV compared to 2010 and 2019 ASGE as well as the ESGE guidelines. The only test characteristic it did not surpass was the specificity of the ESGE guidelines (72.3% in GBM model vs 86.2% ESGE guidelines). The ESGE guidelines carry the strictest indication to proceed with ERCP – requiring clinical cholangitis or BDS on imaging in order to proceed directly to ERCP and the high specificity is at the cost of the lowest sensitivity of all guidelines which is per our
与2010年和2019年ASGE以及ESGE指南相比，我们的机器学习模型具有基于AUC的最高诊断性能（图3）。与2010年和2019年ASGE以及ESGE指南相比，我们的机器学习模式具有最高的灵敏度准确性PPV和NPV。它唯一没有超越的测试特征是ESGE指南的特异性（GBM模型为72.3%，ESGE指南为86.2%）。ESGE指南具有进行ERCP的最严格适应症-需要临床胆管炎或成像上的BDS才能直接进行ERCP，并且高特异性是以所有指南中最低的灵敏度为代价的，这是根据我们的研究。

analysis 46.9% (Figure 4).

The optimal approach to diagnosis and treatment of choledocholithiasis remains unclear but the advent of non-invasive evaluation of the biliary system has led ERCP to be used more judiciously. Given the development of MRCP and EUS ERCP is now largely reserved for therapeutic rather than diagnostic approaches[13]. The appropriate identification of patients’ risk for choledocholithiasis and thus their exposure to ERCP is a pressing concern for gastroenterology. Clinical risk stratification tools aim to stratify which patients should go directly to ERCP. The ASGE’s 2010 proposed classification of patients into high-risk intermediate-risk and low-risk categories was an important first step to appropriately allocate the use of ERCP. Yet the 2010 classification had well-studied limitations in accuracy. One study estimated a 62% sensitivity and 47% specificity[14] and another study estimated
胆总管结石的最佳诊断和治疗方法仍不清楚，但非侵入性评价胆道系统的出现导致ERCP更明智地使用。鉴于MRCP和EUS的发展，ERCP现在主要用于治疗而不是诊断方法[13]。正确识别胆总管结石患者的风险，从而暴露于ERCP是胃肠病学的一个紧迫问题。临床风险分层工具旨在对哪些患者应直接进行ERCP进行分层。ASGE在2010年提出的将患者分为高风险、中风险和低风险类别是适当分配ERCP使用的重要第一步。然而，2010年的分类在准确性方面存在着充分研究的局限性。一项研究估计敏感性为62%，特异性为47%[14]，另一项研究估计

55% sensitivity and 69% specificity identifying choledocholithiasis[8].
诊断胆总管结石的敏感性为55%，特异性为69%[8]。

Considering several studies showing improved specificity with bile duct dilation and bilirubin levels [15 16] the ASGE narrowed their high-risk criteria in 2019. A total bilirubin >4 mg/dL now only satisfies high-risk criteria if it is accompanied by a dilated CBD on US/cross-sectional imaging[5]. To reduce unnecessary diagnostic ERCPs the ASGE recommends that only patients satisfying high-risk
考虑到几项研究显示胆管扩张和胆红素水平的特异性提高[15 16]，ASGE在2019年缩小了其高风险标准。总胆红素>4 mg/dL现在仅满足高风险标准，如果它伴有US/横断面成像上的扩张CBD [5]。为了减少不必要的诊断性ERCP，ASGE建议只有满足高风险要求的患者才能接受ERCP。

criteria proceed to ERCP. A retrospective evaluation of 1042 patients using these new guidelines
符合标准的进行ERCP。使用这些新指南对1042例患者进行回顾性评价

demonstrated the specificity and positive predictive value (PPV) of ASGE high likelihood criteria were 96.9% (95 % confidence interval [CI] 95.4 – 98.0) and 89.6% (95% CI 85.2 - 92.8) for choledocholithiasis
ASGE高IKK标准诊断胆总管结石的特异性和阳性预测值分别为96.9%（95%CI 95.4 ~ 98.0）和89.6%（95%CI 85.2 ~ 92.8

validating the clinical utility of new ASGE criteria for predicting choledocholithiasis[17].
验证新ASGE标准预测胆总管结石的临床效用[17]。

Spontaneous passage of the bile duct stone was shown to occur in over 50% of the patients presenting with obstructive jaundice especially with small stone sizes. [18] The liver function laboratory tests are therefore expected to improve with such spontaneous stone passage and a dynamic assessment of these laboratory tests may help identify patients with such spontaneous stone passage thereby avoiding unnecessary ERCP procedures. [19] Interestingly two cohort studies did not identify a statistical significance for such dynamic assessment of liver function laboratory tests likely due to the limitations with statistical analysis but the use of machine learning-based methods in our current study
超过50%的梗阻性黄疸患者，尤其是小结石，会发生胆管结石自发通过。[18]因此，预期肝功能实验室检查会随着自发性结石通过而改善，对这些实验室检查的动态评估可能有助于识别自发性结石通过的患者，从而避免不必要的ERCP手术。[19]有趣的是，两项队列研究没有发现肝功能实验室检查的动态评估具有统计学意义，这可能是由于统计分析的局限性，但我们目前的研究使用了基于机器学习的方法。

identified this to be among the most important predictors for the presence of choledocholithiasis.
确定这是胆总管结石最重要的预测因素之一。

The introduction of artificial intelligence represents an additional opportunity for improvement in risk stratification. To our knowledge there have been only two other applications of artificial intelligence to the prediction of choledocholithiasis [20 21]. Jovanovic and colleagues constructed an artificial neural network model to see if it could improve the accuracy of selecting patients for ERCP. They applied this model to 291 patients prospectively who underwent ERCP after being referred with firm suspicion for choledocholithiasis. 80% of patients had choledocholithiasis on ERCP. This model had a positive predictive value of 92.3% and a negative predictive value of 69.6% with respect to finding a stone on ERCP. This study has several limitations. the authors included only those with a firm suspicion of choledocholithiasis based on clinical and/or biochemical data to develop markers their populations were enriched with a very high (80%) of patients having choledocholithiasis a substantial number of
人工智能的引入代表了改善风险分层的额外机会。据我们所知，人工智能在胆总管结石预测方面只有另外两个应用[20 21]。Jovanovic及其同事构建了一个人工神经网络模型，看看它是否可以提高选择ERCP患者的准确性。他们将该模型应用于291例患者，这些患者在明确怀疑胆总管结石后接受了ERCP。80%的患者在ERCP检查中发现胆总管结石。该模型在ERCP上发现结石的阳性预测值为92.3%，阴性预测值为69.6%。这项研究有几个局限性。 作者仅包括那些根据临床和/或生化数据明确怀疑胆总管结石的患者，以开发标记物，他们的人群富含非常高（80%）的胆总管结石患者，

patients with stones were misclassified as suggested by the low negative predictive value.
阴性预测值低提示结石患者被错误分类。

Golub et al. reported a single-center experience and only included patients who had a cholecystectomy in addition to either intraoperative cholangiogram or preoperative ERCP which substantially limited generalizability[21]. Importantly neither study compared their work against the ASGE or ESGE guidelines— though admittedly one pre-dated the guidelines—nor translated their
Golub等人报告了一项单中心经验，仅纳入了除术中胆管造影或术前ERCP外还接受胆囊切除术的患者，这大大限制了普遍性[21]。重要的是，这两项研究都没有将他们的工作与ASGE或ESGE指南进行比较-尽管无可否认的是，一项研究早于指南-也没有将他们的工作翻译成

findings into a practical clinical tool.
将研究成果转化为实用的临床工具。

Though machine learning is advancing the field of gastroenterology a recent review has demonstrated there have been few inroads into the actual clinical setting[22]. We addressed that gap. We sought to provide an AI-based risk-estimation of choledocholithiasis that was more accurate than the existing ASGE framework and provide proof-of-principle that it could be translated into a clinically relevant tool. We took advantage of a well-known machine learning algorithm GBM which is an aptly named supervised decision tree learning algorithm [6 10]. Our model correctly avoided 22% of ERCPs recommended by the 2019 ASGE guidelines and appropriately recommended 48% of ERCPs incorrectly
虽然机器学习正在推进胃肠病学领域的发展，但最近的一项综述表明，在实际的临床环境中很少有进展[22]。我们解决了这个差距。我们试图提供一种基于人工智能的胆总管结石风险估计，比现有的ASGE框架更准确，并提供原理证明，可以将其转化为临床相关工具。我们利用了一个著名的机器学习算法GBM，这是一个恰当的命名监督决策树学习算法[6 10]。我们的模型正确地避免了2019年ASGE指南推荐的22%的ERCP，并正确地推荐了48%的ERCP

rejected by the ESGE guidelines providing the most accurate BDS decision tool to date.
被ESGE指南拒绝，提供了迄今为止最准确的BDS决策工具。

This study has several strengths. It relies on diverse data sets drawn from different medical centers thus enhancing our study’s applicability. These novel machine learning-based statistical methods improve the accuracy of choledocholithiasis prediction compared to previously explored and established techniques. Additionally our analysis translates to a user-friendly interface that can be applied in a real- life setting and in real-time. Finally the data input into our risk estimation tool is based on routinely
这项研究有几个优点。它依赖于来自不同医疗中心的不同数据集，从而增强了我们研究的适用性。与以前探索和建立的技术相比，这些新的基于机器学习的统计方法提高了胆总管结石预测的准确性。此外，我们的分析转化为一个用户友好的界面，可以应用在一个真实的生活设置和实时。最后，输入到我们的风险评估工具中的数据是基于常规的

available parameters without requiring additional diagnostic testing.
无需额外的诊断测试即可获得可用参数。

A few limitations however deserve consideration. First our study population had shortcomings related to the eligibility criteria of the cohort studies that we relied on which were retrospective in nature. However the multi-center nature of these cohorts provides more pragmatic data reflective of
然而，一些限制值得考虑。首先，我们的研究人群存在与我们所依赖的回顾性队列研究的合格标准相关的缺点。然而，这些队列的多中心性质提供了更实用的数据，

the real-world setting. Additionally we were unable to include EUS and MRCP in the decision making
现实世界的设定此外，我们无法将EUS和MRCP纳入决策

which are essential pre-ERCP screening tools due to limitations with the data. We however aim to expand this proof-of-concept study by including additional data. For example by use of age-adjusted upper limit for common bile duct dilation rather than a single cut-off of 6mm [23]. Additional model validation in independent cohorts will be important to improve validity of our approach. Lastly the cohorts used had a relatively high prevalence of choledocholithiasis. Future studies to test the prediction
由于数据的限制，这是ERCP前必不可少的筛查工具。然而，我们的目标是通过包括额外的数据来扩展这一概念验证研究。例如，通过使用胆总管扩张的年龄调整上限，而不是6mm的单一截止值[23]。在独立队列中进行额外的模型验证对于提高我们方法的有效性非常重要。最后，使用的队列具有相对较高的胆总管结石患病率。未来的研究将测试预测

model in lower prevalence populations will be important validation.
模型在低患病率人群中的应用将是重要的验证。

In conclusion we developed a machine learning-based choledocholithiasis risk estimation tool that can provide real-time personalized objective probability of the presence of choledocholithiasis in a practical way that appears to be superior to existing guidelines. The study demonstrates that the GBM machine learning model may help to screen patients to identify those at higher risk of having CBD stones and who may be subjected to direct ERCP or may be screened using EUS / MRCP followed by subsequent ERCP as may be required based on other parameters. At this time this is a proof-of- principle study and this risk estimation tool will need to be further clinically validated—ideally in a
总之，我们开发了一种基于机器学习的胆总管结石风险估计工具，可以以一种实用的方式提供胆总管结石存在的实时个性化客观概率，似乎上级现有的指南。该研究表明，GBM机器学习模型可能有助于筛选患者，以识别患有CBD结石的风险较高的患者，这些患者可能会接受直接ERCP，或者可以使用EUS / MRCP进行筛选，然后根据其他参数进行后续ERCP。目前，这是一项原理验证研究，该风险估计工具需要进一步临床验证-理想情况下，

prospective trial— before it can be adopted in a widespread manner.
前瞻性试验-才能广泛采用。

Conflicts statement
冲突声明

Venkata Akshintala: Co-founder and Chief Medical Officer Origin Endoscopy Inc.

Mouen Khashab: Advisory board member and consultant for Boston Scientific Olympus Medtronic.
Mouen Khashab：Boston Scientific Olympus Medtronic顾问委员会成员和顾问。

All the other authors have no disclosures
所有其他作者都没有披露

Figure : Importance score for each of the predictors included in the machine learning model.
图：机器学习模型中包含的每个预测因子的重要性得分。

Importance scores were calculated based on the decrease in the Gini impurity index associated with the inclusion of each predictor (US – ultrasound; ALP – alkaline phosphatase; T. bili – total bilirubin; CBD –
重要性评分的计算是基于与纳入每个预测因子相关的基尼杂质指数的降低（US -超声; ALP -碱性磷酸酶; T. bili -总胆红素; CBD -

common bile duct).

Figure 2: Screenshot of the computer-based risk-calculator and decision-making tool. The predictors of bile duct stone (choledocholithiasis) presence are listed on top and the probability of choledocholithiasis
图2：基于计算机的风险计算器和决策工具的屏幕截图。胆管结石（胆总管结石）存在的预测因素列于顶部，

presence the requirement of ERCP is listed below. (CBD = Common bile duct; US – ultrasound; BDS –
ERCP的要求如下所示。(CBD=胆总管; US -超声; BDS -

bile duct stone; Tbili – total bilirubin; ALP – alkaline phosphatase).

Figure 3: Area under the receiver operating curve (AUC) with 10-fold cross-validation of the machine learning Gradient Boosting Model (GBM) utilizing a single set of biochemical lab tests compared to the 2010 and 2019 ASGE as well as the ESGE guideline-based risk classification in predicting the presence of
图3：与2010年和2019年ASGE以及基于ESGE指南的风险分类相比，使用单组生化实验室测试对机器学习梯度提升模型（GBM）进行10倍交叉验证的受试者工作曲线下面积（AUC），

bile duct stone.

Figure 4: Flowsheet with ERCP decision making in the setting of suspected choledocholithiasis as
图4：在疑似胆总管结石的情况下，ERCP决策的流程图

recommended by original 2010 ASGE guidelines the updated 2019 ASGE guidelines the ESGE guidelines and the machine learning gradient boosting model (GBM) respectively. “ERCP Yes” indicates the model recommends proceeding to ERCP without any additional testing (i.e. MRCP/EUS). “ERCP No” indicates
分别由原始的2010年ASGE指南、更新的2019年ASGE指南、ESGE指南和机器学习梯度提升模型（GBM）推荐。“ERCP是”表示该模型建议进行ERCP而不进行任何额外测试（即MRCP/EUS）。“ERCP编号”表示

no ERCP or further work-up is recommended. The corresponding performance parameters are listed
不建议进行ERCP或进一步检查。列出了相应的性能参数

below. (Stone = choledocholithiasis PPV = positive predictive value NPV = negative predictive value).

`Supplemental Figure : Importance score and area under the receiver operating curve analysis for a
“补充图：

two-lab test machine learning model. A) Importance score for each of the predictors included in the
双实验室测试机器学习模型。A）包括在预测因子中的每个预测因子的重要性得分

two-lab test machine learning model. Importance scores were calculated based on the decrease in the Gini impurity index associated with the inclusion of each predictor. B) Area under the receiver operating curve (AUC) with 10-fold cross-validation of the two-lab test machine learning Gradient Boosting Model
双实验室测试机器学习模型。重要性评分是根据与纳入每个预测因子相关的基尼杂质指数的降低计算的。B）具有双实验室测试机器学习梯度提升模型的10倍交叉验证的受试者工作曲线下面积（AUC）

(GBM) model compared to 2010 and 2019 ASGE as well as the ESGE guidelines.
(GBM)与2010年和2019年ASGE以及ESGE指南相比。

References
参考文献

1. Everhart JE Ruhl CE. Burden of digestive diseases in the United States part I: overall and upper gastrointestinal diseases. Gastroenterology 2009; 136: 376-386. DOI:
1. Everhart JE Ruhl CE.美国消化系统疾病负担第一部分：总体和上消化道疾病。胃肠病学2009; 136：376-386。DOI：

10.1053/j.gastro.2008.12.015

2. Manes G Paspatis G Aabakken L et al. Endoscopic management of common bile duct
2. Manes G Paspatis G Aabakken L等。胆总管内镜治疗

stones: European Society of Gastrointestinal Endoscopy (ESGE) guideline. Endoscopy 2019; 51: 472-491. DOI: 10.1055/a-0862-0346

3. Kochar B Akshintala VS Afghani E et al. Incidence severity and mortality of post-ERCP

pancreatitis: a systematic review by using randomized controlled trials. Gastrointest
胰腺炎：随机对照试验的系统评价Gastrointest

Endosc 2015; 81: 143-149 e149. DOI: 10.1016/j.gie.2014.06.045
Endosc 2015; 81：143-149 e149. DOI：10.1016/j.gie.2014.06.045

4. Committee ASoP Maple JT Ben-Menachem T et al. The role of endoscopy in the

evaluation of suspected choledocholithiasis. Gastrointest Endosc 2010; 71: 1-9. DOI: 10.1016/j.gie.2009.09.041

5. Committee ASoP Buxbaum JL Abbas Fehmi SM et al. ASGE guideline on the role of endoscopy in the evaluation and management of choledocholithiasis. Gastrointest Endosc 2019; 89: 1075-1105 e1015. DOI: 10.1016/j.gie.2018.10.001
5.委员会ASoP Buxbaum JL Abbas Fehmi SM等，ASGE指南，内镜在胆总管结石评价和治疗中的作用。Gastrointest Endosc 2019; 89：1075-1105 e1015. DOI：10.1016/j.gie.2018.10.001

6. Deo RC. Machine Learning in Medicine. Circulation 2015; 132: 1920-1930. DOI: 10.1161/CIRCULATIONAHA.115.001593

7. Yu CY Roth N Jani N et al. Dynamic liver test patterns do not predict bile duct stones. Surg Endosc 2019; 33: 3300-3313. DOI: 10.1007/s00464-018-06620-x

8. Suarez AL LaBarre NT Cotton PB et al. An assessment of existing risk stratification
8.苏亚雷斯AL LaBarre NT Cotton PB等人。现有风险分层的评估

guidelines for the evaluation of patients with suspected choledocholithiasis. Surg Endosc 2016; 30: 4613-4618. DOI: 10.1007/s00464-016-4799-8

9. Friedman JH. Greedy function approximation: a gradient boosting machine. Annals of statistics 2001; 1189-1232.
9.弗里德曼贪婪函数逼近：梯度推进机。统计年鉴2001; 1189-1232。

10. Natekin A Knoll A. Gradient boosting machines a tutorial. Front Neurorobot 2013; 7: 21. DOI: 10.3389/fnbot.2013.00021

11. DeLong ER DeLong DM Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.
11. DeLong ER DeLong DM Clarke-Pearson DL.比较两个或多个相关受试者工作特征曲线下的面积：非参数方法。

Biometrics 1988; 44: 837-845.

12. Elith J Leathwick JR Hastie T. A working guide to boosted regression trees. J Anim Ecol 2008; 77: 802-813. DOI: 10.1111/j.1365-2656.2008.01390.x

13. Ahmed M Kanotra R Savani GT et al. Utilization trends in inpatient endoscopic

retrograde cholangiopancreatography (ERCP): A cross-sectional US experience. Endosc Int Open 2017; 5: E261-E271. DOI: 10.1055/s-0043-102402

14. Adams MA Hosmer AE Wamsteker EJ et al. Predicting the likelihood of a persistent bile duct stone in patients with suspected choledocholithiasis: accuracy of existing guidelines and the impact of laboratory trends. Gastrointest Endosc 2015; 82: 88-93. DOI:
14.亚当斯马霍斯默AE Wamsteker EJ等人预测疑似胆总管结石患者持续性胆管结石的可能性：现有指南的准确性和实验室趋势的影响。胃肠内镜2015; 82：88-93. DOI：

10.1016/j.gie.2014.12.023

15. Kang J Paik KH Lee JC et al. The Efficacy of Clinical Predictors for Patients with Intermediate Risk of Choledocholithiasis. Digestion 2016; 94: 100-105. DOI:
15. Kang J Paik KH Lee JC等人，胆总管结石中度风险患者的临床预测指标的有效性。消化2016; 94：100-105. DOI：

10.1159/000448917

16. He H Tan C Wu J et al. Accuracy of ASGE high-risk criteria in evaluation of patients with suspected common bile duct stones. Gastrointest Endosc 2017; 86: 525-532. DOI:
16.何华谭春武杰等。ASGE高危标准在评价疑似胆总管结石患者中的准确性。胃肠内镜2017; 86：525-532。DOI：

10.1016/j.gie.2017.01.039

17. Jagtap N Hs Y Tandan M et al. Clinical utility of ESGE and ASGE guidelines for prediction of suspected choledocholithiasis in patients undergoing cholecystectomy. Endoscopy
17. Jagtap N Hs Y Tandan M等，ESGE和ASGE指南预测胆囊切除术患者疑似胆总管结石的临床效用。内镜

2020; 52: 569-573. DOI: 10.1055/a-1117-3451

18. Tranter SE Thompson MH. Spontaneous passage of bile duct stones: frequency of
18.特兰特SE汤普森MH。胆管结石自发通过：

occurrence and relation to clinical presentation. Ann R Coll Surg Engl 2003; 85: 174-177. DOI: 10.1308/003588403321661325
与临床表现的关系。Ann R科尔外科工程2003; 85：174-177. DOI：10.1308/003588403321661325

19. Khoury T Adileh M Imam A et al. Parameters Suggesting Spontaneous Passage of Stones from Common Bile Duct: A Retrospective Study. Can J Gastroenterol Hepatol
19. Khoury T Adileh M Imam A等，参数提示胆总管结石自发通过：一项回顾性研究。Can J Gastroenterol Hepatol

2019; 2019: 5382708. DOI: 10.1155/2019/5382708

20. Jovanovic P Salkic NN Zerem E. Artificial neural network predicts the need for
20. Jovanovic P Salkic NN Zerem E.人工神经网络预测需要

therapeutic ERCP in patients with suspected choledocholithiasis. Gastrointest Endosc 2014; 80: 260-268. DOI: 10.1016/j.gie.2014.01.023
疑有胆总管结石患者的治疗性ERCP。胃肠内镜2014; 80：260-268。DOI：10.1016/j.gie.2014.01.023

21. Golub R Cantu R Jr. Tan M. The prediction of common bile duct stones using a neural network. J Am Coll Surg 1998; 187: 584-590. DOI: 10.1016/s1072-7515(98)00241-5

22. Adadi A Adadi S Berrada M. Gastroenterology Meets Machine Learning: Status Quo and Quo Vadis. Adv Bioinformatics 2019; 2019: 1870975. DOI: 10.1155/2019/1870975

23. Beyer G Kasprowicz F Hannemann A et al. Definition of age-dependent reference values for the diameter of the common bile duct and pancreatic duct on MRCP: a
23. Beyer G Kaspericz F Hannemann A et al. MRCP上胆总管和胰管直径的年龄依赖性参考值的定义：a

population-based cross-sectional cohort study. Gut 2023. DOI: 10.1136/gutjnl-2021- 326106
基于人群的横断面队列研究。肠道2023。DOI：10.1136/gutjnl-2021- 326106