Comparison of a chronic kidney disease predictive model for type 2 diabetes mellitus in Malaysia using Cox regression versus machine learning approach.
Clinical kidney journal
Background:Diabetes is one of the leading causes of chronic kidney disease (CKD) and end-stage renal disease. This study aims to develop and validate different risk predictive models for incident CKD and CKD progression in people with type 2 diabetes (T2D). Methods:We reviewed a cohort of people with T2D seeking care from two tertiary hospitals in the metropolitan cities of the state of Selangor and Negeri Sembilan from January 2012 to May 2021. To identify the 3-year predictor of developing CKD (primary outcome) and CKD progression (secondary outcome), the dataset was randomly split into a training and test set. A Cox proportional hazards (CoxPH) model was developed to identify predictors of developing CKD. The resultant CoxPH model was compared with other machine learning models on their performance using C-statistic. Results:The cohorts included 1992 participants, of which 295 had developed CKD and 442 reported worsening of kidney function. Equation for the 3-year risk of developing CKD included gender, haemoglobin A1c, triglyceride and serum creatinine levels, estimated glomerular filtration rate, history of cardiovascular disease and diabetes duration. For risk of CKD progression, the model included systolic blood pressure, retinopathy and proteinuria. The CoxPH model was better at prediction compared with other machine learning models examined for incident CKD (C-statistic: training 0.826; test 0.874) and CKD progression (C-statistic: training 0.611; test 0.655). The risk calculator can be found at https://rs59.shinyapps.io/071221/. Conclusions:The Cox regression model was the best performing model to predict people with T2D who will develop a 3-year risk of incident CKD and CKD progression in a Malaysian cohort.
10.1093/ckj/sfac252
A predictive model for progression of chronic kidney disease to kidney failure.
Tangri Navdeep,Stevens Lesley A,Griffith John,Tighiouart Hocine,Djurdjev Ognjenka,Naimark David,Levin Adeera,Levey Andrew S
JAMA
CONTEXT:Chronic kidney disease (CKD) is common. Kidney disease severity can be classified by estimated glomerular filtration rate (GFR) and albuminuria, but more accurate information regarding risk for progression to kidney failure is required for clinical decisions about testing, treatment, and referral. OBJECTIVE:To develop and validate predictive models for progression of CKD. DESIGN, SETTING, AND PARTICIPANTS:Development and validation of prediction models using demographic, clinical, and laboratory data from 2 independent Canadian cohorts of patients with CKD stages 3 to 5 (estimated GFR, 10-59 mL/min/1.73 m(2)) who were referred to nephrologists between April 1, 2001, and December 31, 2008. Models were developed using Cox proportional hazards regression methods and evaluated using C statistics and integrated discrimination improvement for discrimination, calibration plots and Akaike Information Criterion for goodness of fit, and net reclassification improvement (NRI) at 1, 3, and 5 years. MAIN OUTCOME MEASURE:Kidney failure, defined as need for dialysis or preemptive kidney transplantation. RESULTS:The development and validation cohorts included 3449 patients (386 with kidney failure [11%]) and 4942 patients (1177 with kidney failure [24%]), respectively. The most accurate model included age, sex, estimated GFR, albuminuria, serum calcium, serum phosphate, serum bicarbonate, and serum albumin (C statistic, 0.917; 95% confidence interval [CI], 0.901-0.933 in the development cohort and 0.841; 95% CI, 0.825-0.857 in the validation cohort). In the validation cohort, this model was more accurate than a simpler model that included age, sex, estimated GFR, and albuminuria (integrated discrimination improvement, 3.2%; 95% CI, 2.4%-4.2%; calibration [Nam and D'Agostino χ(2) statistic, 19 vs 32]; and reclassification for CKD stage 3 [NRI, 8.0%; 95% CI, 2.1%-13.9%] and for CKD stage 4 [NRI, 4.1%; 95% CI, -0.5% to 8.8%]). CONCLUSION:A model using routinely obtained laboratory tests can accurately predict progression to kidney failure in patients with CKD stages 3 to 5.
10.1001/jama.2011.451
There is no such thing as a validated prediction model.
BMC medicine
BACKGROUND:Clinical prediction models should be validated before implementation in clinical practice. But is favorable performance at internal validation or one external validation sufficient to claim that a prediction model works well in the intended clinical context? MAIN BODY:We argue to the contrary because (1) patient populations vary, (2) measurement procedures vary, and (3) populations and measurements change over time. Hence, we have to expect heterogeneity in model performance between locations and settings, and across time. It follows that prediction models are never truly validated. This does not imply that validation is not important. Rather, the current focus on developing new models should shift to a focus on more extensive, well-conducted, and well-reported validation studies of promising models. CONCLUSION:Principled validation strategies are needed to understand and quantify heterogeneity, monitor performance over time, and update prediction models when appropriate. Such strategies will help to ensure that prediction models stay up-to-date and safe to support clinical decision-making.
10.1186/s12916-023-02779-w
A Comorbidity Knowledge-Aware Model for Disease Prognostic Prediction.
IEEE transactions on cybernetics
Prognostic prediction is the task of estimating a patient's risk of disease development based on various predictors. Such prediction is important for healthcare practitioners and patients because it reduces preventable harm and costs. As such, a prognostic prediction model is preferred if: 1) it exhibits encouraging performance and 2) it can generate intelligible rules, which enable experts to understand the logic of the model's decision process. However, current studies usually concentrated on only one of the two features. Toward filling this gap, in the present study, we develop a novel knowledge-aware Bayesian model taking into consideration accuracy and transparency simultaneously. Real-world case studies based on four years' territory-wide electronic health records are conducted to test the model. The results show that the proposed model surpasses state-of-the-art prognostic prediction models in accuracy and c-statistic. In addition, the proposed model can generate explainable rules.
10.1109/TCYB.2021.3070227
Growth prediction model for abdominal aortic aneurysms.
The British journal of surgery
BACKGROUND:The most relevant determinant in scheduling monitoring intervals for abdominal aortic aneurysms (AAAs) is maximum diameter. The aim of the study was to develop a statistical model that takes into account specific characteristics of AAA growth distributions such as between-patient variability as well as within-patient variability across time, and allows probabilistic statements to be made regarding expected AAA growth. METHODS:CT angiography (CTA) data from patients monitored at 6-month intervals with maximum AAA diameters at baseline between 30 and 66 mm were used to develop the model. By extending the model of geometric Brownian motion with a log-normal random effect, a stochastic growth model was developed. An additional set of ultrasound-based growth data was used for external validation. RESULTS:The study data included 363 CTAs from 87 patients, and the external validation set comprised 390 patients. Internal and external cross-validation showed that the stochastic growth model allowed accurate description of the distribution of aneurysm growth. Median relative growth within 1 year was 4.1 (5-95 per cent quantile 0.5-13.3) per cent. Model calculations further resulted in relative 1-year growth of 7.0 (1.0-16.4) per cent for patients with previously observed rapid 1-year growth of 10 per cent, and 2.6 (0.3-8.3) per cent for those with previously observed slow growth of 1 per cent. The probability of exceeding a threshold of 55 mm was calculated to be 1.78 per cent at most when adhering to the current RESCAN guidelines for rescreening intervals. An online calculator based on the fitted model was made available. CONCLUSION:The stochastic growth model was found to provide a reliable tool for predicting AAA growth.
10.1093/bjs/znab407
Sepsis Prediction Model for Determining Sepsis vs SIRS, qSOFA, and SOFA.
JAMA network open
Importance:The Sepsis Prediction Model (SPM) is a proprietary decision support tool created by Epic Systems; it generates a predicting sepsis score (PSS). The model has not undergone validation against existing sepsis prediction tools, such as Systemic Inflammatory Response Syndrome (SIRS), Sequential Organ Failure Assessment (SOFA), or quick Sepsis-Related Organ Failure Asessement (qSOFA). Objective:To assess the validity and timeliness of the SPM compared with SIRS, qSOFA, and SOFA. Design, Setting, and Participants:This retrospective cohort study included all adults admitted to 5 acute care hospitals in a single US health system between June 5, 2019, and December 31, 2020. Data analysis was conducted from March 2021 to February 2023. Main Outcomes and Measures:A sepsis event was defined as receipt of 4 or more days of antimicrobials, blood cultures collected within ±48 hours of initial antimicrobial, and at least 1 organ dysfunction as defined by the organ dysfunction criteria optimized for the electronic health record (eSOFA). Time zero was defined as 15 minutes prior to qualifying antimicrobial or blood culture order. Results:Of 60 507 total admissions, 1663 (2.7%) met sepsis criteria, with 1324 electronic health record-confirmed sepsis (699 [52.8%] male patients; 298 [22.5%] Black patients; 46 [3.5%] Hispanic/Latinx patients; 945 [71.4%] White patients), 339 COVID-19 sepsis (183 [54.0%] male patients; 98 [28.9%] Black patients; 36 [10.6%] Hispanic/Latinx patients; and 189 [55.8%] White patients), and 58 844 (97.3%; 26 632 [45.2%] male patients; 12 698 [21.6%] Black patients; 3367 [5.7%] Hispanic/Latinx patients; 40 491 White patients) did not meet sepsis criteria. The median (IQR) age was 63 (51 to 73) years for electronic health record-confirmed sepsis, 69 (60 to 77) years for COVID-19 sepsis, and 60 (42 to 72) years for nonsepsis admissions. Within the vendor recommended threshold PSS range of 5 to 8, PSS of 8 or greater had the highest balanced accuracy for classifying a sepsis admission at 0.79 (95% CI, 0.78 to 0.80). Change in SOFA score of 2 or more had the highest sensitivity, at 0.97 (95% CI, 0.97 to 0.98). At a PSS of 8 or greater, median (IQR) time to score positivity from time zero was 68.00 (6.75 to 605.75) minutes. For SIRS, qSOFA, and SOFA, median (IQR) time to score positivity was 7.00 (-105.00 to 08.00) minutes, 74.00 (-22.25 to 599.25) minutes, and 28.00 (-108.50 to 134.00) minutes, respectively. Conclusions and Relevance:In this cohort study of hospital admissions, balanced accuracy of the SPM outperformed other models at higher threshold PSS; however, application of the SPM in a clinical setting was limited by poor timeliness as a sepsis screening tool as compared to SIRS and SOFA.
10.1001/jamanetworkopen.2023.29729