Seth Gordhandas Sundardas Medical College and KEM Hospital Birmingham, AL
Apoorva Doshi, MBBS1, Dhvani Adhvaryu, PharmD2, Rajiv Mehta, MBBS, MD2 1Seth Gordhandas Sundardas Medical College and KEM Hospital, Birmingham, AL; 2Surat Institute of Digestive Sciences Hospital, Surat, Gujarat, India
Introduction: Early risk assessment is imperative in acute pancreatitis (AP) to improve outcomes. Previous studies have identified the bedside index of severity of AP (BISAP) as a robust index for accurate early risk assessment. We aimed to develop and compare the performance of machine learning (ML)-based models with BISAP in predicting AP outcomes using information available within 48 hours of admission.
Methods: We included AP patients from our tertiary care center and used the following 8 features to develop our logistic regression (LR) and random forests (RF) ML models: age, gender, etiology, BISAP, C-reactive protein (CRP) levels within 48 hours of admission, hemoglobin, white blood cell counts, and creatinine levels at admission. The dataset was randomly split into a 70:30 training-testing ratio. Outcomes of interest such as 30-day mortality rate and increased length of stay (LOS) (defined as more than 5 days) were predicted using LR and RF models, and their performance was compared with BISAP. The best model for each outcome was selected using the area under the receiver operating characteristic curve (AUROC) performance on the testing dataset. The relative importance of features in predicting each outcome was analyzed using SHapley Additive exPlanations (SHAP) method.
Results: In total, 1100 patients with AP (mean age 43.06 ± 17.20 years; 73.9% men) were included. The 30-day mortality rate was 6.2% (n = 68 patients), and the median LOS was 5 days. The RF model (Fig 1.A) achieved a better AUROC of 0.924 (95% Confidence Interval (CI): 0.854-0.996) for 30-day mortality compared to the LR model (AUROC: 0.908; CI: 0.831-0.985) and BISAP (AUROC: 0.905; CI: 0.822-0.988, p = 0.24). For increased LOS (Fig 1.B), the RF model achieved an AUROC of 0.773 (95% CI: 0.718-0.829) compared to an AUROC of 0.771 (CI: 0.715-0.828) for the LR model and an AUROC of 0.615 (CI: 0.553-0.676, p < 0.05) for BISAP. SHAP analysis of the RF model revealed BISAP, creatinine, and CRP to be the important variables for mortality prediction, whereas CRP, hemoglobin, and BISAP were important variables for predicting increased LOS.
Discussion: ML-based models accurately predicted 30-day mortality and performed similarly to BISAP in that regard. Moreover, they significantly outperformed BISAP in predicting increased LOS. These models represent a novel approach towards risk stratification in AP and have the potential to address the limitations of traditional models. Further studies are needed to establish their clinical utility.
Figure: Figure 1: A. Receiver operating characteristic curve comparing performance of RF, LR and BISAP for prediction of 30-day mortality. B. Receiver operating characteristic curve comparing performance of RF, LR and BISAP for prediction of length of stay > 5 days.
Disclosures:
Apoorva Doshi indicated no relevant financial relationships.
Dhvani Adhvaryu indicated no relevant financial relationships.
Rajiv Mehta indicated no relevant financial relationships.
Apoorva Doshi, MBBS1, Dhvani Adhvaryu, PharmD2, Rajiv Mehta, MBBS, MD2. P3456 - Predicting Outcomes in Acute Pancreatitis - Machine Learning Versus BISAP, ACG 2024 Annual Scientific Meeting Abstracts. Philadelphia, PA: American College of Gastroenterology.
