P0500 - The Reproducibility of the AGREE II Tool in the Methodological Appraisal of Clinical Practice Guidelines for the Management of Barrett’s Esophagus

Denisse Camille Dayto, MD¹, Andrew Sephien, MD², Tea Reljic, MPH³, Farina Klocksieben, MPH³, Samuel Slone, MD³, Ambuj Kumar, MD, MPH³, Wojciech Blonski, MD, PhD^4
1Rutgers New Jersey Medical School, Inverness, FL; ²University of Alabama at Birmingham Hospital, Birmingham, AL; ³University of South Florida Morsani College of Medicine, Tampa, FL; ⁴University of South Florida, James A. Haley VA Hospital, Tampa, FL

Introduction: There have been several clinical practice guidelines (CPG) published for the management of Barrett’s Esophagus (BE) and their methodological quality have been last evaluated in 2016 using the validated Appraisal of Guidelines for Research & Evaluation II (AGREE II) tool. Since then, two CPGs have been published as new and updated guidelines. However, the reproducibility of the AGREE II tool for the assessment of the methodological quality of CPGs for BE has not been assessed. Therefore, the goal of this study is to assess the reproducibility of the AGREE II tool in assessing the methodological quality of CPGs for BE management.

Methods: A systematic search of PubMed and EMBASE databases was performed to identify any CPG published from January 1, 2015 until February 10, 2023, addressing the management of Barrett’s Esophagus. Guideline selection was performed by two reviewers independently. The CPG quality was independently assessed by four reviewers using the AGREE II tool. The AGREE tool contains 23 quality items across 7 domains. The reproducibility was summarized as weighted Fleiss kappa statistic and intraclass correlation coefficient using the SPSS statistical analysis package.

Results: The search yielded 585 citations of which 5 were CPGs included for the critical appraisal. Three of the CPGs were published by the gastroenterology associations in the United States (n=3) and two in Europe (n=2). The overall mean weighted kappa score for was 0.220 (range -0.009-0.449) indicating a fair agreement. The overall intraclass correlation coefficient was 0.341 (range 0.259-0.427) for single-rater measures showing poor reliability of ratings based on a single rater and 0.674 (range 0.583-0.749) for average measures indicating moderate reliability of ratings based on all the raters.

Discussion: The findings of our study show that there is only a fair overall interrater agreement in the methodological quality assessment of clinical practice guidelines for the management of BE. The results suggest the need for further training and education on how to utilize the AGREE II tool in future CPG raters to improve its reproducibility.


Disclosures:


Denisse Camille Dayto, MD¹, Andrew Sephien, MD², Tea Reljic, MPH³, Farina Klocksieben, MPH³, Samuel Slone, MD³, Ambuj Kumar, MD, MPH³, Wojciech Blonski, MD, PhD⁴. P0500 - The Reproducibility of the AGREE II Tool in the Methodological Appraisal of Clinical Practice Guidelines for the Management of Barrett’s Esophagus, ACG 2024 Annual Scientific Meeting Abstracts. Philadelphia, PA: American College of Gastroenterology.