P3199 - Evaluation of GPT-4o's Ability to Identify Gluten-Containing Foods in Dietary Recall

Andrew Ford, MD¹, Gengqing Song, MD^2
1Case Western Reserve University / MetroHealth, Cleveland, OH; ²MetroHealth, Cleveland, OH

Introduction: Celiac disease (CeD) is an autoimmune disease affecting the small intestine triggered by the gluten, which is found in many dietary grains such as wheat, barley, and rye. Treatment requires a gluten-free diet, which can be difficult for patients to follow for numerous reasons such as knowledge gaps in foods that may contain the protein. Our study sought to evaluate whether the artificial intelligence tool GPT-4o could identify gluten-containing foods in 24-hour dietary recalls.

Methods: Fifty 24-hour dietary recall examples containing variable amounts of gluten-containing and gluten-free items were entered into GPT-4o, which was instructed to identify items that definitively contained gluten in each instance as well as gluten-free items. The flagged meals were compared against expert manual review to calculate true positive (TP), false positive (FP), true negative (TN), and false negative (FN) rates of identification.

Results: Fifty dietary recall examples were created and entered into GPT-4o. These recalls contained on average 1.15 gluten-containing items and 8.08 gluten-free items in each recall. GPT-4o incorrectly identified 8 gluten-containing items as gluten-free. It correctly identified all gluten-free items. The true positive rate was 98.1%, while the true negative rate was 100%. Conversely, the false positive rate was 0.0%, while the false negative rate was 1.9%.

Discussion: GPT-4o accurately identifies gluten-containing and gluten-free items throughout 24-hour dietary recalls. These data suggest that a generative AI-driven tool could help celiac patients identify sources of gluten in their diets. Further research is warranted to refine the tool and assess its ability to generate recommendations to replace gluten-containing items.



Disclosures:


Andrew Ford, MD¹, Gengqing Song, MD². P3199 - Evaluation of GPT-4o's Ability to Identify Gluten-Containing Foods in Dietary Recall, ACG 2024 Annual Scientific Meeting Abstracts. Philadelphia, PA: American College of Gastroenterology.