P4906 - Use of Large Language Model (LLM) Chatbots for the Generation of Specialized Gastrointestinal Diet Meal Plans: A Pilot Study

Bernadette Lamb, MD, Jonathan Herskovitz, MD, PhD, Marta Jonson, RDN, Harlan Sayles, MS, Faruq Pradhan, MBBCh
University of Nebraska Medical Center, Omaha, NE

Introduction: Gastroenterologists commonly recommend dietary modification to patients for various gastrointestinal (GI) diseases: celiac disease, irritable bowel syndrome, eosinophilic esophagitis. With the rise of use of large language model (LLM) chatbots, it is postulated that LLM chatbots could be used to create sample meal plans to augment counseling of patients on dietary modification in addition to referral to a registered dietician (RD). We performed a pilot study to evaluate the use of LLM chatbots for creation of meal plans for select specialized GI diets.

Methods: Sample 3-day meal plans for a celiac diet, 6-food elimination diet (SFED), and low-fermentable oligosaccharides, disaccharides, monosaccharides, and polyols (FODMAP) diet were created by a RD, ChatGPT 4.0, and Gemini Advanced. Meal plans were anonymized to a group of single-blinded RDs from a single center and scored on compliance to their respective specialized GI diet. They were then asked if the meal plan was created by either a RD or LLM chatbot. The Automated Self-Administered 24-hour Dietary Assessment Tool was utilized to extract nutritional data and calculate Healthy Eating Index (HEI) scores for each meal plan. A two-way ANOVA test was used to compare compliance and HEI scores among the groups.

Results: The survey was completed by 11 RDs at a single center. 3 of 18 meal plans were correctly identified by the majority as either RD or LLM chatbot-created. There was not a significant difference between compliance scores for meal plans created by a RD vs. LLM chatbots for a male or female on a celiac diet or a male on a low-FODMAP diet. There was a significant difference in compliance scores for meal plans created for a male on a SFED by an RD vs. ChatGPT 4.0 (MD 18.18, 95% CI 7.53 to 28.83) and Gemini vs. ChatGPT 4.0 (MD 20.45, 95% CI 9.80 to 31.11), for a female on a SFED by a RD vs. Gemini (MD –22.73, 95% CI -33.38 to -12.08) and Gemini vs. ChatGPT 4.0 (MD 22.73, 95% CI 12.08 to 33.38), and for a female on a low-FODMAP diet by an RD vs. ChatGPT 4.0 (MD -15.91, 95% CI –26.56 to –5.26). HEI scores for the meal plans were not significantly different between the groups.

Discussion: Overall, LLM chatbots appear to be useful tools to augment a clinician’s ability to provide counseling to patients regarding dietary modification for various GI disorders. Neither LLM chatbot appeared to be clearly superior based on our study. In addition to referral to a RD, we propose that these LLM chatbots may be used with clinician supervision for further dietary counseling regarding specialized GI diets.


Disclosures:


Bernadette Lamb, MD, Jonathan Herskovitz, MD, PhD, Marta Jonson, RDN, Harlan Sayles, MS, Faruq Pradhan, MBBCh. P4906 - Use of Large Language Model (LLM) Chatbots for the Generation of Specialized Gastrointestinal Diet Meal Plans: A Pilot Study, ACG 2024 Annual Scientific Meeting Abstracts. Philadelphia, PA: American College of Gastroenterology.