Accuracy of information given by ChatGPT for patients with Inflammatory Bowel Disease in relation to ECCO guidelines
Autores da FMUP
Participantes de fora da FMUP
- Farrugia, Y.
- Sciberras, M.
- Gordon, H.
- Furfaro, F.
- Allocca, M.
- Torres, J.
- Arebi, N.
- Fiorino, G.
- Iacucci, M.
- Verstockt, B.
- Katsanos, K.
- Fenech, V. A.
- Zammit, S. Chetcuti
- Ellul, P.
Unidades de investigação
Abstract
Background: As acceptance of artificial intelligence [AI] platforms increases, more patients will consider these tools as sources of information. The ChatGPT architecture utilizes a neural network to process natural language, thus generating responses based on the context of input text. The accuracy and completeness of ChatGPT3.5 in the context of inflammatory bowel disease [IBD] remains unclear. Methods: In this prospective study, 38 questions worded by IBD patients were inputted into ChatGPT3.5. The following topics were covered: [1] Crohn's disease [CD], ulcerative colitis [UC], and malignancy; [2] maternal medicine; [3] infection and vaccination; and [4] complementary medicine. Responses given by ChatGPT were assessed for accuracy [1-completely incorrect to 5-completely correct] and completeness [3-point Likert scale; range 1-incomplete to 3-complete] by 14 expert gastroenterologists, in comparison with relevant ECCO guidelines. Results: In terms of accuracy, most replies [84.2%] had a median score of >= 4 (interquartile range [IQR]: 2) and a mean score of 3.87 [SD: +/- 0.6]. For completeness, 34.2% of the replies had a median score of 3 and 55.3% had a median score of between 2 and <3. Overall, the mean rating was 2.24 [SD: +/- 0.4, median: 2, IQR: 1]. Though groups 3 and 4 had a higher mean for both accuracy and completeness, there was no significant scoring variation between the four question groups [Kruskal-Wallis test p > 0.05]. However, statistical analysis for the different individual questions revealed a significant difference for both accuracy [p < 0.001] and completeness [p < 0.001]. The questions which rated the highest for both accuracy and completeness were related to smoking, while the lowest rating was related to screening for malignancy and vaccinations especially in the context of immunosuppression and family planning. Conclusion: This is the first study to demonstrate the capability of an AI-based system to provide accurate and comprehensive answers to real-world patient queries in IBD. AI systems may serve as a useful adjunct for patients, in addition to standard of care in clinics and validated patient information resources. However, responses in specialist areas may deviate from evidence-based guidance and the replies need to give more firm advice.
Dados da publicação
- ISSN/ISSNe:
- 1873-9946, 1876-4479
- Tipo:
- Meeting Abstract
- Páginas:
- 1257-1258
- Link para outro recurso:
- www.scopus.com
JOURNAL OF CROHNS & COLITIS Oxford University Press
Citações Recebidas na Scopus: 2
Documentos
- Não há documentos
Filiações
Keywords
- Artificial intelligence; inflammatory bowel disease; health communication; patient education
Proyectos asociados
Gut microbiome and IBD therapy: an interplay?
Investigador Principal: Fernando José Magro Dias
Estudo Observacional Académico (GutIBD) . 2021
Iron deficiency and inflammatory bowel disease. Correlation with inflammation and Vitamin D status.
Investigador Principal: Fernando José Magro Dias
Estudo Observacional Académico (IronIBD) . 2021
Looking 4WARD: The role of dipeptidyl peptidase 4 (DPP-4) in inflammatory bowel disease (IBD) as a novel biomarker for predicting disease activity and monitoring response to therapy in IBD patients.
Investigador Principal: Fernando José Magro Dias
Estudo Observacional Académico (4WARD) . 2021
Isolated Ulceration of Crohn’s Anastomosis
Investigador Principal: Fernando José Magro Dias
Estudo Clínico Académico . 2021
Histologic Features of Colon Biopsies (Geboes Score) Associated With Progression of Ulcerative Colitis for the First 36 Months After Biopsy
Investigador Principal: Fernando José Magro Dias
Estudo Clínico Académico . 2022
Burden of Inflammatory Bowel Disease in Portugal
Investigador Principal: Fernando José Magro Dias
Estudo Clínico Académico (Inflammatory Bowel) . 2023
Therapeutic Drug Monitoring: An Emergent Approach in Inflammatory Bowel Disease
Investigador Principal: Fernando José Magro Dias
Estudo Clínico Académico . 2021
Contributo da endoscopia, biomarcadores e imagiologia na evolução clinica dos doentes com doença inflamatória intestinal
Investigador Principal: Fernando José Magro Dias
Estudo Clínico Académico . 2019
Citar a publicação
Farrugia,Y,Sciberras,M,Gordon,H,Furfaro,F,Allocca,M,Torres,J,Arebi,N et al. Accuracy of information given by ChatGPT for patients with Inflammatory Bowel Disease in relation to ECCO guidelines. GREAT CLARENDON ST, OXFORD OX2 6DP, ENGLAND:Oxford University Press. 2024 p.p. 1257-1258.