Artificial Intelligence-Supported Development of Health Guideline Questions.

Autores da FMUP

• Bernardo Manuel De Sousa Pinto

  Autor

• Rafael José Monteiro Da Silva Vieira

  Autor

• Joana Maria Correia Amaro

  Autor

• Ana Margarida Barbosa Ribeiro Pereira

  Autor

• João De Almeida Lopes Da Fonseca

  Autor

• Ricardo João Cruz Correia

  Autor

Participantes de fora da FMUP

• Marques-Cruz M
• Bognanni A
• Gil-Mata S
• Jankin S
• Pinheiro L
• Mota M
• Giovannini M
• de Las Vecillas L
• Litynska J
• Samolinski B
• Bernstein J
• Dykewicz M
• Hofmann-Apitius M
• Jacobs M
• Papadopoulos N
• Williams S
• Zuberbier T
• Bousquet J
• Schünemann HJ

Unidades de investigação

• Laboratório Associado RISE- Rede de Investigação em Saúde

  

  Investigador

  Fernando Carlos De Landér Schmitt

• Medicina da Comunidade, Informação e Decisão em Saúde

• Recursos Comuns

  

  Investigador

  Guilhermina Maria Da Silva Rêgo

• RISE-Health

  

  Investigador

  Fernando Carlos De Landér Schmitt

Abstract

BACKGROUND: Guideline questions are typically proposed by experts. OBJECTIVE: To assess how large language models (LLMs) can support the development of guideline questions, providing insights on approaches and lessons learned. DESIGN: Two approaches for guideline question generation were assessed: 1) identification of questions conveyed by online search queries and 2) direct generation of guideline questions by LLMs. For the former, the researchers retrieved popular queries on allergic rhinitis using Google Trends (GT) and identified those conveying questions using both manual and LLM-based methods. They then manually structured as guideline questions the queries that conveyed relevant questions. For the second approach, they tasked an LLM with proposing guideline questions, assuming the role of either a patient or a clinician. SETTING: Allergic Rhinitis and its Impact on Asthma (ARIA) 2024 guidelines. PARTICIPANTS: None. MEASUREMENTS: Frequency of relevant questions generated. RESULTS: The authors retrieved 3975 unique queries using GT. From these, they identified 37 questions, of which 22 had not been previously posed by guideline panel members and 2 were eventually prioritized by the panel. Direct interactions with LLMs resulted in the generation of 22 unique relevant questions (11 not previously suggested by panel members), and 4 were eventually prioritized by the panel. In total, 6 of 39 final questions prioritized for the 2024 ARIA guidelines were not initially thought of by the panel. The researchers provide a set of practical insights on the implementation of their approaches based on the lessons learned. LIMITATION: Single case study (ARIA guidelines). CONCLUSION: Approaches using LLMs can support the development of guideline questions, complementing traditional methods and potentially augmenting questions prioritized by guideline panels. PRIMARY FUNDING SOURCE: Fraunhofer Cluster of Excellence for Immune-Mediated Diseases.

Keywords

• Artificial Intelligence; Asthma; Humans; Practice Guidelines as Topic; Rhinitis, Allergic; antiallergic agent; allergic rhinitis; Article; artificial intelligence; ChatGPT; clinician; funding; human; large language model; patient care; practice guideline; questionnaire; scientist; search engine; allergic rhinitis; artificial intelligence; asthma