•  
  •  
 

Abstract

Objective: Large language models (LLMs) are increasingly investigated to support clinical decision-making. However, their comparative performance against human expertise in pediatric infectious diseases remains insufficiently characterized. This study compared diagnostic accuracy and management score between healthcare professionals, medical students, and multiple LLM configurations. Methods: A cross-sectional comparative study was conducted using 30 standardized pediatric infectious disease cases stratified by difficulty (6 easy, 12 medium, 12 difficult). Each case included three multiple-choice questions: a diagnostic gateway question and two management questions. A conditional scoring framework was applied: management questions were scored only when the diagnostic gateway question was answered correctly. The human cohort included 308 participants, generating 2,450 responses. The LLM cohort included 14 model configurations: ChatGPT (5.1 and 5.2, each in Instant and Thinking modes; 5.3 Instant; 5.4 Thinking); Claude (Opus 3, 4.5, 4.6); Gemini (3.1 Pro, Thinking, Fast); and DeepSeek v3 (Normal, DeepThink). Each was evaluated across five independent sessions (2,100 case responses). Results: The LLM cohort demonstrated significantly higher performance than the human cohort across all endpoints. The adjusted difference in diagnostic accuracy was +25.78 percentage points (95% CI: 22.05–29.52; p < 0.001). The overall score was higher by +36.57 points (95% CI: 32.65–40.50; p < 0.001), and the management score by +22.82 points (95% CI: 19.79–25.85; p < 0.001). Among human participants, pediatric specialists achieved the highest performance (overall score: 79.34); no subgroup reached LLM-level performance. Human performance declined with increasing case difficulty, whereas LLM performance remained relatively stable. Variability among LLM configurations was minimal (range: 98.00%–100.00%). Conclusion: Current-generation LLMs demonstrated superior performance in diagnostic accuracy and management score compared with healthcare professionals and medical students on standardized pediatric infectious disease cases. These findings support their potential role as clinical decision support tools. However, further studies are required to evaluate real-world applicability, safety, and human–AI collaboration in clinical practice.

Download

Share

COinS