From Static Symptom Checking To Adaptive Dialogue: Design And Evaluation Of An Agentic Llm System For First-Level Medical Triage

Abstract

First-contact medical assessment requires more than generation of a plausible diagnostic label. Safe support depends on gathering sufficient information, distinguishing low-risk from high-risk presentations, recognising red flags and directing the user toward an appropriate level of care. Conventional symptom checkers commonly use fixed pathways, while large language models can interpret free-text descriptions but may reach conclusions before enough information is available. This study presents the design and simulation-based comparative evaluation of LLMDoctor, an agentic conversational prototype for first-level medical assessment and triage support. The architecture combines natural-language symptom entry, information-gap detection, adaptive follow-up questioning, progressive context updating, preliminary differential assessment, red-flag screening, triage recommendation and human escalation. A comparative case-based evaluation framework contrasts a conventional symptom checker, a single-turn LLM baseline and the agentic condition. A structured simulation comprising 120 constructed cases was used to demonstrate the comparative evaluation framework across diagnostic performance, triage agreement, under-triage, red-flag recognition, interaction quality and information completeness. The article contributes a bounded agentic decision-support architecture, separates diagnostic performance from triage safety and proposes a safety-error taxonomy for evaluating first-level medical AI. The framework positions escalation as a core system capability and retains clinician authority for high-risk or uncertain cases.