Performance of Large Language Models in Image-Based Evaluation of Dupuytren Contracture.

BACKGROUND: Artificial intelligence (AI) has been utilized as a supplemental tool by patients and providers in the evaluation of common hand pathology. This study evaluated the ability of 3 large language model (LLM)-based AI systems (ChatGPT, Gemini, and Claude) to characterize Dupuytren's contracture involvement and recommend treatment based solely on patient images.

METHODS: A heterogeneous set of publicly available images was obtained from open-access databases and literature sources. Images were evaluated by LLMs in independent stateless sessions using a standardized prompt to identify contracture characteristics and recommend treatment. Artificial intelligence outputs were compared with consensus assessments from 3 board-certified hand surgeons.

RESULTS: Large language models demonstrated limited accuracy in characterizing Dupuytren's contracture involvement from patient images. Treatment recommendations were often within 1 ordinal tier of surgeon consensus; however, agreement with surgeons was no greater than chance. No consistent differences in performance were observed among the models.

CONCLUSIONS: Large language model-based AI systems demonstrated limited capability in the image-based assessment and treatment recommendation of Dupuytren's contracture. These findings highlight a gap between current LLM performance and surgeon consensus when evaluating Dupuytren's contracture from single-patient-submitted images. Further research is needed to determine whether the incorporation of standardized clinical information can improve the reliability of AI-assisted treatment recommendations.