(91) Is ChatGPT (4.0) Reliable in Interpreting Temporomandibular Joint MRIs?

Abstract:
Statement of the Problem: Current artificial intelligence (AI) is currently outperforming radiologists in certain scenarios.¹ Currently, most AI efforts are directed towards diagnosis and treatment planning.^2,3 It is important to test current AI capabilities to determine if it can become a useful tool in the field of oral and maxillofacial surgery. The purpose of this study was to determine if Chat GPT (4.0) is currently capable of analyzing magnetic resonance imaging (MRI) and providing an accurate diagnosis for TMJ disorders. The specific aims were to determine its accuracy and compare it to a human surgeon.
Materials and

Methods: The study design was a multi-center retrospective cohort study from private and institutional centers. The sample included 14 patients (N-f28 joints) which suffered from temporomandibular joint (TMJ) disorders. Magnetic Resonance Images (MRI) were obtained, and 28 condyles were analyzed by human surgeons and AI. The most representative images from open and closed-mouth sagittal views were chosen. Two questions were prompted into the software 1) Question 1 (Q1) for the presence/absence of a TMJ disorder (yes/no), 2) Question 2 (Q2) to provide a diagnosis based on Wilkes Classification. Our hypothesis was that Chat GPT (4.0) was able to identify the presence/absence of TMJ disorders and provide the same Wilk’s classification diagnosis determined by the human surgeon. Descriptive statistics were performed, and Kappa Coefficient test was used to compare agreements between the responses from the human surgeons and the software. A P value < .05 was considered statistically significant.

Results: Artificial intelligence over diagnosed the presence of TMD (89%) over human surgeons (82%). The most common diagnosis (human) was Wilkes Type 3 (42.9%), followed by Wilkes 4 (35.7%) (Figure 1). In contrast, AI’s diagnosis encountered most frequently was Wilkes 4 (42.9%), followed by Wilkes 5 (14.3%) (Figure 2). Kappa value showed no statistical associations between human and software responses from Q1 (P > .57) and Q2 (P > .393) (Table 1 and 2).
Outcomes Data: Human and AI diagnosis differed significantly; the software misdiagnosed and provided exaggerated results. Kappa coefficient test was used to determine if there was a statistical agreement between the raters, if the P value is not significant it means that the few present agreements were random.

Conclusion: ChatGPT (4.0) is a tool with potential regarding the interpretation of TMJ MRIs, however, at this point it’s still inaccurate and unreliable. A further investigation undergoing a formal process of machine learning and bigger samples might improve its accuracy.

Disclosure/Conflict of Interest: ChatGPT (4.0) was used in this investigation.

1. Najjar R. Redefining Radiology: A Review of Artificial Intelligence Integration in Medical Imaging. Diagnostics (Basel). 2023 Aug 25;13(17):2760. doi: 10.3390/diagnostics13172760. PMID: 37685300; PMCID: PMC10487271.
2.Krishnan DG. Artificial Intelligence in Oral and Maxillofacial Surgery Education. Oral Maxillofac Surg Clin North Am. 2022 Nov;34(4):585-591. doi: 10.1016/j.coms.2022.03.006. Epub 2022 Oct 9. PMID: 36224076.
3.Is Current Artificial Intelligence (AI) at the Same Level of an Oral and Maxillofacial Surgeon? Sifuentes-Cervantes, Jose S.Peña-Ruiz, O. Alejandro et al. Journal of Oral and Maxillofacial Surgery, Volume 82, Issue 9, S75 - S77
This is part of a 2 series article regarding artificial intelligence, the other one is about wisdom teeth AI analysis, perhaps an oral presentation comprising both of them could be an option.Table 1. Kappa coefficient test. Figure 1. Presence of TMD (yes/no) between human surgeon and AI.Figure 2. Wilk's classification diagnosis between human surgeon and AI.