(93) ChatGPT(4.0) Has the Capacity to Diagnose Orbital Floor Fractures: Fiction or Reality?

Abstract:
Statement of the Problem: ChatGPT is a Generative Artificial Intelligence (G-AI) derived from Large Language Models (LLMs). Its emergence has sparked a paradigm shift in many professions, including radiology, due to its promising ability to recognize image patterns. However, there is still a lack of substantial evidence supporting this capability. Oral and maxillofacial surgeons are currently exploring the potential role of AI within their field. One promising short-term application could be improving diagnostic efficiency, especially in time-critical situations. But should oral and maxillofacial surgeons embrace AI’s ability to diagnose facial fractures instantaneously? A potential way to address this question is by testing AI’s capabilities with real clinical cases.
Materials and

Methods: The study design was a retrospective cohort study. A sample of 30 cases of maxillofacial trauma resulting in orbital floor fractures was selected. Computed tomography (CT) scan images were extracted from Puerto Rico’s Hospital Level I Trauma Center database (Sagittal and Coronal views only). An oral and maxillofacial surgeon was set as the gold standard to compare responses. The aim of this project is to determine whether ChatGPT’s is accurate and reliable enough to be used in a clinical setting for the diagnosis of orbital floor fractures.
Methods and Data Analysis: Descriptive analysis was used to display the results. A Student’s t-test was applied to compare paired responses between the oral and maxillofacial surgeon and AI and to determine if differences were statistically significant (P < .05).

Results: The mean age was 68.4 years (σ = 20.58). The gender distribution was 66.6% male (20 cases) and 33.3% female (10 cases). AI identified orbital floor fractures in 63.5% of cases (19 cases) and no fractures in 36.6% (11 cases) (Figs. 1-2). A Student’s t-test exhibited a statistically significant value (P = .03), suggesting that AI’s responses differed significantly from those of the oral and maxillofacial surgeon, thus rejecting our hypothesis (Table 1).

Conclusion: Age and gender were considered to explain variations in bone morphology and density in the CT images, as not all patients presenting to the ED have the same bone anatomy. Although AI identified more cases as true fractures, the overall result suggests that these findings occurred by chance. Therefore, ChatGPT 4.0’s rapid diagnostic capability is neither reliable nor accurate. Orbital floor fractures still require diagnosis by an oral and maxillofacial surgeon, ensuring that surgeons can verify AI-generated responses before using the tool in clinical settings. Further studies are needed to challenge the clinical applications of this software, particularly given its increasing use among oral and maxillofacial surgeons.

References:

1. Mehandru, Nikhil et al. Machine Learning for Identification of Craniomaxillofacial Radiographic Lesions Journal of Oral and Maxillofacial Surgery, Volume 78, Issue 12, 2106 – 2107 DOI:https://doi.org/10.1016/j.joms.2020.07.010
2. Puladi, B., Gsaxner, C., Kleesiek, J., Hölzle, F., Röhrig, R., & Egger, J. (2023). The impact and opportunities of large language models like ChatGPT in oral and maxillofacial surgery: a narrative review. In International Journal of Oral and Maxillofacial Surgery. Churchill Livingstone. https://doi.org/10.1016/j.ijom.2023.09.005
Figure 1 Figure 2Table 1