Page 27 - AIH-1-2

Artificial Intelligence in Health LLMs-Healthcare: Application and challenges

A study conducted by Agbavor and Liang explored 4.1. Challenges associated with LLMs in
21
the use of GPT-3-generated text embeddings to predict neurodegenerative disorders
dementia, utilizing data from the ADReSSo Challenge Utilizing LLMs in diagnosing and managing
(Alzheimer’s Dementia Recognition through Spontaneous neurodegenerative disorders such as dementia and AD
Speech only challenge), which focuses on identifying presents several challenges. First, the complexity and
22
cognitive impairment through spontaneous speech. The variability of these conditions require highly accurate and
author proposed using the model to identify individuals deep understanding, which LLMs may not always provide
with dementia against healthy individuals as controls. due to limitations in their training data. The ethical and
Using the 237 speech recordings derived from the privacy concerns about handling sensitive patient data pose
ADReSSO Challenge, the authors used a 70/30 split and
obtained 71 data samples as the testing set and 166 as the significant hurdles. Furthermore, integrating these models
training set. In the training set, 87 individuals had AD, and into clinical workflows demands substantial validation to
79 were healthy controls. GPT-3 was innovatively used for ensure they complement, rather than complicate, health-
embedding the transcribed speech texts. Then, the model care professionals’ decision-making processes. Finally,
extracts the acoustic features such as temporal analysis there is a need for continuous updates and improvements
(periodicity of speech, pause rate, phonation rate, etc.) and in these models to keep pace with the latest medical
speech production (vocal quality, articulation, prosody, research and clinical practices.
etc.). These features serve as the input for the classification 5. Dentistry
model used in AD prediction. GPT-3 embeddings are then
compared with BERT and traditional acoustic features. The World Health Organization reports that oral diseases
The findings reveal that text embeddings outperform impact approximately 3.5 billion individuals globally, with
traditional acoustic methods and compare well with fine- dental caries, periodontal diseases, and tooth loss being
tuned models such as BERT. This suggests that GPT-3’s text the most prevalent. These conditions, largely preventable
embeddings offer a promising approach for early dementia and manageable with early diagnosis, have seen the
diagnosis. application of AI methodologies in recent years, including
29
Another study conducted by Mao et al. outlines the diagnosis of dental caries 27,28 and periodontitis.
23
developing and applying a deep learning framework Despite this, exploring LLMs in dentistry remains notably
utilizing the BERT model for predicting the progression of scarce, with limited studies demonstrating their practical
an array of diseases ranging from mild cognitive impairment application.
(MCI) to AD using unstructured electronic health records LLM-based deployment strategies within dentistry
(EHR). The study cataloged 3657 MCI-diagnosed patients proposed by Huang et al., mark an emerging area of
29
and their clinical notes from Northwestern Medicine research with significant potential for advancement.
Enterprise Data Warehouse (NMEDW) between 2000 To showcase the effectiveness and potential of applying
and 2020, using only their initial MCI diagnosis notes for LLMs in dentistry, this work introduced a framework for
analysis. These notes underwent de-identification, cleaning, an automated diagnostic system utilizing multi-modal
and segmentation before training an AD-specific BERT LLMs. This innovative system incorporated three distinct
model (AD-BERT). AD-BERT transformed patient note input modules, namely, visual, auditory, and textual data,
sections into vector forms, which were analyzed by a fully enabling comprehensive analysis. Visual inputs, such as
connected network to predict MCI-to-AD progression. dental X-rays and computed tomography (CT) scans, are
For validation, a similar methodology was applied to evaluated for anomalies using vision-language models
2,563 MCI patients from Weill Cornell Medicine (WCM). to facilitate precise diagnostics. Audio inputs serve dual
AD-BERT outperformed seven baseline models, showing purposes: detecting voice anomalies and understanding
superior accuracy in both patient groups, evidenced by its patient narratives, which are converted to text for further
area under the curve (AUC) and F1 scores. analysis by LLM. To illustrate the capabilities of the multi-
29
In the diagnosis of complex conditions like AD, modal LLM AI system in dental practice, Huang et al.
medical professionals use a variety of data such as images, proposed its application in diagnosing and planning
patient demographics, genetic profiles, medication history, treatment for dental caries. The process begins with
cognitive assessments, and speech data. Some of the inputting a tooth’s X-ray into the system, where vision-
recent studies have proposed multi-modal AD diagnosis language modeling is employed to detect any decay on the
or prediction methods leveraging the popular pre-trained tooth. Once identified, the system utilizes LLM to propose
LLM to add text data sources, in addition to images and a comprehensive treatment plan, articulated through
other data types. 24-26 seven detailed steps. These steps range from initial patient

Volume 1 Issue 2 (2024) 21 doi: 10.36922/aih.2558