New Study Finds That Generative AI Can Track Cognitive Decline using EMRs!

A new study out of Boston suggests that combining large language models (LLMs) with traditional AI and deep learning models enhances accuracy in identifying early signs of cognitive decline, offering new hope for early diagnosis and effective treatment of dementia. The study recently published in in eBioMedicine evaluated the effectiveness of LLMs in identifying the signs of cognitive decay in electronic health records (EHRs). They also compared the performance of large language models with conventional AI models trained with domain-specific data.

Using the International Classification of Diseases, tenth revision, clinical modification (ICD-10-CM) criteria for Mild Cognitive Impairment (MCI), the researchers analyzed proprietary and open-source LLMs at Boston’s Mass General Brigham. They studied medical notes from four years before a 2019 MCI diagnosis among individuals 50 years and over.

The study dataset looked at nearly 5000 clinical note sections of about 2000 individuals, among whom 53% were female with a mean age of 76 years. Cognitive function keywords filtered the notes to develop study models. The testing dataset without keyword filtering comprised 2000 sections of clinical notes from about 1200 individuals, among whom 53% were female with a mean age of 77 years.

The researchers tested a proprietary GPT-4 LLM tool and an open-source LLM – Llama 2. The team found GPT-4 more accurate and efficient than Llama 2. GPT-4 highlighted dementia therapy options like Aricept and donepezil. It also detected diagnoses like mild neurocognitive disorders, major neurocognitive disorders, and vascular dementia better than previous models. GPT-4 addressed the emotional and psychological consequences of cognitive problems, such as anxiety, often disregarded by other models.

Combining the two, along with a traditionally trained AI model, into an ensemble model dramatically improved performance. The ensemble study model attained 90% precision, 94% recall, and a 92% F1 score, outperforming all individual study models regarding all performance metrics with statistically significant results.

The researchers concluded that LLMs trained using general domains need additional development to improve clinical decision-making. Future studies should combine LLMs with more localized models, using medical information and domain expertise to improve model performance for specific tasks and experimenting with prompting and fine-tuning tactics.

You can read the full study entitled Enhancing early detection of cognitive decline in the elderly: a comparative study utilizing large language models in clinical notes by following this link.

How BigRio Helps Bring LLM and Advanced AI Solutions to Healthcare

It’s interesting that the Boston researchers found that the LLM tools in the study worked best for healthcare analysis when they are proprietary, as opposed to “open source” like Lama 2. Furthermore, they found that even the proprietary tool in the study worked even better when it was customized specifically for the task at hand.

That kind of customization is the cornerstone of BigRio’s mission – to create LLM solutions that specifically target your healthcare organization’s unique needs. BigRio has long been a facilitator and incubator in leveraging AI to improve healthcare delivery, originally in the field of diagnostics and research. We have recently been focusing our efforts on supporting startups and developing our own solutions that use LLMs and GAI to improve those areas of healthcare, as well as direct patient interactions, customer relationship management, EMR integration, and so much more.

We are proud to have recently launched our proprietary LLM-driven tool, Odyssey Accelerator. It is a first-of-its-kind solution that enables businesses of all sizes who want to use GAI to transform the way they search or query enterprise data, design workflows, derive enterprise analytics, report generation, dashboards, and more, to build a proof of concept before scaling it enterprise-wide.

Leading healthcare IT innovator Vivid Health partnered with BigRio to develop an LLM-driven end-to- end solution, that today can assess a patient’s status on any combination of more than 100 chronic conditions across sixteen specialties, generating personalized care plans at scale in under 30 seconds. That’s over 75 million possible care plan combinations in a fraction of the time it typically takes to pull together individualized plans.

You can read much more about how AI is redefining healthcare delivery and drug discovery in my book Quantum Care: A Deep Dive into AI for Health Delivery and Research. It’s a comprehensive look at how AI and machine learning are being used to improve healthcare delivery at every touchpoint.