AI and physicians as “thought partners” with large language models

Our work in medicine is driven by narratives.

Whether it is figuring out a diagnosis, formulating a treatment plan, or simply learning about a patient’s life, we construct narratives to understand and exchange ideas in order to provide care. At the core of this interaction is human language, which enables the rich and high dimensional information exchange that we have evolved to be naturally good at.

Amidst all the buzz about large language models (LLMs) and system such as ChatGPT, what stands out to me the most is not the their information retrieval capabilities (which is impressive albeit with limitations) or that it even represents a new form of “intelligence.” Yann LeCun, Chief AI Scientist at Meta and an early pioneer of convolutional neural nets, articulates these limitations well:

Rather, it is that LLMs now enable humans to exchange complex ideas with AI systems in a way that is more like how we talk to each other. This is a big change from other machine learning (ML) models that only give quantitative predictions or discrete classifications.

In other words, LLMs provide a new “high fidelity API” between humans and AI.

“Cognitive integration” and why it is difficult with current AI systems

In a previous Byte to Bedside post, I wrote about the concept of “thought partnership” and “cognitive integration” of AI into physician workflows.

“Traditional workflow integration strategies focus on deploying technologies based on how they can streamline physician tasks. However, this task oriented approach misses a critical source of value for AI solutions, which is the integration of AI into how physicians think (and vice versa). This “cognitive integration” will be increasingly relevant as AI systems become more intelligent. The use case described above is a good example of how AI and physicians can team up as “thought partners” to enhance each other’s performance. “

True “thought partnership” requires a bidirectional, collaborative relationship where each party brings a unique perspective to tackle a complex problem. The output is greater than the sum of its parts because of the dynamic “back and forth” that leads to new and sometimes unexpected insights. This is what makes being a physician challenging and fun – we are surrounded by thought partners (our fellow clinicians, support staff, and patients and their families), which allows us to continuously learn and tackle the complex problems we encounter in our practice.

It is difficult to achieve this type of thought partnership with ML models that only produce predictions and classifications. The predictions may be highly accurate and come from complex, high dimensional models, but the output is still one dimensional: a single number. Regardless of how accurate that number is, it is far from able to enable the complex, dynamic high dimensional information exchange that human beings expect and need in order to collaboratively solve difficult problems.

This limitation, in my view, is a key reason why many clinician-facing AI systems in healthcare have not gained adoption – the unidirectional deployment of a number (regardless of how accurate it is) does not fit into the complexity of patient care. It also does not seem very “intelligent.”

Language as a “high bandwidth” interface between humans and AI systems

Soon after its launch, ChatGPT spread like wildfire among physician circles. This was something special – after long years of AI innovators struggling to find use cases and gain traction in healthcare, physicians instead were excitedly sharing self discovered use cases in their practice with ChatGPT. This post by a urologist, David Canes, is one of my favorites – a great mix of real world clinical and operational uses inspired by the needs of a practicing physician.

What is different this time? ChatGPT is indeed impressively accurate, but no more accurate than a high quality web search. It does not generate new information or predictions (even though it seems like it does sometimes). The difference is that we can now interact with an AI system with human language. It feels more accessible and attractive because our brains are wired that way — we evolved to communicate, learn, and innovate through language. ChatGPT unlocked an exponentially higher bandwidth link between humans and AI systems.

An example of “thought partnership” with Chat GPT

Consider the following example of how I, as a physician, may use a LLM application such as ChatGPT. I also look forward to trying this out with Google’s Med-PaLM!

I start by consulting ChatGPT about a hypothetical patient I am caring for with heart failure.

The output is fairly accurate, although reads like a laundry list of generic recommendations. However, I focus on step #5 given the patient’s complaint of chest pain, which reminds me that I may be missing a myocardial infarction (heart attack).

The exchange continues:

The conversation takes a turn. I know how to treat an MI so do not necessarily need ChatGPT to tell me specifically which medications to administer. However, its output reminded me that beta blockers may need to be used with caution in a patient with both heart failure and myocardial infarction. I am unsure about the evidence, so ask it to synthesize the latest research. Here, it would have been great if I could directly navigate to those papers (or see snippets of summaries) – a feature I am sure is in the works by some company out there.

Based on my assessment, I do decide to administer beta blockers to the patient, but I remember that his son had expressed anxiety and hesitancy to me when I had mentioned beta blockers earlier.

These explanations are pretty good and incorporate the use of patient friendly language that many physicians are not good at applying.

ChatGPT then helps me and the patient’s son remember that the patient was previously on a medication called labetalol. It also reminded me that there is a beta blocker named levobunolol (as an internist, I often forget the names of eye drops…).

The purpose of this example is to illustrate three points:

1) a patient encounter can consistent of a complex, non linear set of tasks and decisions,

2) back and forth dialogue between a physician and an AI system can help navigate through these tasks and decisions, and

3) the richness of information conveyed through language that far exceeds what would otherwise be possible if the AI system was only capable of putting out a discrete classification or quantitative prediction.

A call to designers and builders: create AI enabled products that can facilitate thought partnership with language.

We will like using it because this is how our brains are wired. The accuracy of the output from the AI system is of course important and sources, bias, errors will need to be transparently conveyed while preserving usability. However, the capability to actively exchange information via human language is, in my view, what is transformative.

What features, designs, and UIs will enable true “thought partnership” between physicians and AI systems? I look forward to finding out.