AI Tools for Doctors: Fact vs. Fiction with Dr. Spencer Dorn

“When people think of AI in medicine, they think of replacing people. They think of this omniscient bot that will perfectly diagnose and treat and make everyone happy and be empathetic. And that's a gross caricature that I find kind of silly and not helpful.”

Dr. Spencer Dorn, a gastroenterologist, informatics physician, and professor at UNC, is passionate about the intersection of technology and healthcare. Spencer, who frequently writes about AI for Forbes, is helping to shape the conversation around real-world applications of AI in medicine, and how AI can complement (rather than replace) physicians.

In this episode of How I Doctor, Offcall co-founder Dr. Graham Walker talks with Spencer, a longtime friend, about how AI is being used in medicine now, and where it has the most potential going forward. Artificial intelligence and machine learning tools have been part of healthcare for a while, from EKG analysis to clinical documentation. But what does the next chapter look like? And what are the biggest misconceptions about AI’s impact on doctors, patients, and our healthcare system? They also cover AI literacy standards for doctors, whether perfection is an unrealistic expectation for AI, and how AI will affect medical training.

This episode is a crash course on AI in medicine — what’s already happening, where things are heading, and how physicians should understand AI’s expanding role. Here are three highlights from their conversation.

Information overload is one of the biggest problems that AI can solve

“What do we do as clinicians? We help people tell their story and we gather objective data and we try and synthesize that into some sort of capsule summary or impression. And then hopefully we're drawing on our experience in the medical literature to bring in and help make the best decisions with the patient for what they need. The challenge is that we're drowning in information.”

When people talk about use cases for AI in medicine, ambient scribes — which transcribe patient interactions and generate clinical notes — come up a lot. Being able to outsource note-taking is certainly a win for overextended physicians, but Spencer believes that even more value lies in summarization. At least a dozen workflows in medicine involve summarization, Spencer says, and AI could make them less burdensome. Potential opportunity areas include pre-charting, discharge summaries, referral letters, and literature reviews.

AI tools don’t need to be perfect to be valuable

“The expectation should be that these tools are as good as we are, and hopefully a bit better. But perfection is just an impossible bar to clear.”

Offloading summarization to AI agents is beneficial even if they don’t perform flawlessly, Spencer says. There’s an idea that AI needs to achieve perfection in order for it to be worthwhile. But human doctors don’t take notes perfectly, or have perfect records with clinical decision-making. So, why would perfection be the standard for tasks that would otherwise be done (more) imperfectly by humans? For an analogous debate, we can look at self-driving cars: They’re much safer and less accident-prone than human drivers, but they still make mistakes. Does that mean they shouldn’t be on the road?

Spencer suggests expectations of AI perfection are partly due to the fact that human doctors are held accountable for poor performance. Doctors are licensed and reprimanded for subpar outcomes. AI and digital tools aren’t; they don’t operate within any regulatory framework. Right now, we don’t have a standard for AI-generated results that are imperfect but still good enough.

Fears that AI bots could replace doctors are overblown

“We’re sentient beings. We understand what our patients, on some level, are going through because we also have bodies. That’s what fundamentally will always separate us from machines and why there will always be a role for human clinicians.”

The possibility that ever-smarter machines might eventually replace human clinicians is a frequently cited AI concern. But Spencer doesn’t see it happening, probably ever. For starters, he doesn’t think patients would be on board — they want to form human connections with their healthcare professionals.

What’s more, he thinks physicians both undersell the complexity of what they do and place too much importance in humans having unique abilities.

Because, it’s true that AI is far better at information recall. Doctors need to accept this and let it go, Spencer says: “The kids who did well on the recall test — big deal. Those days are over.”

But there’s a lot more to practicing medicine than recalling facts. What makes doctors truly special is how they think and understand the world.

Spencer cites a Harvard informaticist from the 1990s who said that any doctor who can be replaced by a computer should be replaced by a computer.

Connect further with Spencer on LinkedIn here.

To make sure you don’t miss an episode of How I Doctor, subscribe to the show wherever you listen to podcasts. You can also read the full transcript of the episode below.

Offcall exists to help restore balance in medicine and improve the wealth and wellbeing of physicians. Sign up for Offcall here to bring about more physician compensation transparency and join our physician movement.

—

Transcript

Spencer Dorn:

When people think of AI in medicine, they think of replacing people. They think of this omniscient bot that will perfectly diagnose and treat and make everyone happy and be empathetic, and we know that's a gross caricature that I find kind of silly and not helpful, right? This whole notion that we're just going to replace doctors and nurses because that's just... We do way more than what most people realize. And also most people love their doctors and nurses. I don't think people want to move to a world where they're only interacting with machines even if it were possible.

Graham Walker:

Welcome to How I Doctor where we're bringing joy back to medicine. Dr. SD is a gastroenterologist and vice chair and professor of medicine at UNC with a deep interest in the intersection of technology and healthcare. Spencer's also a good friend and I have to say one of my favorite people to geek out about the future of medicine and AI and technology. I like to joke that Spencer and I are twins, but he actually already has a twin so I would be a long-lost triplet. We've had so many conversations about the potential pitfalls and potential of AI, so I'm really thrilled to get to talk with him today. Dr. SD, thanks for being here.

SD:

Thanks so much, Graham. It's so fun to be here with you.

GW:

I think there's probably not a day that goes by that people don't want to know about what the experts or physician leaders in this space think about AI in healthcare. So maybe we'll start with the current state of things, like how people, doctors, nurses, other people are using AI today. Where do you see people using AI successfully in the clinical space in medicine?

AI has been widely used in healthcare for years

SD:

We've been using AI for quite a long time, arguably for decades, right? When I was a medical student and I'd look at an EKG, I'd look at that little printout at the top of the EKG. That's AI, right? That and-

GW:

It's a machine learning bot.

SD:

There's a machine making... There's a cognitive decision. And we've used electronic health records for decades. I think the leading use case right now that people are most excited about is AI for clinical documentation, specifically ambient intelligence. I think people are very excited about that because it's a clear pain point. Physicians spend way too much time writing their notes, both during encounters with patients, after encounters. And when they go home at night, after dinner, the proverbial pajama time. So I think that is probably the area we're seeing there's the fastest uptake. Also, because the technology just really fits that. And even before the fancy language models that came out more recently, it's a lot of the technology that birthed this opportunity was already readily available, and I think that's why we're seeing really fast uptake of tools, both at large systems like yours and mine and also in smaller independent practices.

GW:

Yeah, I would agree, especially as medicine has gotten more and more challenging. A lot of our notes have become more and more templated as there is more and additional documentation for regulatory or quality metrics or Medicare diagnosis refresh, all that stuff. It also has made the note longer, less valuable maybe. And what is generative AI good at? Well, it's really good at recording information, summarizing it and putting it in a particular format.

There are more exciting and impactful use-cases for AI than note-taking

SD:

I think there is a potential downside to these notes. It's actually not that hard to write notes, at least it's not that hard to write crappy notes. We all have macros and we could pull in phrases. So that's one thing. The second thing is the conversation with the patient actually informs a relatively small portion of the notes, especially in inpatient medicine, and I'm guessing a lot in emergency medicine as well. Outpatient ambulatory care probably informs, I would say, maybe half the note or more. But yeah, I think the technology that exists today is not perfect. I think it can help offload some of the burdens compared to the current state, but I still think we need to do more.

GW:

I have heard that with some specialists that will do what they're calling pre-charting, maybe especially if they're seeing maybe a new patient who is particularly complex. Specialists will put in a lot of time maybe the night before or the morning of before they actually see the patient to review all the CAT scans they've had, all the imaging tests they've had, all the labs they've had, all the other workup they've had in preparation for the visit. Are you able to use those technologies, Ambient Scribe stuff for pre-charting as well, using it more as like dictation?

SD:

In a very limited way, yes, but it still requires you to review the records and then talk to the machine and say, "Here are some pertinent records I want you to include in my note." The day before I'm in clinic, I spend an inordinate amount of time reviewing records. Many of these records are captured in PDFs that aren't labeled well, that are upside down, that there's a lot of time and effort that specialists, especially at academic centers but everywhere, are spending doing this pre-charting for new patients that the current ambient technologies really don't address.

And I'd also say on the inpatient side, a little known secret is that most inpatient physicians write almost their entire note before they see the patient. They're writing in their workroom, they're pulling together, "Well, the infectious disease consultant said this and the MRI showed this," and the ins and outs for the last 24 hours. So it's a lot of data aggregation, pulling information together. And then when they go see the patient, sometimes the patients may not talk or they may not be oriented, or even if they do, that's probably only informing a very tiny percentage of their overall notes.

So I think ambient is a useful technology, but I think also it's been, perhaps it's oversold as a solution to all the documentation challenges. I'm really excited about summarization technologies. And I think ultimately these different applications will converge, but right now they're offered by different products from different companies.

GW:

I look at more as, oh, we're changing one problem for another that we've decided this is a more acceptable problem to have than the prior problem. That's why we're implementing this particular solution, but that the solution will create new challenges, new problems, and that's okay. I think, again, lots of pearls and pitfalls and that could be tremendously helpful, but it could also, if it misses things, that could be bad too.

Information overload is a huge problem in medicine, and AI-driven summarization can help

SD:

I think it's a fascinating field. I really like what Isaac Kohane... He's a esteemed informaticist-pediatrician at Boston Children's Hospital. He has this axiom that medicine is fundamentally an information processing discipline. And if you think about it, I think there's a lot of truth to that. What do we do as clinicians? We help people tell their story and we gather objective data and we try and synthesize that into some sort of capsule summary or impression or whatever you'd like to call it. And then hopefully we're drawing on our experience in the medical literature to bring in and help make the best decisions with the patient for what they need.

The challenge is that we're drowning in information. There's healthcare data reportedly accounts for around one-third of the world's data, and it moves faster than we can generate meaning from it. And a stat that I like, it's this came from the University of Pennsylvania. I'm sure you've seen this. The average patient medical record is more than half the length of Hamlet, Shakespeare's longest play. So you're an emergency physician, you don't have a half hour to sit around and peruse the medical record. You need fast answers related to the patient's problems now, right? So that's the challenge, and I think that's the opportunity of summarization.

And if you look at that, there are literally at least a dozen workflows I can name that involve summarization. So for example, we talked a little before about pre-charting. Another one would be your primary care doctor. You're sending your patients to all these specialists. How do you make sense of what they're all saying? The patient showed up in the emergency department last week. You could read the emergency department discharge summary, but wouldn't it be nice if you can have that condensed for you and presented to you in a capsule?

I think summarization is great for writing referral letters. I think it's even better for processing referrals. There's a lot on abstracting data for quality registries and population health approaches, risks coding, HEC capture. There are just so many opportunities for us to apply summarization to the medical record. There's also a lot of opportunities to apply summarization to the medical literature so you can find the specific study that's relevant to the patient in front of you more rapidly or more dynamically.

GW:

You're absolutely right, Spencer. It is not possible to read through most patients' medical record, especially with our older and sicker patients. The best case scenario is maybe you read one or two of their recent discharge summaries, but we have many really complex patients who are often in and out of the hospital frequently, and you're trying to as both thoroughly and rapidly find the pertinent details of that hospitalization or information that might clue you into what's going on in this ER visit or hospitalization as well. Maybe the key detail is in a nursing note two hospitalizations ago, or it's just in one line of the intern's note from three days ago.

So I don't know that we can expect perfection from these summarization tools, but do you have a sense of how the industry or how informaticists are thinking about what is accuracy look like?

Humans aren’t perfect, and neither are machines

SD:

We're not perfect, and I think we have to continually remind ourselves and be humble that if I'm an anesthesiologist preparing for a case, I may not know that the patient had an echocardiogram eight years ago that showed mild pulmonary hypertension. That happens all the time, right? So we are imperfect. We don't write perfect notes. We write pretty poor quality notes, I'd argue. We don't perfectly abstract the medical record. We don't always make great clinical decisions, and I think that's just something we need to put down as a base case in that it's a bit unrealistic to expect the machines to be perfect because we're not.

With that said, we obviously still need and deserve and demand higher quality, and I think part of the tension is that we're held accountable. And I know you've written about this. I have. We've gone through training. We are regulated. As physicians, we are regulated. We report. We have to get licensed and we report to regulatory bodies. If we continue to have poor performance, we'll be reprimanded. We may lose our hospital privileges. We may lose our licenses. And we don't have that yet. There's no regulatory framework that enables that for artificial intelligence or other digital tools. So that's where I would start is the expectation should be that these tools are as good as we are and hopefully a bit better or maybe could do it less painfully, but perfection is just an impossible bar to clear.

GW:

There's two schools of thought about expectations for AI, the Tesla or the Waymo here in San Francisco, the Waymo self-driving cars. And some people say, "Well, these cars, this software needs to be just better than the average driver." That's improvement. That's the standard we should set. Other people say, "No, this needs to be a hundred times better than the average driver because we are," it's a little bit of a philosophical concept, "because we're giving up control. We are allowing a software to control us and drive us around." Do you fit into either one of those camps?

SD:

Like most things, I'm probably in between. I usually gravitate towards the middle ground. I think it's been proven that Waymo and self-driving cars are far safer and less accident-prone than human drivers. I think there needs to be a willingness to accept that Waymo will crash once in a while, and you know what? It'll crash much less frequently than if there was a human behind the wheel.

But at the same time, I think self-driving is a really interesting example for healthcare because when people think of AI in medicine, they think of replacing people. They think of this omniscient bot that will perfectly diagnose and treat and make everyone happy and be empathetic, and we know that's a gross caricature that I find kind of silly and not helpful, this whole notion that we're just going to replace doctors and nurses because that's just like we do way more than what most people realize. And also most people love their doctors and nurses. I don't think people want to move to a world where they're only interacting with machines even if it were possible.

How AI can be used to complement physicians, not replace them

GW:

Thinking about the future state, how do you think we should be approaching AI so it supports doctors, nurses, social workers, pharmacists rather than replacing us?

SD:

I often go back to this another quote by an earlier Harvard informaticist, Warren Slack, who was I think an internist, general internist, in the 1990s. He had this quote that, "Any doctor that can be replaced by a computer should be replaced by a computer." So that's where I align with this. If a computer can do everything I do, then fine, I'll go out and find a new job.

I think we both underestimate the value of what we do and we overestimate. We sometimes think, "Well, only humans could do this, right?" I memorized the Krebs cycle and mitosis and all these things.I memorized again and again as an undergrad and med student. And you know what? Information recall is the computers can do it far better than we can. We should just accept that, right? We're fine. But we are also so special because the complexity of what we do and the way that we think and we live in the world and we understand the world, that's very different than what a machine. At least machines today and for the foreseeable future, we need to be a little more humble today and be willing to give up some ground and part of our identity, is the kids who did well on the recall tests, big deal. Those days are over.

But at the same time, let's not undersell what we do. Let's not undersell the value of what we bring as a human being, as a human clinician. That can't be replicated. There's so much surface area where we at least now far exceed what any machine can do. And I believe for the foreseeable future, really forever, we're sentient beings. We understand what our patients on some level are going through because we also have bodies. That's just to me what fundamentally will always separate us from machines and why there will always be a role for human clinicians, caregivers, healers, et cetera.

GW:

Sam Altman said, the CEO of OpenAI, said, "These tools are really good at tasks but not jobs." So we do jobs which includes a variety of different tasks. And the other thing that I think about is the making the judgment call. I'm dynamically making decisions about when I need to adjust my course or maintain my course and say, "Nope, I really do think this is severe asthma and I need to just keep throwing more continuous albuterol at it." Those are judgment calls. And I don't know, at least for now until this episode airs, if AI will start being able to do that.

Spencer, I'm going to ask you to put your professor hat on. Thinking about maybe starting with attendings, what do you think other attending colleagues are going to need to know as AI starts to get more and more baked into our EHRs and our workflows? And then maybe after that, we'll touch on the training needs for training the next generation of physicians as well?

Raising physicians’ AI literacy levels

SD:

In general, I think there clearly needs to be some literacy around AI. We have to raise the literacy level, but the question is where is that bar? It's clearly physicians don't really need to know deep computer science. They don't need to know about graphs and vectors and the inner workings of how language models or symbolic AI or any deep learning, anything, any of that works. But they probably do need to understand the difference between rules-based and more probabilistic models, how these tools are processing, broadly processing data and generating responses.

And there's this great JAMA study out of Stanford recently. I don't know if you were on it or you're friends with the investigators where they gave doctors a bunch of vignettes and they also gave them GPT, ChatGPT. And guess what? With ChatGPT, they didn't do any better on diagnosing these vignettes. And Jonathan Chen, the lead investigator afterwards in a New York Times interview said, "The doctors are using it like Google. They were using it like a search engine." So you probably need to know these tools are not search engines. This is a different class. This is a completely different product class than traditional consumer internet services.

GW:

I've been thinking about it as if kind of similar to some of the evidence-based medicine terms we have. It's not pharmacology. I don't think people need to know the mechanism of action of a large language model. They don't need to know the details of how this thing was formed and vector databases and all that stuff. But I think they do need to know how accurate these things are, where they can make mistakes in the same way that we're taught, oh, CT scans will miss 2 or 3% of appendicitis, or the gold standard test for this disease is X, or a D-dimer is a very sensitive test but it has terrible specificity, that level of detail about how these tests work so that we can still have that gestalt of like, should I trust this thing or not?

And that's what we think about CAT scans. When I get a negative CT for appendicitis, if the patient's still tender there, in the back of my head, there's still that like, "Oh, is this one of these rare times where the CAT scan's actually wrong?"

SD:

That's a great analogy. I guess another analogy would be the old physician's desk reference. You give a patient this medication, 8% of people have nausea with this medicine. I think the challenge is we don't have the level of precision in the data.

And one of the challenges I think for at least for a lot of the predictive AI is that a tool that's developed and implemented in the Kaiser San Francisco Emergency Department may have completely different performance in the UNC, right? It's not like Prilosec in San Francisco is the same as Prilosec in Chapel Hill. So I think it's a little harder in that are you working for an organization that's doing all this due diligence to test the performance of these models in your environment?

I love the concept. I think it's in some ways challenging to pull off, but I think that's a great way of looking at it. I think it's a great framing. It's also about learning how to adapt the way you work. So are these tools, let's understand the basics of these tools and maybe the indications for use, the clinical uses, and then how do you adapt what you do? So I think there are several layers, I think, of education that we need to work on and create greater awareness around.

Figuring out where AI belongs in medical training

GW:

I'm going to give you the hardest question I have. What do we do about medical students and residents as AI takes over? And as an ER doctor, I think a lot about the gestalt and my gut feeling and my spidey sense that impacts so much of how we practice medicine. And I don't know how that works when there's an AI model maybe telling you the answer at the beginning or giving you the differential before you've even finished writing your note or as you're writing your note.

SD:

Yeah, I think what you're speaking to is there's in some ways a benefit of learning how to do these things longhand or the manual process before you start using the tools. Because-

GW:

Yeah, show your work, getting and stuff.

SD:

Show your work. You've done it manually for all these years, and therefore when the AI gives you something you can contextualize and you have that spidey sense and the experience to make sense and recognize that maybe it's off base. Whereas if you were training today and using these tools from the start, you may not have that.

Probably this concern comes up repeatedly in history. I'm guessing that when calculators came into school, people were saying, "Oh, you don't know how to do math by hand." And more specifically in medicine, I think the best parallel I could think of is laparoscopic surgery. Surgical residents in the 1990s learned how to do open cholecystectomies, open procedures it's generally. And then things started shifting in laparoscopy. We realized laparoscopy is a much better way of doing this. It's safer, it's more effective, much better recovery. And I spoke to one of my surgical friends who leads a residency program, and he said, "Now, the typical surgical resident probably only does maybe two or three open cholecystectomies their entire training. They're doing almost only laparoscopic." So then the question is, well, if you have a complication during a laparoscopic procedure, you need to have that open.

GW:

You need to switch to open, yeah.

SD:

Right, and what if you really haven't done open procedures? They're certainly historical parallels both outside of medicine and within medicine to this concern that you're raising, which maybe we call it de-skilling. Maybe we call it never being skilled at all in the first place. I don't know. I don't know the answer. Medical students today, I could tell you the ones that I work with, they're digital natives. They're very comfortable and adept with using digital technologies, and many of them are quickly gravitating towards AI tools because it just seems logical. It seems like the next step.

GW:

There are a lot of physicians now building, a lot of physicians creating not just AI solutions but solutions that solve physician problems. What do you think the key factors are that make a healthcare company and probably a tech company more successful when... We have also, these companies all have competition from Epic and Cerner and even Microsoft and other tech companies as well.

SD:

I think fundamentally, they need to be humble. I think there's been a tradition of tech entrepreneurs who try to overfit their solution to certain situations or problems that don't really exist. So I think there needs to be empathy and humility and a willingness to actually go and speak to people who are actually experiencing healthcare patients, doctors, nurses, technicians, whoever they are, and figure out, well, what's making your day hard? And how can we fix that? What's the incentive that people will want to use your product? And maybe it's just that you're making their lives much easier. That's a very noble, powerful incentive.

GW:

What are you using as your resources to learn? Who are you, besides me, of course, who are you following? Are you reading journals, newsletters? What do you recommend to... How do you get inspired for your own ideas?

SD:

Certainly you, Graham. Graham, you're putting out... Obviously, I'm flattering you because you invited me in your podcast, but that's the truth. I love that you actually have opinions. It's important to find people who actually have opinions about things, not just... And are willing to put their money down, because I think that's where you start to learn. I actually like reading a lot of non-healthcare technology things, following people like Ben Thompson who writes Stratechery, Benedict Evans. I like reading general tech because healthcare is different but it's not that different. Certainly within healthcare, I read a lot as well. Stat has a great newsletter and there's a lot of really good healthcare tech content out there. Health Tech Nerds, I love. Kevin's newsletter and community is great.

GW:

Similar to you, I like reading tech, non-healthcare stuff and healthcare non-tech stuff because then I can take some of the ideas from the tech people and add a healthcare flair to it, or I can take stuff from the healthcare people and add my idea about tech to it. And I find that to be a really helpful way of having an opinion and also understanding stuff from people outside of my exact sphere too. Spencer, it has been great talking with you. Where's the best place to find you, your columns, your LinkedIn? Where do you send people to read all your amazing stuff?

SD:

Well, I always like to give a plug for UNC Department of Medicine. I love the work we're doing. We have a phenomenal group, phenomenal people, phenomenal clinicians and researchers. So I'm very proud to be part of this organization. And you could check out our website, UNC Department of Medicine. I have a column now that I'm writing in Forbes about one or two times a month, so you can find me there. LinkedIn, I don't post as often as you do. I post fairly regularly. I try and get-

GW:

You do, yeah.

SD:

... a few posts out there a week at the least. So those would be the main places. But yeah, you could reach out to me there if there are specific topics you'd like to discuss, or I always like having thoughtful discussions so I welcome the opportunity

GW:

Well, Dr. SD, so excited to have you on the podcast. Thank you so much for being here.

SD:

Thanks so much, Graham.

GW:

Thanks for joining me today. If there's someone you think I should interview for How I Doctor, drop us a line at podcast@offcall.com. Make an account on Offcall to confidentially share your details about your work and sign up for our newsletter where you can hear more about the latest trends we're seeing in physician pay. You can find How I Doctor on Apple, Spotify, or wherever you listen to podcasts. We'll have new episodes weekly. This has been and continues to be Dr. GW. Stay well, stay inspired, and practice with purpose.