Augment, Do Not Replace: Why AI in Medicine Only Works With the Physician in the Driver's Seat
For more than a decade, AI companies have promised to change medicine. Many of them tried to replace what doctors do. The results are in, and most of those companies have failed. The ones that succeeded took a different approach. They built tools that made doctors better, not tools that stood in for doctors.
This piece looks at why physician-replacement AI keeps failing, and why the tools that keep the physician in the driver's seat are the ones that earn trust and stay in practice. It matters because the next generation of surgical AI — including Surgeon Decision Intelligence — is being built on that lesson.
The Companies That Tried to Replace Doctors
The most famous example is IBM Watson for Oncology. IBM promised that Watson would recommend cancer treatments better than oncologists. Internal documents leaked in 2018 showed that Watson often produced unsafe and incorrect recommendations, in part because it had been trained on hypothetical cases rather than real patient data [1]. In 2022, IBM sold Watson Health assets to Francisco Partners for roughly $1.06 billion — a small fraction of what IBM had invested [2]. The unit is now a standalone company under a different name.
A second example is the Epic Sepsis Model, which is built into one of the most widely used electronic health record systems in the United States. The tool was designed to alert clinicians when a patient looked like they might be developing sepsis. In 2021, researchers at the University of Michigan published an external validation in JAMA Internal Medicine showing that the model missed about two-thirds of sepsis cases (sensitivity 33 percent, area under the curve 0.63) [3]. Alerts came too late or too often to change bedside care.
A third is Babylon Health, which built a chatbot designed to replace general-practitioner triage. Regulators raised safety concerns. The tool over-triaged routine complaints and missed serious ones. Once valued at nearly $2 billion, the company filed for U.S. Chapter 7 bankruptcy on August 9, 2023 and liquidated [4].
Why Physician-Replacement AI Keeps Failing
The failures share a small number of causes. All of them point in the same direction.
The training data does not match reality. Watson learned from hypothetical cases. Sepsis models trained at one hospital have performed poorly when deployed at another. Real patients are messier than the data these models see during training, and the gap shows up at the bedside.
The physician cannot check the model's work. When an AI outputs a recommendation without explaining why, the physician has no way to verify it, no way to push back on it, and no way to defend it later. Black-box outputs are hard to trust in clinical decisions with real consequences.
Liability does not transfer. When an AI-influenced decision goes wrong, the doctor is still responsible in court. Any tool that pretends otherwise creates risk for the physician using it. Physicians notice this quickly, and they stop using tools that expose them.
The doctor sees things the model cannot. Bedside impression, patient preferences, family conversations, prior history that never made it into the chart — none of this reaches the model. A tool that ignores what the physician alone knows is missing information that matters.
Why the Physician Belongs in the Driver's Seat
The physician holds the parts of a decision that a model cannot. They read the room during the consultation. They know the patient's goals and fears. They carry the legal and moral responsibility for the outcome. They have to explain the decision to the patient, to the family, and, if it goes wrong, to a jury.
AI in medicine earns trust when it makes the doctor's judgment better, not when it tries to substitute for it. The closest historical example is pulse oximetry in anesthesia. When pulse oximeters were introduced in the 1980s, they did not replace the anesthesiologist. They made the anesthesiologist more effective. Every call still belonged to the physician. But the calls got better because the physician had better information. The standard of care rose because the human in the driver's seat had a better dashboard.
That is the model that works for AI in medicine. The physician stays in the driver's seat. The tool sits alongside, not above.
How SDI Does It Differently
Surgeon Decision Intelligence is designed around this principle from the start.
SDI does not tell surgeons what to do. It surfaces biological and structural signals the eye cannot easily see in an MRI — muscle quality, disc health, vascular calcification, endplate changes, bone quality — and presents them to the surgeon in a form they can act on. The surgeon reads the information. The surgeon makes the call.
Every output is explainable. The surgeon can see which signals drove a specific insight and weigh them against the patient in front of them. This is not a black box. It is a structured view of the biology the surgeon is already trying to reason about.
SDI is implant-agnostic. It does not push a specific device or a specific technique. It gives the surgeon information about the patient. What to do with that information belongs to the surgeon.
The training data reflects real practice. The models are built on tens of thousands of real surgical cases with real longitudinal outcomes, not hypothetical scenarios. When the model tells a surgeon that a particular tissue profile is associated with a specific outcome trajectory, that association comes from patients who actually had the surgery and were actually followed for years.
The Bottom Line
AI in medicine is not a contest between the machine and the doctor. The machine loses that contest every time. The winning model is the doctor and the machine together, with the doctor in the driver's seat and the machine giving them a better view of the road.
That is what SDI is built for. It informs judgment. It does not replace it.
References
[1] Ross C, Swetlitz I. IBM's Watson supercomputer recommended 'unsafe and incorrect' cancer treatments, internal documents show. STAT News. July 25, 2018.
[2] IBM to sell Watson Health assets to Francisco Partners. IBM press release, January 21, 2022. The transaction closed in June 2022; the assets are now operated as Merative.
[3] Wong A, Otles E, Donnelly JP, et al. External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients. JAMA Internal Medicine. 2021;181(8):1065–1070. doi:10.1001/jamainternmed.2021.2626. PMID 34152373.
[4] Jennings K. Digital Health Company Babylon Files For Bankruptcy In U.S., Will Liquidate. Forbes. August 15, 2023. Chapter 7 filing dated August 9, 2023; see also Healthcare Dive coverage of the wind-down and eMed asset sale.