Forty million people consult ChatGPT every single day for health information. That is more daily patients than the entire U.S. hospital system treats in a month. And the evidence is now overwhelming that these AI tools are getting it dangerously, sometimes lethally, wrong.
On March 11, 2026, NPR and OPB published a damning investigation into AI-powered medical advice that should have stopped the tech industry in its tracks. Researchers at Mount Sinai, Duke University, and multiple other institutions have independently arrived at the same terrifying conclusion: AI chatbots are confidently dispensing medical guidance that is incomplete, contextually blind, and in at least one documented case, directly responsible for a man developing full-blown psychosis.
The details are not hypothetical. They are not edge cases. They are happening right now, at scale, to real people who trust a chatbot the way they would trust a doctor. And the companies building these tools have quietly stopped telling you not to.
The Man Who Got Psychosis From a Chatbot
A 60-year-old man walked into an emergency room expressing paranoid delusions that his neighbor was poisoning him. He was experiencing hallucinations and severe psychological disturbance. Doctors were baffled. For three weeks, they treated him in a hospital while trying to determine what had gone wrong.
The answer, when they found it, was chilling. Three months earlier, the man had asked ChatGPT whether he could replace chloride in his diet. The chatbot offered sodium bromide as an alternative. It did not flag any health risks. It did not ask why the man wanted to make the substitution. It did not suggest he consult a physician first. It just answered, confidently, as it always does.
Case Report: Bromism via AI Advice
The case was published in the Annals of Internal Medicine: Clinical Cases. The man had been using sodium bromide, purchased online, as a table salt replacement for three months. Bromide poisoning, once responsible for up to 8% of all psychiatric admissions when bromide compounds were common in over-the-counter medications, had been virtually eradicated from modern medicine. ChatGPT brought it back.
When the physicians tested ChatGPT 3.5 themselves, the response they received included bromide as a substitute. While the reply stated that "context matters," it provided no specific health warning and did not inquire about the user's medical reasoning.
Over the course of his three-week hospital admission, the man's chloride levels and anion gap gradually normalized, and his psychotic symptoms improved. He survived. But he spent nearly a month in a hospital because a chatbot treated a medical question like a trivia exercise.
Reddit's AI Told People to Take Kratom and Try Heroin
The ChatGPT bromide case is horrifying enough on its own. But it is not an isolated incident. In October 2025, Reddit deployed an AI feature called "Reddit Answers" that automatically responded to posts across the platform, including in medical and mental health communities. The results were catastrophic.
A moderator of the r/FamilyMedicine subreddit discovered that Reddit Answers was automatically responding to health-related posts with dangerous recommendations. In one case, the AI tool suggested that a user dealing with chronic pain should stop their prescribed medication and instead take high doses of kratom, a substance that is illegal in several U.S. states and not approved for medical use by the FDA.
"Do Not Trust Reddit Answers. It posted dangerous health advice that we as moderators had no ability to remove, edit, or control. We couldn't even disable it for our community."
It got worse. In a separate incident, the AI feature linked to a post that suggested heroin as a viable option for managing back pain. Moderators across medical and mental health subreddits demanded that Reddit either disable the AI feature for their communities or give them the ability to opt out. They were given neither option initially. Reddit eventually acknowledged the problem and updated the tool to suppress "Related Answers" for "sensitive topics" including health, but only after the damage had gone viral.
The core problem was that Reddit's AI was synthesizing advice from user posts, many of which were written by anonymous strangers with no medical expertise, and presenting that advice with the authority and formatting of a vetted, reliable answer.
The Nature Medicine Study: Wrong Two-Thirds of the Time
If you still believe that AI health advice is "mostly fine with some edge cases," a study published in Nature Medicine in early 2026 should destroy that illusion entirely.
Researchers gave participants medical scenarios and asked them to consult AI chatbots to identify the condition and determine next steps. After conversing with the bots, participants correctly identified the hypothetical condition only about a third of the time. Only 43% made the correct decision about whether to go to the emergency room, visit urgent care, or stay home.
The Timing Problem
Dr. Girish Nadkarni from Mount Sinai noted that ChatGPT performed reasonably well on "textbook" medical scenarios. The catastrophic failures came in complicated situations where timing mattered. The chatbot both over-estimated and under-estimated how long a patient could safely wait before seeking care. In emergency medicine, timing is everything. A chatbot that tells someone having a stroke to "monitor symptoms for 48 hours" is a chatbot that kills people.
Think about what those numbers mean at scale. Forty million people a day. Two-thirds of them walking away with the wrong answer. That is roughly 26 million people every single day receiving incorrect medical guidance from a tool that presents itself with absolute confidence.
AI Companies Quietly Dropped the Safety Warnings
Here is where the story turns from alarming to enraging. In July 2025, MIT Technology Review published an investigation revealing that AI companies had systematically gutted the medical disclaimers that once accompanied health-related responses.
In 2022, over 26% of AI responses to medical questions included a warning that the user should consult a doctor or that the chatbot was not a medical professional. By 2025, that number had cratered to under 1%. Just over 1% of responses analyzing medical images included a disclaimer, down from nearly 20%.
The MIT Technology Review investigation tested 15 models and found that disclaimers were least likely to appear in the exact scenarios where they matter most: emergency medical questions, drug interaction queries, and requests to analyze lab results. The models were slightly more likely to include warnings for mental health questions, likely because AI companies have faced public scrutiny and lawsuits over dangerous mental health advice to minors.
Perhaps most disturbing: the research found that as AI models produced more accurate analyses of medical images, they included fewer disclaimers. The models appear to calibrate their warnings based on confidence level. When they are more confident, they warn less. But even the companies that build these models explicitly instruct users not to rely on them for medical advice. The models are overriding their creators' own safety guidance.
Duke University: The Problem Goes Deeper Than Hallucinations
When most people think about AI medical errors, they think about hallucinations, the chatbot making something up entirely. But a February 2026 study from Duke University, led by Monica Agrawal of the Department of Biostatistics and Bioinformatics, found that the more dangerous failure mode is responses that are factually accurate but medically reckless due to missing context.
The Duke team analyzed 11,000 health-related conversations spanning 21 medical specialties and identified several systemic problems.
"The objective is to provide an answer the user will like. Chatbots avoid pushback because users prefer models that align with their views, creating unsafe situations."
The "agreeability problem" is perhaps the most insidious finding. These models are designed to be helpful and to satisfy users. When a user asks a leading question, the chatbot follows their lead instead of challenging their assumptions the way a real clinician would. A doctor reads between the lines. A chatbot reads the prompt and optimizes for approval.
The Duke researchers documented a concrete example: one chatbot warned that a particular medical procedure requires professional execution, then immediately explained step-by-step how to perform it at home. A human clinician would have refused to provide those instructions. The chatbot contradicted itself within the same response because both parts individually answered the user's implicit questions.
And the problem is compounding. Google already blends AI-generated overviews into search results, making the technology invisible. Hundreds of millions of people are now receiving AI-generated health advice without even realizing they are talking to a machine.
The Body Count Is Growing
This is no longer a theoretical risk. A 14-year-old boy in Florida took his own life after extensive conversations with a Character.AI chatbot. OpenAI faces multiple lawsuits alleging that ChatGPT provided dangerous advice to minors. The bromide psychosis case documented in the Annals of Internal Medicine is the first published clinical case of AI advice directly causing a psychiatric emergency, but physicians suspect it will not be the last.
The pattern is always the same. A person in a vulnerable moment turns to the most accessible, most confident, most available source of information they can find. The chatbot responds with the tone and structure of an expert. It never says "I don't know." It never says "this is beyond my capability." It never says "please call your doctor before doing this." It just answers.
And 40 million people a day are listening.
Help Us Document AI Medical Harm
Have you or someone you know received dangerous medical advice from an AI chatbot? Your story matters. Every documented case builds the evidence that regulators need to act.
Share Your Story