Google Scraps Its AI Medical Tool. ChatGPT Health Misses Half of Emergencies.
The AI Healthcare Crisis Is Getting Worse.

Something remarkable happened in March 2026. Not remarkable in the aspirational Silicon Valley sense, where "remarkable" means a breakthrough that changes the world. Remarkable in the quieter, more alarming sense: two of the biggest technology companies on Earth, within days of each other, demonstrated that their AI medical tools are not ready for the people using them.

Google scrapped a feature that had been feeding crowd-sourced health advice from Reddit and online forums to millions of users. OpenAI's ChatGPT Health, used by roughly 40 million American adults every day, was exposed by a peer-reviewed Mount Sinai study as incapable of recognizing more than half of the life-threatening emergencies presented to it.

These are not isolated glitches. They are symptoms of an industry-wide failure to treat medical AI with the gravity it demands.

Google Quietly Kills Its Crowd-Sourced AI Medical Feature

In mid-March 2026, Google pulled the plug on "What People Suggest," an AI-powered search feature launched in March 2025 that aggregated health advice from Reddit, Quora, and other online forums. The feature used AI to distill comments from strangers on the internet and present them alongside search results for medical queries, giving unvetted personal anecdotes the same visual prominence as guidance from the Mayo Clinic or the CDC.

Think about that for a moment. You search "chest pain when breathing," and Google's AI was pulling answers from a thread where someone named xXBongLord420Xx shared their experience. Not a cardiologist. Not a peer-reviewed journal. A Reddit commenter whose medical credentials are entirely unknowable.

A Google spokesperson attributed the removal to "routine simplification of search results," insisting it had nothing to do with quality or safety concerns. That explanation is difficult to reconcile with the timing. In January 2026, The Guardian published an investigation revealing that Google's AI Overviews, the broader AI-generated summary boxes that appear at the top of search results, were actively spreading false and misleading health information to an audience of roughly 2 billion monthly users.

When Reddit Became Your Doctor

The premise of "What People Suggest" was not entirely unreasonable on paper. Lived experiences can be valuable. Knowing that other people have gone through what you are going through, hearing how they navigated a confusing symptom or a difficult diagnosis, has genuine emotional and practical value. Patient communities on Reddit and elsewhere have helped millions of people feel less alone.

But there is a canyon-wide difference between reading a forum thread and understanding that it represents one person's unverified experience, and having Google's AI extract key claims from those threads and serve them as structured, authoritative-looking answers in search results. The former is a support group. The latter is algorithmic malpractice.

The Guardian investigation documented specific examples that illustrate how dangerous this can get. Google's AI Overviews advised pancreatic cancer patients to avoid high-fat foods, directly contradicting established dietary guidance for people with that diagnosis. It misinterpreted liver test results. It provided incorrect information about women's cancer screenings. These are not minor errors. For a patient making real decisions about their care, wrong information about cancer screening protocols could mean a delayed diagnosis. A delayed diagnosis could mean the difference between Stage 1 and Stage 4.

Google's decision to quietly remove "What People Suggest" without acknowledging the safety implications is itself a telling choice. If the feature was safe, why remove it? If it was not safe, why did it take a year and a damning Guardian investigation before action was taken? Neither answer reflects well on a company that processes the majority of the world's health-related searches.

ChatGPT Health: 52% of Emergencies Missed

While Google was pulling its crowd-sourced health feature, the evidence against OpenAI's ChatGPT Health was growing even more damning. On February 23, 2026, Nature Medicine fast-tracked the publication of what researchers described as the first independent safety evaluation of ChatGPT Health since its launch in January 2026.

The study, conducted by researchers at Mount Sinai, was rigorous. It was not a casual test of a few prompts. The team created 60 clinician-authored medical vignettes across 21 clinical domains, then ran them under 16 factorial conditions to produce 960 total triage responses. Three independent doctors reviewed the cases using established clinical guidelines before comparing the AI's recommendations against professional assessments.

The headline finding: ChatGPT Health undertriaged 52% of gold-standard medical emergencies. More than half the time, when a real doctor would have said "go to the emergency room now," ChatGPT Health said something closer to "schedule a visit in the next day or two."

The system did perform well on textbook emergencies. Stroke symptoms, severe allergic reactions, the kinds of scenarios that even a first-year medical student would flag correctly. But it struggled catastrophically with nuanced presentations where the danger is not immediately obvious. Those are precisely the cases where clinical judgment matters most, because they are the cases most likely to be fatal when missed.

The researchers described what they called an "inverted U-shaped" pattern of failures. Performance was worst at the two extremes: genuine emergencies (52% undertriage rate) and nonurgent presentations (where 64.8% of safe individuals were incorrectly told to seek immediate care). The system was simultaneously too relaxed about real danger and too alarmed about minor issues. It could not reliably distinguish between the two.

The Inverted Suicide Safeguards

The most disturbing finding in the Mount Sinai study involved ChatGPT Health's suicide prevention safeguards. OpenAI had designed the system to direct users to a suicide crisis line when it detected high-risk situations. This is, in theory, exactly the kind of safety measure a responsible AI health tool should have.

In practice, the safeguards were backwards.

Researchers found that crisis line alerts triggered more reliably when users described no specific method of self-harm than when they articulated a concrete, detailed plan. The safety net activated for lower-risk users while failing to activate for higher-risk users. The system effectively inverted the relationship between actual risk and safeguard activation.

The study also uncovered a significant anchoring bias. When a family member or friend in the scenario minimized a patient's symptoms, saying something like "I'm sure it's nothing," triage recommendations shifted dramatically toward less urgent care. The odds ratio was 11.7. In practical terms, a casual reassurance from a bystander, filtered through the AI, could be the difference between an emergency referral and a recommendation to wait it out.

ECRI Names AI Chatbots the Top Health Hazard for 2026

These failures did not emerge in a vacuum. The nonprofit patient safety organization ECRI, which has been identifying health technology hazards since the 1960s, named the misuse of AI chatbots in healthcare as the number one health technology hazard for 2026. Not number five. Not "something to keep an eye on." Number one, topping a list that includes surgical robot malfunctions and medication dosing errors.

ECRI documented cases where AI chatbots suggested incorrect diagnoses, recommended unnecessary testing, promoted subpar medical supplies, and, in one particularly creative failure, invented body parts that do not exist. All while maintaining what ECRI described as "the tone of a trusted expert."

That last detail is crucial. The danger of AI medical advice is not just that it is sometimes wrong. It is that it is wrong with absolute confidence. A Google search result that links to a Reddit thread at least allows the user to see where the information is coming from and calibrate their trust accordingly. An AI system that synthesizes bad information into a fluent, authoritative-sounding paragraph removes that critical layer of skepticism.

A study from the University of Oxford, published in February 2026, reinforced this concern: patients who received AI-generated health advice were significantly more likely to act on it without seeking professional confirmation than patients who found the same information through traditional search results. The medium changes the message. AI makes bad advice feel trustworthy.

Timeline of AI Medical Failures

The Pattern Nobody Wants to Acknowledge

Step back and look at the full picture. Google built a feature that fed unvetted Reddit health advice to billions of users, kept it running for a year, then quietly killed it while insisting nothing was wrong. OpenAI launched a health product used by 40 million Americans daily that fails to recognize more than half of life-threatening emergencies. ECRI, the organization whose entire purpose is identifying technologies that harm patients, put AI chatbots at the top of their danger list for 2026.

The pattern is unmistakable. Tech companies are deploying AI into healthcare not because it is ready, but because the market opportunity is too large to wait. Health information is one of the most searched categories on the internet. Capturing that traffic, keeping users inside your ecosystem when they are frightened and searching for answers about a lump or a chest pain or a child's fever, is enormously valuable. The incentive structure rewards speed over safety.

Professor Paul Henman, a digital sociologist at the University of Queensland, warned that widespread domestic use of ChatGPT Health could "feasibly lead to unnecessary harm and death" by creating a dual failure: a surge in unnecessary medical visits for minor conditions alongside a simultaneous failure to seek care during genuine emergencies. The Mount Sinai data supports exactly this prediction. ChatGPT Health was simultaneously too cautious with low-risk cases and too casual with high-risk ones. It is the worst possible combination.

The fundamental problem is not that these tools make mistakes. Every diagnostic tool makes mistakes, including human doctors. The problem is that these tools are being deployed at a scale of billions of interactions per month, without the regulatory frameworks, clinical validation, or informed consent processes that we require of every other medical technology. A new blood pressure monitor undergoes years of clinical trials before it reaches the market. ChatGPT Health was launched, used by 40 million people daily, and only then independently tested.

There is a version of the future where AI genuinely helps patients. Where it catches the rare disease a doctor missed. Where it flags an interaction between medications that could be dangerous. Where it democratizes access to medical knowledge for people who cannot afford to see a specialist. That future is worth building toward.

But we do not get there by deploying half-finished tools at planetary scale and hoping the failure rate stays low enough that nobody notices. We do not get there by pulling health tips from Reddit and serving them as medical guidance. We do not get there by building suicide safeguards that activate for the wrong people.

We get there by doing the work first. By running the clinical trials before the launch, not after. By treating medical AI with the same regulatory seriousness as every other technology that can harm patients. By acknowledging that moving fast and breaking things is fine for social media features, but it is unconscionable when the thing you break is a person's health.

Until then, if you are experiencing a medical emergency, call your doctor. Call 911. Go to the emergency room. Do not ask an AI chatbot. In 2026, that is still the safest advice anyone can give you.