1.2 Seconds Per Claim: What Happens When AI Decides Who Gets Healthcare

According to a class-action lawsuit, a major health insurer used an algorithm to review and deny hundreds of thousands of patient claims, spending an average of about 1.2 seconds on each one. A separate suit against another large insurer alleges its AI tool for evaluating elderly patients' care had a startlingly high error rate, and that the company leaned on it anyway.

Both companies dispute how their systems have been characterized. Those disputes matter, and this article takes them seriously. But whatever the courts ultimately find, these cases have already surfaced the defining question of automated decision-making: what happens when a business uses AI to make high-stakes decisions about people at a speed and scale no human could match, and what does the business owe the people on the other end of those decisions?

This is the heaviest case in the series, and one of the most important, because the lesson reaches any organization thinking about using AI to decide, approve, deny, prioritize, or flag anything that materially affects a human being.

What the lawsuits allege

The first case, filed in 2023, concerns a system one insurer used to review claims against preset criteria. The lawsuit alleges the tool let the company deny large batches of claims with little or no individual medical review, and cites a period in which roughly 300,000 claims were denied over about two months, an average of just over a second per claim. The plaintiffs argue that patients were told a medical professional would review their claims, and that an automated batch process is not that.

The insurer disputes this characterization. It has said the tool in question is not AI or machine learning at all, but a long-standing sorting technology that matches billing codes, and that the great majority of claims it touches are actually approved. That's a genuine dispute of fact, and it's part of why the case is being litigated rather than assumed.

The second case, filed later in 2023, concerns a different insurer's tool used to predict how much post-acute care, such as a stay in a skilled nursing facility, an elderly patient should need. The suit, brought by families of deceased patients, alleges the tool was used to cut off coverage when a patient's actual recovery ran longer than the algorithm predicted, that it carried a very high error rate on appeal, and that staff were pressured to stay close to the algorithm's projections. The insurer responds that the tool is only a guide to inform care planning, not the thing that makes coverage decisions, and that those decisions follow plan terms and government criteria.

Courts have allowed parts of these cases to move forward. The facts will be sorted out there. What's already clear is the shape of the risk.

The number that frames everything

Set aside which side is right about the technology, and look at the pattern the plaintiffs describe, because it's the one every business automating decisions should study.

One figure in particular does a lot of work: the allegation that only a tiny fraction of patients, well under one percent, ever appeal a denial, while a large majority of those who do appeal win. If both of those are true, you have a system that loses most of the fights it picks but is rarely challenged, because the people affected mostly don't have the knowledge, energy, or resources to push back. An automated denial engine aimed at a population that rarely appeals can be, from a narrow financial view, highly effective precisely because it's wrong in a direction that mostly goes uncontested.

That is the moral and legal trap of automated decision-making laid bare. When AI makes an adverse decision about a person, the harm lands on an individual who often can't tell whether the decision was sound, and frequently lacks the means to contest it. Speed and scale, the things that make automation attractive, are the same things that make an unfair automated decision so damaging: it happens instantly, in volume, to people poorly positioned to fight back.

Why this matters even if you're not an insurer

Most businesses won't ever deny a medical claim. But the underlying capability, using AI to make or heavily shape decisions about people, is spreading into lending, hiring, pricing, fraud flagging, eligibility, and beyond. Wherever it goes, the same principles follow, and they echo the Workday case earlier in this series: when you delegate a consequential decision to a system, you own the outcome, and "the algorithm decided" is not a defense that protects you or serves the person affected.

The regulatory direction is unmistakable. Automated decisions that affect people's health, money, or livelihood are drawing increasing legal and legislative scrutiny, and the standard emerging is that a business can't hide behind the tool. If your system makes a decision, you are responsible for its fairness, its accuracy, and the recourse available to the person it affects.

If you're going to automate decisions about people

Keep meaningful human review on high-stakes decisions. There's a real difference between AI that helps a qualified person decide and AI that decides on its own. For decisions that materially affect someone's health, money, or rights, a human with the authority and the time to actually review, not a rubber stamp, needs to stand behind the outcome. If the promise to the customer is that a professional will review their case, an automated batch process has to honor that promise, not quietly replace it.

Measure accuracy by outcomes, especially reversals. If a large share of your automated decisions get overturned when someone challenges them, that is not a footnote, it's a flashing warning that the system is wrong at scale and only getting away with it because most people don't appeal. Track your reversal rate and treat a high one as the emergency it is.

Make appeal easy, not theoretical. A right to appeal that almost no one uses isn't functioning as a safeguard. If your automated decisions are sound, an easy appeals process costs you little; if they're not, a hard one is just hiding the problem. Design recourse for the person, not against them.

Never let a cost target drive the decision logic. The most dangerous version of this is an automated system implicitly or explicitly tuned to produce a financial result, denials, savings, throughput, rather than an accurate one. When staff are pushed to conform to the algorithm's output regardless of the individual case, the tool has stopped being a decision aid and become a way to launder a predetermined answer. Build systems to be right, and watch closely for incentives that quietly redefine "right" as "cheaper."

The real lesson

We help organizations automate real work, and this is the category where we push clients hardest to slow down and design carefully, because decisions about people's health, money, and livelihoods are where automation done badly does the most harm and carries the most liability. AI can genuinely help here, it can triage, surface information, and bring consistency to decisions that were previously erratic. But it has to be built with human review where the stakes are high, honest measurement of whether it's actually right, real recourse for the people affected, and a hard wall against letting cost targets shape the outcome.

The allegation at the center of these cases, a decision about a human being made in a little over a second, is a warning about what becomes possible when speed and scale are pursued without those safeguards. Whatever the courts decide about these specific tools, the businesses that automate decisions about people responsibly will be the ones that treated the person on the other end as someone they answer to, not a claim to be cleared.

A note: this article touches on serious illness and loss. It's a discussion of business and legal accountability, not medical or legal advice; anyone facing a coverage denial should consider speaking with a qualified attorney or patient advocate.