Medha Thapliyal

Sheet 03 Writing Revised October 2026

Sheet 03/06. 2 April 2026. 3 minute read.

Five thousand alerts, zero outcome studies

India's media-scanning surveillance system has flagged thousands of potential outbreaks. Nobody has yet shown it changed the course of one. Surveillance AI needs the discipline we demand of drugs.

03/06 · Essay

Since 2022, the National Centre for Disease Control has run a system that reads news. Built with a non-profit AI lab, it processes hundreds of millions of articles in thirteen languages, extracts health events, and hands a shortlist to epidemiologists who decide whether to act. The published description reports more than ninety-five thousand events extracted and more than three thousand five hundred shortlisted as potential outbreaks by the human reviewers. Press coverage cites over five thousand alerts in three years.

I think this is good engineering. Event-based surveillance from open sources is a recognised complement to indicator-based reporting, and keeping a human in the loop at the point of decision is the right design. I also think it is a useful case for a question the field keeps avoiding: what would it take to know whether a surveillance AI works?

The missing study

Working is not the same as running. A system can process every article in the country, surface thousands of events, and still make no difference to any outbreak, because the outbreaks were already known through the routine channel, or because the alert arrived after the district had acted, or because the alert arrived in time and the district could not act. I have not found a published evaluation that asks any of these questions. There is no comparison of alert timing against the integrated disease surveillance program’s own detection, no analysis of what happened after an alert, no estimate of cases averted.

This is not a criticism of the team. It is a description of what the field accepts. We would not license a vaccine on the basis that it was administered five thousand times. We would ask what happened to the people who received it compared with those who did not. For surveillance AI we ask only whether it produced output.

What a real evaluation would look like

The hybrid effectiveness and implementation designs that implementation scientists use for clinical tools apply here directly. A Type 1 hybrid would ask whether the system detects outbreaks earlier than the routine channel, using a matched historical period or a stepped introduction across states, while also recording how district teams actually use the alerts. A Type 2 would test the alert and the response pathway together, because an alert into a district with no outbreak budget is an email. A Type 3 would assume the detection works and study the implementation strategy: who reads the alerts, how fast, with what authority to mobilise.

The water-borne hepatitis outbreaks I have spent time around are a good test case. They are seasonal, they cluster around predictable failures in water supply, and they are often reported in local-language press before they appear in a facility register. If a media-scanning system cannot show an advantage there, it is hard to see where it would.

The deeper point

AI in public health is accumulating impressive pipelines and almost no outcome evidence. This is partly because the pipelines are built by engineers who measure throughput, and partly because the funders who pay for them are satisfied by throughput. Implementation science exists to ask the next question. We should be asking it of surveillance systems with the same stubbornness we bring to a new intervention in a clinic, and we should expect the answer to be sometimes disappointing. That is what evidence is for.

Sources

03 · Adjoining

Keep reading.

  1. May 2026 ABHA before algorithms India's bet is that identity, registries, and consent come before models. I think the consent manager, not the model, is the real AI policy.
  2. Sept 2026 The 0.9 percent Rapid qualitative analysis appears in less than one percent of implementation science studies. AI may finally make it common. The question is what we trade for the speed.
  3. Aug 2026 The red alert and the ASHA A clinical copilot in Nairobi cut diagnostic errors most when it interrupted. What would an interruption mean for a community health worker with no clinical backstop?