03/03 · Essay
There is a version of the future of public health research in which we stop running trials. We have the registries, the surveillance data, the program records. We fit a flexible model that adjusts for everything, estimate the effect of an intervention from observational data, and move on. Several recent papers make a more careful version of this argument, and a few make the careless one. I want to defend a middle position, which is that causal machine learning is one of the best hypothesis generators we have had and one of the worst substitutes for a trial.
What the methods do well
Targeted maximum likelihood estimation and its relatives let you use machine learning for the nuisance parts of a causal analysis, the parts where you are modelling who got the intervention and what would have happened to them anyway, without letting the machine learning dictate the causal question. That is a real advance. The methods are honest about their assumptions in a way that the older regression habits were not, and a well-known review posed thirteen questions that any applied analysis should be able to answer before anyone believes its estimate. I have found that list more useful than most methods courses.
A 2025 roadmap paper titled, accurately, not a panacea, lands where I do. These tools are for finding where to look. In a dataset of maternal outcomes across hundreds of facilities, a causal forest can tell you that the effect of a supervision intervention seems to be concentrated in facilities with a particular staffing pattern. That is a hypothesis worth a field trial. It is not a result.
Why I keep saying trial
The reason is not methodological purity. It is that the things that break programs are not in the data. In the growth monitoring work, the variable that best explained follow-through was whether the supervisor framed her visit as problem-solving or inspection. No register recorded it. A model fit to the registers would have found a strong facility effect and attributed it to whatever happened to be recorded: staffing, caseload, distance to the district hospital. The attribution would have been wrong and confident.
Trials are expensive precisely because they force you into the field, where the unrecorded variables live. The temptation of causal machine learning is that it offers the vocabulary of causation without the obligation to go and look.
A workflow I can defend
Use the observational data and the flexible models to find heterogeneity, surprising subgroups, and dose-response patterns. Treat every finding as a question. Then go to the field with a short qualitative study to find out what the pattern means, because it will often mean something nobody put in the dataset. Then, and only then, design the trial that could change a policy, and let the model help with the sample size and the pre-specified subgroups.
The sequence matters. A model that generates a hypothesis has done its job. A model that settles a question has usually been asked to do a job it cannot do, by people who could not afford to find out otherwise. The honest response is to be clear about which is which, and to keep the trial budget.
Sources
- Causal machine learning in health research: not a panacea but a roadmap. 2025.
- Balzer LB, Westling T. Demystifying statistical inference when using machine learning in causal research: thirteen questions. American Journal of Epidemiology.
- Targeted maximum likelihood estimation for causal inference in observational studies: a review.
- Trinkley KE, et al. Leveraging artificial intelligence to advance implementation science. Implementation Science, 2024. https://implementationscience.biomedcentral.com/articles/10.1186/s13012-024-01346-y