Definition Yes, Analysis No. A comment on Analyses of change scores do not estimate causal effects in observational data

This is a pre-copyedited, author-produced version of an article accepted for publication in International Journal of Epidemiology following peer review. The version of record is available online at: https://doi.org/10.1093/ije/dyab202

Definition Yes, Analysis No. A comment on “Analyses of ‘change scores’ do not estimate causal effects in observational data”

Tennant et al. make the bold claim that “analyses of ‘change scores’ do not estimate causal effects in observational data” [1]. However, it seems to me that this claim is related to the definition of the effect of interest (the estimand) and not the estimation. It is well established, under the potential outcomes framework, that “the causal effect is the comparison of the potential outcomes, for the same unit, at the same moment in time post-treatment. In particular, the causal effect is not defined in terms of comparisons of outcomes at different times as in a before and after comparison…” (2, p6 – original emphasis). The authors’ estimand for the change score is defined by the outcome at baseline and follow-up so by definition they are correct that the change score from baseline is not a causal estimand, and this is well established. I am quoting from a major textbook on the subject co-authored by Donald Rubin who extended the potential outcomes framework from randomised controlled trials to observational studies. However, the same book makes it clear that for the estimation of causal effects, as opposed to their definition, we can, with assumptions, use outcomes at different times to estimate missing potential outcomes as it is not possible to observe all potential outcomes - the fundamental problem of causal inference. To quote Imbens and Rubin again, “a before-and-after comparison of the same physical object involves distinct units in our framework, and also the comparison of two different physical objects at the same time involves distinct units. Such comparisons are critical for estimating causal effects, but they do not define causal effects in our approach” (2, p7 – original emphasis). Hence, estimating causal effects using pre intervention observations to inform missing potential outcomes is very well established in many disciplines including public health [3]. It would seem a shame if readers were put off such methods by the ambiguous title of Tennant et al.’s paper.

Best wishes,

Frank Popham

Fife, Scotland frank.popham@protonmail.com

References

1. Tennant PWG, Arnold KF, Ellison GTH, Gilthorpe MS. Analyses of ‘change scores’ do not estimate causal effects in observational data. International Journal of Epidemiology [Internet]. 2021 Jun 7 [cited 2021 Jun 15];(dyab050). Available from: https://doi.org/10.1093/ije/dyab050

2. Imbens GW, Rubin DB. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press; 2015.

3. Wing C, Simon K, Bello-Gomez RA. Designing Difference in Difference Studies: Best Practices for Public Health Policy Research. Annual Review of Public Health. 2018;39(1): 453-469. Available from https://doi.org/10.1146/annurev-publhealth-040617-013507