State of the Art Causal Inference in the Presence of Extraneous Covariates: A Simulation Study

Abstract

The central task of causal inference is to remove (via statistical adjustment) confounding bias that would be present in naive unadjusted comparisons of outcomes in different treatment groups. Statistical adjustment can roughly be broken down into two steps. In the first step, the researcher selects some set of variables to adjust for. In the second step, the researcher implements a causal inference algorithm to adjust for the selected variables and estimate the average treatment effect. In this paper, we use a simulation study to explore the operating characteristics and robustness of state-of-the-art methods for step two (statistical adjustment for selected variables) when step one (variable selection) is performed in a realistically sub-optimal manner. More specifically, we study the robustness of a cross-fit machine learning based causal effect estimator to the presence of extraneous variables in the adjustment set. The take-away for practitioners is that there is value to, if possible, identifying a small sufficient adjustment set using subject matter knowledge even when using machine learning methods for adjustment.