PDF
Citation
Print
INTRODUCTION
Randomized controlled trials are considered a gold standard for establishing causal relationships.1) However, random allocation of intervention is often impossible in human studies due to reasons such as ethical considerations, among others. Therefore, several methodologies to infer causal relationships using observational data have been developed and used in the fields of human sciences. The essence of these methods is to consider factors affecting intervention assignment and to identify appropriate controls in the absence of controlled experiments.2)
The difference-in-differences (DID) method is one of the most popular methods in these “quasi-experimental” approaches.3) The method is widely used because its key concept is intuitive and unknown or unmeasured confounding factors can also be controlled.
Therefore, we briefly described the issues regarding the DID method, such as key concepts and outline, assumptions, model specification, and further considerations.
KEY CONCEPTS AND OUTLINE OF A DID ANALYSIS
The key idea of the DID method is to compare the differences between exposure and control groups before and after the exposure period. To estimate the potential outcome of the exposure group during the post-exposure period if exposure does not occur (counter-factual outcome of the group during the post-exposure period in the absence of exposure), the observed outcome change of the control group between the pre- and post-exposure periods is added to the observed outcome of the exposure group during the pre-exposure period (Figure 1).
In Table 1, the observed outcome of the exposure group during the post-exposure period is A+B+C+D and the potential outcome in the absence of exposure during the post-exposure period is A+B+C. Therefore, the effect of exposure is estimated as D (DID between pre- and post-exposure period).
Because the effect of exposure is evaluated by comparing the observed and potential outcomes of the same exposure group, unmeasured potential confounders can be cancelled out by the design, which is considered one of the strengths of this method.
WHEN TO USE THE DID METHOD
The DID method can be applied to data with observations before and after exposure in the exposure and control groups, respectively. Although the method can be applied to data with only one observation before and after exposure in exposure and control groups, respectively (Figure 1, Table 1), it can also be applied to data with multiple time points and exposures.
Application of method is appropriate in the case that the difference between exposure and control groups needs to be relatively stable if exposure does not exist. This parallel-trends assumption (i.e., the same time trend between groups) cannot be directly tested but can be evaluated through visual inspection of the similar trends during the pre-exposure period between groups.
CONSTRUCTING MODELS FOR THE DID ESTIMATOR
Although the causal relationship between exposure and outcome can be estimated by simple subtraction (Table 1), regression models are more commonly used because they can provide the results adjusted for potential confounders. In addition, such models can also provide statistical testing results and confidence intervals.
Regression models for estimating the causal relationship between exposure and outcome using the DID approach can be summarized as follows:
Ygp = b0 + b1×G + b2×P + b3×G×P + b4×Cgp + e
where Ygp is an outcome of interest for group G (control group, 0; exposure group, 1) in period P (pre-exposure period, 0; post-exposure period, 1), Cgp is potential confounders of group G in period P, and e is an error term. The causal relationship between exposure and outcome can be evaluated by testing the interaction between period (pre vs. post) and group (control vs. exposure) variables. Therefore, the regression coefficient of interaction term b3 is a coefficient of interest. If b3 is statistically different from 0, it can be interpreted that there is a causal relationship between exposure and outcome.
The following model can be used for DID analysis with multiple periods and groups:
Ygp = b0 + b1× Sgp + δg + γp + b2×Cgp + e
where Ygp and Sgp are an outcome and exposure for group G in period P, respectively. δg is a group fixed effect, γp is a period fixed effect, Cgp is potential confounders of group G in period P, and e is an error term. The regression coefficient for the interaction term b1 is a coefficient of interest. If b1 is statistically different from 0, it can be interpreted that there is a causal relationship between exposure and outcome.
FURTHER CONSIDERATIONS IN THE DID MODEL: DIFFERENCE-IN-DIFFERENCE-IN-DIFFERENCES
Until now, only one control group was considered. However, causal inference can be strengthened with additional control groups. For example, if certain health policy aims to lower mortality in low-income groups, controls can be included for those whose income is not low in the groups (e.g., regions) to which health policy is applied and not applied. Considering additional sources of variation in this way is known as difference-in-difference-in-differences (DDD). In most cases, results from DID analysis are presented as a sensitivity analyses to the main DID analyses due to the difficulties in justifying the adequacy of two control groups.
Regression models for the DDD analysis can be summarized as follows:
Y = b0 + b1×G + b2×V + b3×P + b4×G×V + b5×V×P + b6×P×G + b7×G×V×P + b8×C + e
where Y is an outcome of interest, G is a group (control group, 0; exposure group, 1), V is an additional source of variance (control, 0; exposure, 1), P is a period (pre-exposure period, 0; post-exposure period, 1), C is potential confounders, and e is an error term. The causal relationship between exposure and outcome can be evaluated by testing the three-way interaction term G×V×P and the coefficient of interest is b8. If b8 is statistically different from 0, it can be interpreted that there is a causal relationship between exposure and outcome.
CONCLUSIONS
This study explained key concepts, outline, assumptions, model specification, and further considerations of the DID method. Despite the strengths of the method, the assumptions, such as parallel trends and proper comparison groups, need to be carefully considered before application.4) If used properly, this method can be a useful tool for epidemiologists and clinicians to make causal claims based on observational data.
XML Download