Bayesian estimation of impact in experimental interventions with continuous outcome

Abstract

Impact evaluation in experimental intervention is an estimation of how design and implementation strategies affect the overall outcome and effectiveness of interventions by establishing causation. It provides the approach for determining the “Average treatment effect” and the “Average effect of treatment on the treated” otherwise known as impact. One of several approaches to estimating impact in experimental intervention is the use of the Difference-in-Differences (DID) estimation procedure based on classical regression approach which often overstates reality and rely on assumptions that are often violated in real life. Bayesian estimation is popular in literature but not widely applied in evaluation of experimental intervention. This study developed an approach for estimating impact based on the Bayesian estimation procedure. It derived a distribution for the difference of difference variable when outcome is continuous and normally distributed using the convolution procedure. The likelihood of distribution of the difference of difference variable was combined with the normal prior, at a specific value of the prior hyperparameter obtained from previous study, to estimate the posterior distribution. Using data from computer simulation and secondary data, the posterior mean was estimated. Also, classical regression approach was used to estimate the impact. Results from the Bayesian approach produced lower impact estimate and lower mean square error compared with the classical approach. This study provides a better alternative to estimating impact in experimental interventions.