Monday, January 25, 2010

Replicating experiments with propensity score matching

I've been trying to learn some propensity score matching and consequently have been perusing some papers. The Smith-Todd paper "Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators?" was a good useful starting point for me. I was a little perplexed by the desire of the authors to "hit" the experimental estimate though. Presumably, if the experiement were repeated, it would not achieve the same treatment effect as the original - the treatment effect has a standard error or confidence interval around it.

Agodini and Dynarski's paper "Are Experiments the Only Option? A Look at Dropout Prevention Programs" was also interesting. The authors don't try to match the experimental effects but ask if the direction of the experimental effect can be concluded based on propensity score methods. Also interesting was the whole question surrounding the standard error of the propensity score estimate - whether the simple random sample estimate is "close" to the bootstrapped estimate or not since there have been claims that the standard error from the propensity score estimator is from an estimate based on nonlinear methods and hence not reliable. They find that the bootstrapped estimates are similar to an SRS standard error.

No comments: