July 25, 2011 § Leave a comment
As proof, instrumental variable approaches are basically useless, because even attacks from people who don’t really get that correlation isn’t causation are difficult to dismiss.
I’m willing to say IV estimates, when assumptions are close to met, are better than naive regressions, but how much better? I need to look at whatever work has been done on the robustness of IV estimates.
(edit: I don’t think the Tragedy of the Commons is the biggest issue, but it also deserves further consideration)
May 16, 2011 § Leave a comment
Noah Smith writes about a draft paper linked to by some right-wing economists that purports to use instrumental variables to show that the Obama stimulus caused a net loss in jobs. He correctly states that the evidence presented for a net loss is not statistically significant. But it seems to me that the fundamental problem is not that they’re interpreting the numbers wrong; it’s that they have the wrong numbers. So let’s take a moment to explain why the instruments are bad.
The point of instrumental variables is to get around confounding. A necessary condition for this to work is that the instrument and the outcome must not be confounded themselves.
The common causes of stimulus funding and employment changes one might worry about include wealth, income distribution, and urbanisation. Let’s group these together under the heading “development”, and see if they affect the proposed instruments. If they don’t, they may be usable instruments.
Will development affect highway improvement funding? Yes.
Will development affect Federal taxes and spending? Yes.
Will development affect the relative importance of sales tax? Yes.
Will development affect the strength of a balanced budget rule? Yes.
Will development affect whether the governor is a Democrat? You betcha.
All the proposed instruments are themselves confounded. It is very difficult to think of a variable affecting stimulus funding that appears as if randomised with respect to development. This is because stimulus funding is not random.
This means that if you want to use a regression-type approach, you need to control for development and all other confounders. This may be impossible. If, however, it is possible to control for all confounders, then you don’t need instrumental variables!
This is why the instrumental variable approach — not just the instruments used here — is the wrong way to work on this problem, and many problems like it.