Thursday, June 30, 2011

Randomized Control Trials

Dear avid readers:

This may seem naive...but why do we care about causality in the LR?
We want to teast out causal effects so that we know "de-worming-> increases school attendance", and isn't confounded by some other thing that's driving up school attendance.

But who cares..if my sample is big enough and I see a positive correlation, then in general I know deworming affects something that affects school attendance.
Even if we determine whether de-worming and increased school attendance is not causal, and don't implement the program, it doesn't mean we'll figure out the variable that is, and why not still implement the program?

Perhaps it is because we are concerned that:
1) if it's a confouder that increase school attendance via de-worming, what is it, and what if it drops off?
2) if it's a confonder and we dont' know what it is, then we can't scale this up

What if the market and people that de-worming is offered to is pretty constant. Why do we care about causality then?

Just something I've been kicking around on runs...

Reader1: I think that we care about it for the two reasons that you mentioned
-- if we implement de-worming but the correlation is actually due to a
third variable that may "drop off" or that may have a change in how it
is correlated with the other variables (or something else happens that
impacts that third variable such that the correlation changes) then
the de-worming stops working and we do not know why or what we are to
do then. so causal is most important with respect to policy
implications. But if the third variable is very stable and the
correlations do not change, then from a policy perspective, the
de-worming works so who cares? the only reason we might care in that
case would be if there were a cheaper or more tenable policy option
that we are blind to because we ignored the potential third variable.

In terms of defining relationships, I believe causality does not matter.

Reader 2: The thing about de worming as an intervention is that it is super cheap and it works. In fact, it probably the cheapest thing you could do other than nothing. The only way this study is interesting in an economic sense is if school attendance post deworming goes down. kids lives are improved in other dimensions and you now have a tradeoff to study. But this is not the result, so we are done and nothing is surprising. Now we can see how more costly interventions compare.

Reader 3: I agree, causality is key for policy. Suppose there is only correlational evidence of deworming and schooling. i.e. we observe that some villages have implemented deworming programs and these villages also have greater educational attainment than villages that have not implemented deworming. one possible 'threat" to a causal interpretation of this evidence is that the villages that deworm might also happen to have higher than average income (or lower than average income if these programs are sponsored by donors and targeted at poor villages) and it is income that is really causing differential investments in education and educational attainment.

If a government is deciding whether to invest in a deworming program, there is no guarantee at all -- based on this evidence -- that schooling will improve if a deworiming program is implemented in the villages that have not already implemented the program. hence the need for an RCT.

but maybe you are thinking about the mechanisms- proximal vs. ultimate causes... rather than correlations?

R Squared

Why and When to Use it?

Economists are not so keen on using R-squared, even the Adjusted R-Squared, which due-fully adjusts for the the number of parameters being estimated in a model.

Here are some 2 cents about it:

Reader 1: I thought I'd put in my 2 cents here -- so far int he stuff I have
done with Ken and Andreas and Erkut, no one seems to care about the R2
at all! And papers I have read recently do not discuss it, though I
always report it. I n general, if the adj R2 goes up, it sggests that
the model fit is better with the increased variable included. We use
adjusted because adding a var will always make the regular R2 go up so
it does not really tell us anything (which you probably already know).

Reader 2: I don't ever use R squared. I used the adjusted R squared for the informal use of Altonji (2005) to see robustness of the coefficients to additional independent variables. So lets say you are looking at if years of schooling leads to higher growth. Now someone might claim your estimation is too parisomonious. Lets say you may not have included inflation. So you add inflation, see if the adjusted r squared improves. If it improves, its a better fit. And if your coefficient of years of schooling is still significant, it implies your estimation is robust. That is basically when I use adjusted r squared. I read somewhere that the F-stats are better.

Reader 3:
Yes to adjusted r-squared...both the Fstat and adjusted r squared do the same thing in terms of adjusting for the # of parameters being estimated (hence the adjustmnet), otherwise r squared straight up increases as the # variables in your model (i.e parameters being estimated) increases.

Why? because over a concave space, you will always acheive a lower minimum (summ of sqrd errors or residuals will go down) with more variables in your function. Hence SSerrors goes down and Rsquared=1-(SSE/SST) goes up.

R squared adjusted divides by the number of variables (parameters) to counteract the SSE going down. Same thing with F stat.

Response to Reader 2: This is the sentence I wanted: "And if your coefficient of years of schooling is still significant, it implies your estimation is robust."
So, reader 2 uses adjusted r squared for robustness.

But what if adj r sq goes up but schooling becomes insig?
Not robust-right?
Throw out the model?

Reader2: If your adj- r squared goes up, and years of schooling looses significance, it implies that inflation is a) important to your model and b) years of schooling is capturing something inflation is explaining. So your estimation is not robust and you have to either justify why inflation is incorrect to put in the model theoretically, or claim that inflation steals away an important years of schooling affect. Maybe inflation reflects status of the economy, and whne you have a bad economy, schooling plummets. Anyways, here is where everything becomes an art form. Obviously if you add every possible variable out there, you will eventually lose significance, so theory has to guide your specification. Of course this is just a robustness check. I found a paper that claimed it wasn't the greatest robustness check either, as you are hadn picking measurable variables, when your issue is ommitted variable bias.

Reader 3: I'm a non believer in adj r-squared over except for Asif's above.
Why does everyone else use it so much? sociologists? business? do they not know better?

To be (Sig)?

Q: If you multiply a significant coefficient with another significant ceofficient, is the resulting coefficent always significant? (standard errors probably calculated by delta method)

A:No, there's no statistical reason that it should be.
If you multiply them together, you are estimating a new statiscal average, that has a different sd.