Related Links: Steven Levitt's response to Scheiber's argument, and Scheiber's response to Levitt.
One of the few papers I actually read as a grad student was written by a pair of economists named Josh Angrist and Alan Krueger. In the early '90s, Angrist and Krueger set off to resolve a question that had been gnawing at economists for decades: Does going to school increase your future wages? Intuitively, it seemed obvious that it did. When you compared the salaries of, say, Ph.D.s with those of high-school dropouts, the grad-school set almost always did better. The question was whether education accounted for the difference. What if it was simply the case that smarter people spent more time in school and that their bigger salaries reflected intelligence, not education? One couldn't be sure. The only way to get to the bottom of it would be a ghastly social experiment, wherein you took a group of students and randomly sent half to the local vo-techinstitute while forcing the other half to study feminist literary theory. Even an economist wouldn't be so audacious.
That's where Angrist and Krueger came in. In the paper, they pointed out that two features of the public school system allowed you to answer the question without all the uproar. First, most states force students to attend school until age 16. Second, for many decades, students started school the year they turned six. The upshot was that, if I were born in January and you were born in December of the same year, and we both dropped out at 16, then the rules forced you to stay in school almost one year longer. (We’d start school the same year, but I'd turn 16 midway through tenth grade and you'd hit 16 midway through eleventh.) The additional schooling foisted upon one group by this arbitrary state of affairs produced a scaled-down version of our experiment, allowing Angrist and Krueger to conclude that education did, in fact, help people earn more money.