Tuesday, November 04, 2008

10 Reasons Why You Should Ignore Exit Polls

FiveThirtyEight's analysis

I have one reason: John Kerry.
Here are the first four from Fivethirtyeight: :
1. Exit polls have a much larger intrinsic margin for error than regular polls. This is because of what are known as cluster sampling techniques. Exit polls are not conducted at all precincts, but only at some fraction thereof. Although these precincts are selected at random and are supposed to be reflective of their states as a whole, this introduces another opportunity for error to occur (say, for instance, that a particular precinct has been canvassed especially heavily by one of the campaigns). This makes the margins for error somewhere between 50-90% higher than they would be for comparable telephone surveys.

2. Exit polls have consistently overstated the Democratic share of the vote. Many of you will recall this happening in 2004, when leaked exit polls suggested that John Kerry would have a much better day than he actually had. But this phenomenon was hardly unique to 2004. In 2000, for instance, exit polls had Al Gore winning states like Alabama and Georgia (!). If you go back and watch The War Room, you'll find George Stephanopolous and James Carville gloating over exit polls showing Bill Clinton winning states like Indiana and Texas, which of course he did not win.

3. Exit polls were particularly bad in this year's primaries. They overstated Barack Obama's performance by an average of about 7 points.

4. Exit polls challenge the definition of a random sample. Although the exit polls have theoretically established procedures to collect a random sample -- essentially, having the interviewer approach every nth person who leaves the polling place -- in practice this is hard to execute at a busy polling place, particularly when the pollster may be standing many yards away from the polling place itself because of electioneering laws. read the rest