Election Polls: Part 5: Size and Margin of Error

When judging the accuracy of polls you see, be sure to look at the size of the poll and the margin of error. In polling lingo, that's MOSE (margin of sampling error).

For those of you pretentious types, you can pull that out at your next cocktail party. As in, "because the MOSE of the last Gallup Poll was plus or minus three, it's a pretty good reflection of the current mood of the country." Now let's go through the basics.

Pollsters have figured out the mathematical model for accuracy of poll results based on the size of the poll. Here's the general breakdown of the margin of error by sample size:

Sample 50 people, plus or minus 15% margin of error

Sample 500 people, plus or minus 6% margin of error

Sample 1000 people, plus or minus 3% margin of error

Sample 5000 people, plus or minus 1% margin of error

Many state polls use 500-person poll samples. This means if the state poll shows Obama and McCain at 50/50, they could actually be anywhere between 46% and 56%, which is not really a dead heat.

A 1,000 sample is standard among national polls, but at a cost of $50,000, it's too expensive for many state polls. There's nothing wrong with using smaller polls, but it won't give the most statistically accurate result. Always look at the methodology and the margin of error.

The Associated Press has guidelines for reporting these polls. If the difference between the candidates is more than twice the sampling error margin, reporters can say the candidate is "leading." This is usually 7 or more percentage points in a 1000 person poll. If the sampling error margin is 3-6 points, then the candidates are "close." If they candidates are 0-2 points apart, they are considered "tied." But remember, if the sample is only of 50 people, there is a 15-point margin of error, so it wouldn't be accurate to say that candidates are tied, even if the poll shows them at 50/50 results.

Today's Gallup poll shows Obama leading McCain by 4 points. Obama's at 49%, and McCain's at 45%. This is based on a 2,720 person sample, with three-day rolling averages of responses to interviews. So, the margin of error is plus or minus 2%. Since twice the margin of error is 4%, it is safe to report Obama as in the lead under the AP guidelines.

But hopefully these numbers are not impacted by the Bradley Effect we reviewed in the last post. This would happen if bigoted poll participants either refrained from voting or lied (to the pollsters or themselves). Sometimes people answer polls differently than they actually vote when alone in the booth.

Test out your knowledge so far. Here's the link to the Gallup poll from today: Gallup Daily: Obama Leading McCain by 4 Points. Be sure to look for who, when, and the MOSE. You will see that is a sample is registered voters, so you know from previous posts that this does not show new voters or possible swing voters. Look at the "Survey Methods" section to evaluate.

Now you know just enough to be dangerous - like the pollsters!

Subscribe to Kimberlie Ryan's Working Wellness