Understanding the science of presidential polling and what it really means

September 17, 2012 at 7:37 am Leave a comment

There are seemingly three certainties in life; death, taxes and the inconsistency of polling numbers.  One day the polls say one thing, the next something entirely different.  How is it possible to have such fluctuations within very short periods of time?  Understanding the science of polling provides the best insight into both quality and accuracy.   By understanding the principles of polling, it becomes easier to look at the polls as nothing more than a simple data point amidst thousands of other data points.  Paramount to this is also the recognition that polling is far from an exact science and is quite often wrong.

In 1980, Jimmy Carter was beating Ronald Reagan by eight percentage points just a week before the election.  He lost in a landslide.  Remember the poll in 1984 that had Walter Mondale defeating Reagan by 16%? Another landslide.  In 2004, John Kerry was up on George Bush by double digits.   Each and every election the polls spin out of control in the 24/7 news cycle moving so fast that your head might spin.

Before the presidential conventions, polls are relatively meaningless.   The American people are engaged in other aspects of their lives and really don’t pay much attention.   Shortly after their respective conventions, there is typically a bounce for each candidate.  Of course, that bounce is short lived and the polls remain relatively meaningless until the debates begin.   Only as the election looms closer do meaningful trends begin to emerge.

But what about the numbers?  Are they meaningful?  When an AP poll says that one candidate is favored over the other by 47% to 45%, what exactly does that mean?  First, one must look at the sample size to see if it was statistically valid.  In calculating sample size, to establish a margin of error with a 95% confidence level for roughly 185 million voters, 600 people need to be surveyed.  The accuracy of the survey is determined by two critical components; confidence interval and confidence level.

The confidence interval, or margin of error, is the plus-or-minus figure usually reported in newspaper or television opinion poll results. For example, if you use a confidence interval of 4 and 47% percent of your sample picks an answer you can be “sure” that if you had asked the question of the entire relevant population between 43% (47 minus 4) and 51% (47 plus 4) would have picked that answer.

The confidence level tells you how sure you can be. It is expressed as a percentage and represents how often the true percentage of the population who would pick an answer lies within the confidence interval. Most researchers and pollsters use the 95% confidence level.

When you put the confidence level and the confidence interval together, you can say that you are 95% sure that the true percentage of the population is between 43% and 51%. The wider the confidence interval you are willing to accept, the more certain you can be that the whole population answers would be within that range.

In a recent poll, 47% said that they would pick Barack Obama if the election were held today.  This means that the actual percentage from which you can be 95% certain would vote that same way is between 43% and 51%.   In other words, if you spoke to roughly 600 random people you can be 95% certain that between 43% and 51% would vote for Obama, a swing of 8 points.

Also, another key factor is the proportion of the electorate that is being polled.  For example, if 30% of the polling takes place in a state that makes up 2% of the population, there is disproportionate representation.  The same holds true for party affiliation, in particular this year, as many pollsters are consistently oversampling Democrats.  Their formula is based upon 2008 voter turnout, where Democrats did, indeed, show up in higher numbers.  But, from a historical perspective, voter turnout is generally higher among Republicans and Independents.

This is where the very important enthusiasm gap comes into place.  In other words, which party is more “fired up” about the coming election.  For example, in 2008, the Democrats were overwhelmingly more enthusiastic, whereas in 2012 this gap has shifted to Republicans.  However, many pollsters aren’t factoring this in, hence the disproportionate surveying of Democrats based upon turnout in the last election cycle.

It is also important to read the fine print at the bottom of the poll to understand whether those being polled are registered voters, likely voters or voters who actually voted in the last presidential election.  While millions are registered to vote, only about 55-60% will actually show up on Election Day.    Likely voters provide a more accurate picture, but even among likely voters, only about 80% will show up at the polls.

As you can see, polling is a science that even on its best day has opportunities for error.  Complicating matters is a media that often is cited as having its own agenda, hence skewing sample sizes and demographics to garner an outcome they desire.   For example, if a polling company wanted to show Romney with a sizeable lead, they would oversample Republicans or “red” states.   Conversely, oversampling Democrats or states like California or New York can show a big win for Obama.

With the polls being so tight, it is all that much more important to pay attention to the details.  Consider a recent Pew poll that included the following;  459 Republicans, 813 Democrats and 599 Independents.  Voter registration nationwide is fairly evenly spread between the two parties and Independents now enjoy a clear plurality, so just looking at the numbers tends to call into question the credibility of this particular poll.   As expected, this particular poll showed an Obama advantage.  Had the numbers been reversed, it would have likely favored Romney.  Had a true representation of likely voters been polled, it would have given a more realistic probable outcome.

As the news media bombards us with a flurry of polls, keep in mind that the devil is in the details and understand that there may be other agendas at play, as well.  At the end of the day, the only poll that matters is the one that is reported after the polls close, and that is the actual election outcome.  So don’t let the polls sway you and don’t let them deter you from voting.  Set aside some time on Election Day to go out and let your voice be heard.

Christopher Tidball is an executive consultant and author of multiple books including Kicked to the Curb: 20 Essential Rules For Coming Out On Top When Your Life Has Been Turned Upside Down.   His rules for success have been featured on CBS Market Watch, The Wall Street Journal, ABC News, MSNBC, Kiplinger’s and dozens of other media outlets.  To learn more, please visit www.christidball.com or e-mail chris@christidball.com


Entry filed under: Subrogation. Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , .

Comparative Negligence: The Million Dollar Answer Unbundled: How to review a BI demand and pay the right amount

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 145 other followers

Contact the Author

Chris Tidball is a claims and revenue management consultant and author of the "20 Essential Rules" series of self and organizational improvement books. You can ask him a question at chris@christidball.com

Kicked to the Curb

Kicked to the Curb


Finding Millions on Twitter

Error: Twitter did not respond. Please wait a few minutes and refresh this page.

%d bloggers like this: