Voting Research - Methodology and Demographics
Click here to read about the purpose of our research (voting theory).
Survey Methodology
Throughout 2012 we conducted an online survey to solicit opinions on the US presidential election. The survey asked participants to score 11 people as choices for president. The 11 people were:
Democrats
Hillary Clinton
Barack Obama
Republicans
Michele Bachmann
Newt Gingrich
John Huntsman
Ron Paul
Rick Perry
Mitt Romney
Rick Santorum
Libertarians
Gary Johnson
R. Lee Wrights
We present four different polls.
Google Ads Poll
Recruiting
This ongoing poll has received over 550 completed surveys this year (after removing duplicate IP addresses) in response to the following two Google ads, which appeared on Google searches with generic keywords such as 'presidential election’ and 'poll,’ nothing specific to party or the primary process:
Feel free to click on the ads and take the survey if you'd like. We keep these responses separate from the randomly sampled responses.
The graph below shows the response rate. You can see that we fluctuated in the amount of advertising that we purchased during the first few months and then stuck with a steady amount for the remainder of the year.
 |
Notice that more self-declared “registered Republicans” have been responding than self-declared “registered Democrats.” Many polls would attempt to correct this representation by scaling the results from the different groups. However, this would require a judgment call that we are not willing to make (i.e. scale according to registered voters?…likely voters?…the population at large?). Instead we show the unscaled total votes and also separate the survey data by party.
Demographic
The total survey sample, broken down by party, was 36% registered Republicans, 22% registered Democrats, 31% not registered with either party, and 11% undeclared.
Survey
The survey presented the 11 options for president in random order, while keeping them clustered within parties (parties also presented in random order). The participants were asked to indicate their level of support for each candidate with a number between 0 and 100. They are also encouraged not to use the same number twice.
The participants are free to omit scores to indicate “no opinion” as they complete the survey, and they are warned that this will not be interpreted to mean that they don't like that candidate. This is an important distinction when calculating the vote outcomes; however, it is clear from inspection that many of the survey participants did not follow the instructions and omitted scores for their less preferred candidates. For example, some survey takers only gave a score of 100 for one candidate and didn't score anyone else.
The survey had a second page that asked for demographic information.
Time varying plots
To create time varying plots, we calculated a localized average. We used the Gaussian kernel, with a width (standard deviation) of one month, to form a weighted average of survey responses. We also used a similar weighted sum to calculate the response rate in the graph above, using a narrower width for higher resolution.
New York Times Blog Poll
Recruiting
On January 13 and 14, 2012, we placed a banner ad on a political New York Time blog, fivethirtyeight.com. We received 188 completed surveys (after removing duplicate IP addresses). The ad, shown below, featured Republican candidates, which probably explains the extreme Republican bias in the sample.
 |
Demographic
The survey sample, broken down by party, was 89% registered Republicans, 3% registered Democrats, 0% not registered with either party, and 8% undeclared.
Survey
The survey was exactly the same as the Google Ads Poll survey.
Mercer County Poll
Recruiting
Princeton University has recruited a panel of volunteers in Mercer County (the county where Princeton is located) to participate in opinion surveys. We obtained permission in February, 2012, to email this panel and solicit responses to our survey about the US presidential election. We received 288 completed surveys.
Demographic
The survey sample, broken down by party, was 16% registered Republicans, 47% registered Democrats, 33% not registered with either party, and 4% undeclared.
Survey
This survey used Qualtrics through Princeton University. Instead of entering numbers by typing, participants were asked to move a slider to indicate their preferences. The slider begins in the center position, which gives a score of 0, and can be moved to the right to a maximum score of 100 or to the left to a minimum score of -100. “No opinion” is recorded if the slider is not touched or the “no opinion” button is checked.
The survey had a second page that asked for demographic information.
Mechanical Turk Poll
Recruiting
Throughout February, 2012, we offered 10 cents for each completed survey about the US presidential election through Amazon's Mechanical Turk System. This system provides an inexpensive way to reach out to large groups of individuals. We received 472 completed surveys.
Demographic
The survey sample, broken down by party, was 15% Republicans, 16% Independents leaning Republican, 13% Independents, 24% Independents leaning Democrat, and 32% Democrats.
Survey
The survey was similar to the Google Ads Poll survey.
Flaws
“No Opinion”
We learned a few things from conducting this survey. The most important is that we need a new protocol for allowing the participant to not give input about a candidate. In our survey administered by Survey Monkey, the participants were told that blank responses would be interpreted as “no opinion.” But this is problematic. From inspection we see that people tend to leave blank entries for candidates that they do not like, as if to indicate a score of zero.
Here's how we handle blank scores. In the Condorcet method, if a score is blank on a ballot for either candidate in a pairwise comparison, then that ballot is not considered for the vote between that pair. In the plurality method and instant run-off, a blank score is equivalent to a zero (or, technically, -1). The ballot will only be cast in support for candidates who were scored. For this reason, it is possible to have the final two candidates of instant run-off finish in the opposite order of the Condorcet order. In range voting and Borda count, a blank score is not included in the average. Otherwise, ties in Borda count take the average value of the Borda scores for the tied positions.
A much simpler and more practical way to do preferential voting is to interpret blank entries as indications that the candidate is lowest in the voter's preference list. And, of course, the voters must be told this.
Our Qualtrics survey, used for the Mercer County Poll was better in this respect. There was an explicit check-box for “no opinion,” so it was clear to the participants what this meant. However, there was still a problem in that sliders that were not touched were interpreted by the software as “no opinion,” even though they might appear to give a score of zero (the middle of the range).
Download Survey Data
Data file: votedata.xls
Explanation: readme.txt