Randolph-Macon College issued the following announcement on Sept.
Election polls have played a prominent role in every presidential election since George Gallup introduced opinion research in 1936. For Grace Holderman ’23, understanding the various ways public opinion is collected is one of the most fascinating aspects of political science, so much so that it formed the basis of her Schapiro Undergraduate Research Fellowship (SURF) project, titled “Survey Says….! Comparing Poll Accuracy Over Time and Polling Methods in the 2016 and 2020 Presidential Elections.”
“Polling helps us understand what the people around us think, and it also influences how the media report on public opinion, which in turn shapes public opinion even further,” Holderman said. “It’s important for polling to be accurate so that the public isn’t misled.”
Her SURF project evolved out of coursework in Professor of Political Science and Dean of Academic Affairs Lauren Bell’s political science research methods class, during which she researched poll accuracy in the 2016 election. Dr. Bell recommended Holderman broaden her research topic over the summer with Communication Studies Department Chair Dr. Joan Conners, who teaches a class focused on campaigns and elections in the Communications Studies department. Together, they took Holderman’s modelling of the 2016 polls, expanded it, then applied it to the 2020 election to see how the accuracy of those polls fared between election cycles.
The Polls Are Open
Holderman collaborated with Dr. Conners during her SURF fellowship to analyze two different data sets from popular polling data aggregator RealClearPolitics—one from 2016, the other from 2020. Each contained 250 different polls from more than 20 pollsters like MSNBC, CNN, and The Hill/HarrisX. Together, they dug into each polling firms’ methodology to understand what types of questions were commonly asked and whether the polls controlled for variables like race, gender, and age.
“Some of the pollsters make that information really difficult to find,” Holderman said. “And I found that those are usually the ones that are the least accurate. What are they hiding, and why?
Holderman and Conners isolated variables like demographic data, polling methodology, and sample size to determine which ones did or didn’t influence each poll’s accuracy. Their research showed that polls soliciting more demographic data beyond race, gender, and age—particularly respondents’ education and location—tended to be more accurate than those that didn’t. They were surprised to learn that poll size had only a weak to moderate effect on poll accuracy. One of the polling firms they looked at surveyed upwards of 40,000 people. Despite the large sample size, it turned out to be no more accurate than another that surveyed just 2,500 people.
“We expected there to be a correlation because usually a larger sample size is supposed to reduce your margin of error,” Dr. Conners said. “In this case it didn’t matter very much.”
They were similarly stunned by the relationship between a poll’s accuracy and how far in advance of the election it was conducted. Polling voters in the period just days or hours before the election seems like it would garner the most accurate snapshot of their voting choices. Not so, Holderman said. In 2020, polls conducted a year before election day were the most accurate, even when compared to those performed just one week before election day.
“I’m still not sure why that is, but it would make for a great addition to this research,” Holderman said.
And The Results Are In
So were the polls all that different between 2016 and 2020? Although the headlines might lead you to believe otherwise, Holderman said the 2016 polls were unfairly maligned as inaccurate. “The polls did what they were intended to do,” Holderman said, “by correctly predicting the popular vote—not the electoral vote.”
Most of the variation between polling results could be attributed to differences in polling methodologies. For instance, interactive voice response polling, also known as robocalling, is often more reliable than human-to-human surveys because of the “social desirability” effect.
“That’s when a respondent may say something they don’t actually believe, and would instead give the response they think would be the socially desirable one,” Holderman explained. She suggests an easy improvement for most polls would be to use more non-human polling methods like internet surveys to control for that bias, as well as create more representative samples by polling different demographics like unregistered voters.
Citizens, on the other hand, should look for polls that are transparent about their design and methods. According to Holderman, a good poll will be so confident in its research that the general public can easily access information on how the poll was conducted, which questions were asked, and how many people were polled. When in doubt, she suggests looking to websites like FiveThirtyEight, a common polling ranker, for trustworthy pre-election sentiment.
“It’s really easy to use a tool like that to determine if a specific poll is reputable or not,” Holderman said.
Beyond the Classroom
Holderman, a Biology and Political Science double-major hailing from Roanoke, Virginia, first heard about SURF when she was applying to Randolph-Macon. She says she was immediately drawn to the opportunity to perform graduate-level research alongside faculty while still an undergraduate. Holderman has her eyes set on graduate school, possibly in data science or biology, and appreciates how SURF allowed her to sample what a full-time career in data science might look like. For now, she rounds out her college experience as a member of R-MC’s women’s tennis team as well as the Franklin Debating Society.
“SURF gives me the practical experience to learn more about those fields in a hands-on way,” she said. “I couldn’t have done this project without Dr. Conners’ mentorship. It’s a very large project, and she helped me with everything from idea generation to pinpointing the direction of the research to sorting through sources.”
Original source can be found here.
Alerts Sign-up