Polling Is Ubiquitous, But Is It Bad For Democracy?

TERRY GROSS, HOST:

This is FRESH AIR. I'm Terry Gross. Now that the presidential primaries are underway, there are new political polling results just about every day. Is polling good or bad for democracy? That question frames the article Jill Lepore recently wrote in The New Yorker, called "Politics And The New Machine."

She reports that polling may never have been less reliable or more influential than it is now.

She writes (reading) from the late 1990s to 2012, 1,200 polling organizations conducted nearly 37,000 polls by making more than 3 billion phone calls. Most Americans refused to speak with them. Lepore is a staff writer for The New Yorker and a professor of American history at Harvard University. Later, we'll hear from Nate Silver, founder of the popular website FiveThirtyEight, which analyzes political polls and predicts outcomes.

Jill Lepore, welcome to FRESH AIR. So, you know, you write in your article that polls may be more influential than ever. You also write that you think polls may be less reliable than ever. A typical response rate to polls is in the single digits. Less than 10 percent of people who are asked to respond to a poll actually follow through and respond?

JILL LEPORE: Yeah. Isn't that a little staggering to think about?

GROSS: Is it the low single digits or the high single digits?

LEPORE: (Laughter) There's a range. I think nine is not uncommon. I mean, it can be very much at the high end of the range. I think that - to know whether that's a crisis or not requires understanding what pollsters do or what public opinion researchers do more broadly. They use a sample survey method, so they conduct interviews with what is supposed to be a statistically representative sample of the electorate. That sample is going to be tiny, and a sample works to represent all of our views as an electorate, so long as it is well-chosen. Choosing that (laughter) sample is harder when the response rate is lower. When public opinion survey research started in the 1930s, the response rate was well above 90. People considered it a civic duty. It was like a thrill. Someone would come knock on your door and want to talk to you for 45 minutes about politics. Now I think if someone calls you at 6:01 p.m. (laughter) and wants to talk to you about politics while you're trying to get the, you know, the chicken fried, you hang up the phone. There are a whole bunch of forces that are behind that. There's just a fatigue. There are so many pollsters. And then there's - just people don't have landlines anymore. I think it under - only 40 percent of Americans have land lines, and you can't do random dialing to cell phones. The FCC made that rule, and they recently reinforced it.

And then you have the problem of the people who answer the phone are really different from a random sampling of the electorate, right. The people who answer the phone tend to be older. They tend to be more conservative. Those are the people who have landlines. And then the people who answer the phone and are going to really participate (laughter) in your survey, they tend to be people who have a very strong sense of civic obligation, and they're involved in their communities. They tend to be voters. You know, they're very goodhearted (laughter), civic-minded people, and they're overrepresented, unless you work harder to represent the smaller number of people that you can get who are not those people. So pre-election polling, in particular, is just in crisis.

So you know what pollsters will say - because pollsters want to get the right answer, right? It's not actually a fraudulent industry. Their bread-and-butter is making an accurate prediction. They will say they can accommodate and moderate the ill effect of a very low response rate by weighting their responses and by doing more calling. And so that means polls get more expensive and the results don't really bear out those promises. So there's this whole other kind of cottage industry in trying to figure out which polls might be reliable.

GROSS: So if the people who actually respond to polls are older, they're more civic-minded. They're more likely to be a churchgoer. What's the population of people who are less likely to respond to polls? In other words, who's being left out of the polls?

LEPORE: The representative sample in a poll should represent the possible eligible voters. What happens with a really low response rate is that the representative sample in a poll represents the likeliest voters. That is to say the people who (laughter) are mostly be to answer a poll are also most likely to vote. So this, for pre-election polls, where really what you're trying to do is figure out what's going to happen in an election, in fact, in some ways, the low response rate makes those polls more likely to be accurate, right, because this a big problem of predicting an election. You can get people to tell you who they might vote for, but you can't tell whether they're going to vote or not. But actually, the bigger question then is - isn't that actually completely inimical to representative democracy?

Like, should we really be - then we're disenfranchising everybody who doesn't answer the poll. Do you know what I mean? Like, there's so many forms of disenfranchisement in our political culture, and polls have, in a weird and completely unintentional way, become yet another one of them. So in a world of widening political inequality and economic inequality, polls have become - you know, they're just riding that same wave.

GROSS: Let's talk a little bit about polling history. You write that pollsters rose to prominence by claiming that measuring public opinion is good for democracy. What is that argument and who first made it?

LEPORE: That argument was made in the 1930s by George Gallup who founded the first public opinion polling research institution in Princeton, N.J. in 1935. And before Gallup began conducting his public opinion surveys, there really weren't polls in anything like we know them today. Polls - they're so ubiquitous it's easy to forget that there was a time before there were polls. Like, people are always calling elections, like, you know, it'll be the 1828 election between Jackson and John Quincy Adams and people - I wonder who's going to win. I think I know who's going to win. And you might even go around to your precinct and ask people, knock on their doors - they're like, who you voting for? Go down to the bar now - who you voting for? Trying to make a guess and deciding maybe where to put more resources if you were running for office.

But doing what Gallup did, which is using the sample survey method of social science which was new at the time to predict elections, was only really done by him in the 1930s. Gallup actually wanted to be a journalist. He wanted to be a newspaper editor. But when he was in college, there was no journalism program, so he studied psychology instead. And then he went and got a PhD in psychology. But think about the 1930s then when he starts doing this and saying, you know what? Public opinion - if we could - elections come only every two years. But we in this fast-moving world, he said, we need to be able to measure public opinion every minute instantly. We need to know the will of the people. And part of the reason for that was there was tremendous anxiety in the 1930s about basically the kind of demagoguery that Americans feared that the radio - here I am talking to you on the radio - might be introducing. Radio was greeted with great celebration, but it was also very gravely feared.

I mean, the war of the worlds is just a few years after George Gallup founds the American Institute for Public Opinion. Public opinion, it can be contorted by a voice coming into your living room and telling you what to think. And that's how fascism grew in Europe. So a countervailing method - they would go the opposite direction from the broadcast of a radio - would be to actually call somebody or go to their door and ask them what they think and have them tell you. And then you would report that. That is, you could turn the radio upside down, and this would be the way democracy could fight fascism almost in this kind of very mechanical way. But Gallup, when he started out, he wanted to measure what he called public opinion - that is, the views that ordinary Americans held. He actually didn't want to forecast elections or conduct polls - pre-election polls. He was doing opinion surveys. But he found that no one really believed his opinion survey results because how are you going to check that? Like, 47 percent of Americans believe in God (laughter). Well, I don't know - says George Gallup. Why should I believe George Gallup? I mean, he made a bunch of phone calls? I don't know.

So he - to prove that his method was accurate, that he was actually measuring something real, he started asking people how they were going to vote so that he could then predict an election just basically as a stunt in order to convince people that his public opinion research had teeth. And then he found -kind of, I think, somewhat he was disheartened to find people loved these predictions. They, you know, people eat that stuff up. (Laughter) Obviously we do. People love to read election predictions. And so that, more or less, took on a life of its own. But he really didn't defend that as good for democracy. He defended public opinion measurement as good for democracy.

GROSS: You know, there are so many newspapers and TV stations and radio stations and other media outlets who sponsor their own polls now. When did that proliferation in polling start to catch on?

LEPORE: It really started in the 1970s. So for a long time, Gallup was the poller. There was also Roper and then the Harris Poll followed soon after. So there was significance - in almost a way there were kind of, you know, the network television stations. There's kind of the big three. There were the big three for a long time in the '40s, '50s and '60s. And then there grew to be a lot of concern about what seemed to be the proliferation of smaller polling organizations. So much so that in 1972, Congress debated a Truth-In-Polling Act because they wanted to help voters discriminate between good polls and bad polls. And the act didn't pass. And then in 1975, The New York Times and CBS, I believe, collaborated on the first basically, you know, major newspaper poll. And it's been off to the races ever since. I've, you know, talked about this piece on, like, a local radio station here in Boston. And that local radio station had just launched its own polling collaboration with a local college. And I said, like, how do you justify that?

Like, in the 1970s, people said you're actually creating news and your job is to report news. But it was tremendously profitable for those newspapers and those television stations and radio stations to pay for polls to be done in order to report those polls. I think the proliferation most recently, like in the last 10 years, the number of those kind of smaller organizations - small colleges, for instance, that get into this industry, smaller radio stations, smaller town newspapers - like, I think the Union Leader in New Hampshire has its own pollster. That is actually about the incredible shrink in sources of deep journalism. So it's about the crisis of newspapers and the crisis of news more generally with the rise of the Internet and kind of niche reporting. It's very cheap to conduct a poll relative to doing the kind of deep reporting that would be required to actually get some sense of an electorate and report in a - using something other than pure quantification.

GROSS: If you're just joining us, my guest is Jill Lepore, and we're talking about her recent article in the New Yorker called "Politics And The New Machine: What The Turn From Polls To Data Science Means For Democracy." And Jill Lepore is a staff writer for The New Yorker and a professor of history at Harvard. Let's take a short break here, then we'll talk some more. This is FRESH AIR.

(SOUNDBITE OF MUSIC)

GROSS: This is FRESH AIR, and if you're just joining us, my guest is Jill Lepore. We're talking about her recent article in The New Yorker about polls, called "Politics And The New Machine." She's a professor of history at Harvard and a staff writer for The New Yorker.

So everybody paid so much attention for so long to Iowa - to predicting what the outcome of the Iowa caucuses will be and then reporting on the outcome and then trying to read the tea leaves to predict what the outcome of the election is going to be. And I realize that in Iowa its caucuses. It's not primaries. But, you know, in America we pride ourselves on having a secret ballot. You go to the voting booth. People aren't allowed to give you campaign literature within a certain number of feet of the polling booth. You go in. You close the curtain. You vote. No one knows what you vote unless you tell them. It's your decision whether you want to tell anybody or not.

You're under no obligation to. In the Iowa caucuses, you're in a gym or you're in a room. Everybody who's there knows what you're doing because you're telling them. Or you're walking to a side of the room to indicate what you're doing. And there's something about it that just seems so counter to the idea of it's an American democracy with a secret ballot. There's so - I mean, there's such possibility for, like, peer pressure, for intimidation, for, like, wanting to go with the crowd, you know, wanting to go along with your spouse or your best friend because it's...

LEPORE: Yeah, and it's so funny that you...

GROSS: ...Visible and public.

LEPORE: Yeah, but it's so funny that you would have a kind of suspicion that that's somehow anti-American but that polling, which is, of course, a version of that, like, that bandwagon effect, wouldn't necessarily leap out at you in that same way. But in fact, the secret ballot is a latecomer in American political culture. It isn't really adopted - the first presidential election where the secret ballot is the preponderant mode of voting is 1896, which is also the first presidential...

GROSS: Wow, really?

LEPORE: ...First presidential election where some is not killed on Election Day. No, voting in public is far more ancient and more American in that sense. So the whole idea of being a good citizen requires publicly exercising your franchise. So people - you go to the polls and you would - the government didn't supply a ballot. You would - you know, first you'd have to - first, they were all viva voce. You'd basically be like a caucus. You'd go to the polling place and be like, OK, if you are voting for Smith, stand over here against the butcher's. And if you're in favor of Jones, stand over there down by the library. And that's how the votes were made. And the polls would be counted. That - poll means the top of your head, so people would count the tops of people's heads, and that's why it was called the polls (laughter).

So the call to reform public voting, or what was known as open voting, was super controversial because - Massachusetts was the first to do it. And they had this idea that the government would supply these envelopes, and you could bring - so people would start - when the party system got really strong, and newspapers were partisan, like, the Republicans would print a ballot - like, a whole party ticket. It would be, like, we'll say it's red. And the democratic newspaper would print a blue party ticket. And so you'd go to the town hall - this is when oral voting had kind of been replaced by paper voting because people were literate. But still, you'd have this giant, long sheet, like a railway ticket, like two-foot long. It would be brightly colored. And so people would try to prevent you from getting to the ballot box and casting your vote. The parties would hire these thugs to go down there. Democratic thugs, you know, would beat up all the Republicans with their blue tickets and prevent them - but people would die.

People were killed every election (laughter) in these incredible battles over trying to get to the - then there was this thing called vest-pocket voting. So this was a little bit sneakier. If you wanted to keep your vote private, you'd fold up your long, you know, blue ticket, and you'd stick it in your vest pocket and try to get to the polls without anybody, you know, knocking you on it. But this was considered unmanly. So when Massachusetts in the 1850s tried to say, well, we'll just supply envelopes and people can put their tickets in the envelopes before they come to the ballot box, it was repealed because people said that only cowards would use an envelope to vote, you know. So it was a really controversial thing. It takes years for the secret ballot to be adopted, and it's part of, actually, a lot of forces that are not - people wanted to - didn't realize there was a lot of corruption as a result. All kinds of party machine nonsense - people are being beaten up, you know, that's obviously not good. Also, women wanted the right to vote, and they were like, we wouldn't need to vote in secret. We don't want to get hit in the head. So suffragists were sort of supporting the secret ballot. And it was first passed in Massachusetts and New York in the 1880s.

But then for years, the only other places that adopted the secret ballot - which is a written ballot supplied by the government to each voter - was the South after Reconstruction because it was a way to disenfranchise newly-enfranchised black men who - none of them knew how to read. I mean, they'd been, you know, raised in slavery, lived their entire lives as slaves on plantations. And so it was - the real success of the secret ballot as a national political institution had to do with the disenfranchisement of black men.

GROSS: So the secret ballot was a way of helping them get the vote.

LEPORE: No, it's helping - it was preventing them from voting. If you could cut your ballot out of the newspaper, and you're going to vote a party ticket, and knew you wanted to vote Republican, and that ticket was going to be read, you didn't have to know how to read to vote. Immigrants could vote. Newly-enfranchised black men in the South could vote. It actually was a big part of expanding the electorate. But people in the North were like, hey, we don't really like when all those immigrants vote. And people in the South were like, we really don't want these black guys to vote, so they, in a sense, kind of colluded over - and there were good reasons for the secret ballot too. But they - very much motivated by making it harder for people who were illiterate to vote. It's essentially a de facto literacy test. And so...

GROSS: Because in the polling place - like, in the election booth - it wouldn't be, like, code-colored like that, or you couldn't ask anybody to read it to you. Is that what you're saying?

LEPORE: You couldn't - right. And so there's some counties in Virginia, I think it is, that in the 1890s they print some regular ballots. But then they print ballots in Gothic type - like, deep medieval Gothic type. And they give all those ballots to the black men. It's a completely illegible ballot. So there's a very - so anyway, people did - people debated it a lot of the time for those reasons, and also - I mean, if you think about it, why does Congress vote in the open? Well, it could be if you're a congressperson, your vote should be known to the public. That's part of transparency that we believe in, and it seems obvious to us. People believed that way about ordinary citizens voting as well for a long, long time. So the caucus in Iowa, that method which is so kooky and kind of fascinating - that's deliberative democracy. That is a certain kind of exercise of civic virtue that needs to be conducted in public.

GROSS: So I am stunned by everything you have just told me because I thought the secret ballot was one of the principles American democracy was built on. Did you always know this did you find that out later in life?

LEPORE: No, I totally didn't know. I always have to find things out because I just get curious about them. Remember the election with the Florida ballot - the hanging chad and everything? So I was asked to write a piece about voting machines or technologies in voting because we were looking like we were maybe moving to Internet voting. And I thought, I don't know if that's - going from paper voting to Internet voting doesn't seem that big of a deal, but what's a really big deal would be going from oral voting to paper voting. I wonder what that was like. And so then I did all this research and realized that, oh, the secret ballot is just such an incredible latecomer. And it's - the whole story of its origins utterly shocked me, and was really illuminating because it made me think about how Victorian our voting is. Do you know what I mean? It's like, go in this little booth with a little curtain, and you've got to be alone, and it's going to be dainty and private. It is actually a kind of Victorian domestic, quiet, sacred, middle-class space. And that was carved out by middle-class reformers, you know, against the kind of hurly-burly of, you know, the rowdy, exciting, drunken voting day.

GROSS: My guest is Jill Lepore. Her article "Politics And The New Machine," about whether polling is good or bad for democracy, was published in The New Yorker. After a break, we'll talk about how campaign season got so long. Also, we'll talk with Nate Silver, the founder and editor of the website FiveThirtyEight, which for many politics-watchers is the go-to place for analysis of the polls and predictions of election outcomes. I'm Terry Gross, and this is FRESH AIR.

(SOUNDBITE OF MUSIC)

GROSS: This is FRESH AIR. I'm Terry Gross back with Jill Lepore. She wrote a recent article in The New Yorker about whether political polls are good or bad for democracy. We've been talking about that and about some fascinating election history. She's a staff writer for The New Yorker and a professor of American History at Harvard.

So here's another American history question for you. It always seems that the presidential campaign gets longer and longer and longer. And I don't even mean the permanent campaign where after you win the election, you are campaigning immediately for reelection. I just mean like the relentless press coverage of the horse race. It seems longer and longer each presidential election. So if we look back historically, how much time was devoted to the actual campaigning?

LEPORE: So campaigning used to be considered completely gauche. So there was no presidential campaigning at all until Andrew Jackson. So people thought if you actually tried to convince the people to vote for you that you were a demagogue, that you should be such an esteemed man in society that everyone would know who you are and would be familiar with your accomplishments and let it to them to decide. This is very kind of courtly notion. This is a very small electorate of, you know, propertied white men. They're the only people who can vote anyway. Those people do know who you are. How you get to run is to do with, you know, your party nominates you in this completely closed proceeding. Andrew Jackson comes along. Andrew Jackson is not a natural candidate.

GROSS: What year are we talking here?

LEPORE: 1824. He's not a son of - he's not a founding father and he's not a son of a founding father. And everybody who had been president so far had been either a founding father or a founding son. And he's - you know, he's kind of like an orphan, brutal soldier guy. But he's famous for this battle. And he's - I'm going to start campaigning. And he hires a biographer to write the story of his life. It's the first campaign biography. And then he starts, like, talking to people. You know, he invites people. When he is inaugurated, he invites, like, ordinary people to come see him be inaugurated. I mean, this is the beginning of - so much of our political culture is - can be traced to Jackson. So then there's this great age of campaigning, but it's very - it is absolutely last-minute. In most (unintelligible), you've got to have someone write your biography about it maybe a year in advance. You get those books out. And then you got to buy a lot of whiskey. And you get to go - there is, like - when people become the log cabin candidates - the Whig Party - they hired these, like, rolling log cabins and like, yeah.

William Henry Harrison I think campaigns in these rolling log cabins like John McCain's bus tour or something. So there's a bunch of campaigning in the 19th century. In fact, it retracts in the 20th century because - (unintelligible) Wilson is the first guy to, like, have a record - he is - or Harding maybe - has a record. He cuts an album on his campaign speech. So he says, I'm not going to travel. Like I could travel by railroaded. I'll just do these, you know, whistle stops. But there's this sort of weird retraction. FDR, you know, he does newsreels. He is the first newsreel president. From 1932 he uses newsreels. So there's a kind of remoteness almost. The campaign stretches in time. But the candidate becomes more remote from the people, which is in some ways what the caucuses and primary movement is about, right? Getting these candidates to actually come to your states.

GROSS: Yeah, maybe FDR did the newsreels because of his polio.

LEPORE: Oh, yeah. Absolutely, absolutely. But then, by then, by 1932, that's the first newsreel. 1933 is the first political consulting firm, this Campaigns Inc., they start using the radio. And they - you know, they will come into your living room. And they will tell you you should never vote for FDR. The man will doom, doom, doom the country, you know what I mean? Like, they have this whole other - they have actually an amplifier.

GROSS: Right, and that gets to where we started (laughter)...

LEPORE: Yeah.

GROSS: ...Which is that public polling was seen as a way to combat radio 'cause through - strangers coming into your home and telling you who to vote for, it could lead to fascism.

LEPORE: Yeah.

GROSS: But if we had public polling, the public opinion would be represented, and you wouldn't be relying on that voice intruding.

LEPORE: Yeah, the voice of the people is the answer to propaganda.

GROSS: Right, OK. Jill Lepore, thank you so much.

LEPORE: Thanks for having me, Terry. It's been a blast.

GROSS: Jill Lepore's article about whether polling is good or bad for democracy was published in The New Yorker.
TERRY GROSS, HOST:

With so many political polls coming out, it's hard to evaluate which are most accurate and how much weight we should give them in predicting winners, which is why so many people turn to Nate Silver for his polling analysis. He's a statistician who's famous for founding the website FiveThirtyEight, which analyzes polls based on accuracy and methodology, aggregates polls and forecasts outcomes. In 2008, he correctly called all 35 Senate races and the winners of the presidential contests in 49 of 50 states.

The site's title, FiveThirtyEight, is a reference to the number of votes in the Electoral College. There's a new "FiveThirtyEight" podcast in which Silver and other reporters from the site analyze polls and primary outcomes and talk politics. In addition to political polls, the site reports on science and health, economics, culture and sports. "FiveThirtyEight" is now affiliated with ESPN and was formerly affiliated with The New York Times.

Nate Silver, welcome to FRESH AIR. Polls are supposed to measure public opinion. But do you think political polls also sway public opinion?

NATE SILVER: I think they can. There is evidence of something called the bandwagon effect, which is people like to be associated with a winner. Donald Trump, I think, knows this effect well. Or when you go to any one of his speeches and he spends literally half the speech talking about how great his polls are here and there. He stopped that a little bit after Iowa, but I'm sure he'll start it back up now after he won New Hampshire. So there is some of that. I would say, though, it's more prevalent in the primary than the general election. The reason being in the primary you have multiple candidates. Until recently, you had 17 Republican candidates. People usually like several of those candidates, and they have to coordinate and decide - which one of the six Republicans I really like, should I vote for? And it's usually the guy - probably a guy on the Republican side, especially, who's up in the polls.

GROSS: Why is that?

SILVER: Partly, it's technical. I mean, maybe I like Jim Gilmore, the former governor Virginia. But if I know that only 0.01 percent of people...

GROSS: (Laughter).

SILVER: ...Are voting for him, I'm not really having much of an impact with my vote, whereas choosing between Trump and Ted Cruz and Marco Rubio and Jeb Bush - that will have quite an effect on where the delegates go in my state and how the results are interpreted as we go into other states.

GROSS: You rate the polls based on their accuracy and on their methodologies. Most people don't know the difference between a good poll and a bad poll, an accurate poll and a poorly done poll. Do you think bad polls have a bad effect?

SILVER: Oh, for sure. And you have a lot of bad polls who are dependent upon the good polls to even be remotely accurate. So what they'll do is they'll say well, we have some bad data. Maybe we used a Robo Dialer. Instead of having a human being talk to people, you just set up an automated script. Maybe they'll call 30,000 phone numbers, actually only get 500 respondents, so a very low response rate. And then they'll say, you know, these numbers don't look very good. But Pew Research or the NBC/Wall Street Journal poll or the ABC/Washington Post poll, they have Clinton up 3, so I'm going to tweak my assumptions and have Clinton, let's say, up 4 or 5 instead. That's not what the good pollsters do. What the good pollsters do is a lot more scientific. But there are a lot of bad pollsters that kind of trade on the name and the reputation that, you know, frankly, scientific, but also very expensive, polls have built up over a number of years.

GROSS: If I heard you correctly, you're saying some polls cheat, that they just kind of make up the score.

SILVER: Let's - I would say -I want to be careful about what I would say. Right? I'd say that these polls usually have a lot of choices in terms of how they model turnout. For example, how they determine who's a likely voter and who isn't, how they weight for demographics. And they tend to make choices that are in line with what other polls do, so the technical name for this is herding. The polls sometimes all say the same thing. So if you remember when...

GROSS: That's H-E-R-D.

SILVER: H-E-R-D-I-N-G. So if you remember when then-Senator Clinton upset Senator Obama in the New Hampshire primary eight years ago in 2008, she had been down by 8 points. What was remarkable is that it wasn't just that she was down in one or two polls. It was, like, the same margin in every poll - 8 points in this very, very volatile environment. And once one pollster weighs in, especially a good poll, then people say - you know, what? - I'm not quite sure what's going on here, but I feel more comfortable in the pack. And so of the many different valid models that I might choose from, I'm going to pick this one pollster - pick this one model that is consistent with the consensus.

So ordinarily, in most walks of life when you see all the signs pointing to the same thing, that's a good, reassuring sign. Sometimes, in polling, that can be worrisome (laughter). It means that everyone is making the same assumptions. Everyone is looking at what one another are doing, and therefore, you can have real surprises when every single poll is wrong.

GROSS: How far ahead do you think it's reasonable to try to predict what's going to happen based on polls?

SILVER: So it's a different answer for the general election and the primaries.

GROSS: I mean right now (laughter).

SILVER: You know, I would say about three weeks out, it's worth looking at polls in individual states. But there can be switches right up until the last minute. In 2012, I think in, like, the first 7 states that voted, for the Republicans, the candidate who was ahead in the polls a week before the race, lost. So you can look at the polls, but you need these very big margins of error around them. And it's kind of an exercise in humility in the primaries, whereas in the general election, they have a record of being pretty accurate. So basically, all that we do is we run around during the primaries telling people - be careful. Be careful. The polls are not that reliable. And then in the general election, people are trained to be very wary of the polls. They, historically, have turned out to be very accurate instead. So we kind of flip around and say, you know, actually, I know it's only October, but the polls have been right nine out of 10 times by this point in the year.

GROSS: If you're just joining us, my guest is Nate Silver, who's the founder and editor in chief of FiveThirtyEight, the famous website which grades political polls based on their accuracy and methodological standards and forecasts winners in elections.

Let's take a short break here then we'll talk some more. This is FRESH AIR.

(SOUNDBITE OF MUSIC)

GROSS: This is FRESH AIR. If you're just joining us, my guest is Nate Silver, the founder and editor in chief of the website FiveThirtyEight, which is famous for grading political polls based on their accuracy and their methodological standards, for aggregating polls to see what the terrain, and based on that, for forecasting winners. So the way the primaries and the caucuses are set up now, Iowa and New Hampshire really count for a lot. As a pollster, when you look at voting from, you know, a mathematical point of view, and you try to see, like, who's representative, does it make sense to you to have those two states be the first states and to set the tone for what's to come?

SILVER: I think not particularly. They're not the most representative states. And we've spent time in both states, I like both states a lot as a kind of election tourist, but as I think should be obvious, they're both extremely white states. They don't have a lot of big cities. There aren't really major cities at all in New Hampshire. They both tend to be middle to high income and education levels. And so there's an attempt to rectify that with Nevada and South Carolina. But it is strange that these two states that are, you know, some of the less-representative states of the country as a whole have so much influence over how the rest of the country will wind up voting.

GROSS: Every vote in America is supposed to be equal, but it often seems like they're not, in the primaries and the caucuses - that some votes count more than others. Some states count more than others. And then you've got the delegates and the superdelegates, and the math is done differently for that. For example, you know, Hillary won by the tiniest margin in Iowa and Bernie Sanders won by a really wide margin in New Hampshire, yet she's ahead in delegates.

SILVER: Well, I'd be a little bit careful. I know a lot of news organizations lump together elected delegates and superdelegates when they report the numbers. But remember, in 2008, Clinton began with a large lead over Obama in superdelegates. And that flipped as the election went on because Obama had done well enough where he won the majority of elected delegates, and superdelegates said, I don't want to override this mandate that Obama won from democratic voters. So if Clinton is counting on that to hold up if Sanders - let's say he beats her in Nevada and South Carolina, then I think you'll see panic, number one. But eventually, I think Democrats would say, you know what? Even though we might not want Sanders as our nominee, to override what our voters just told us would be an even bigger and more consequential disaster for the party in the long term. But again, that's just a guess. If you had as big an upset - Sanders beating Clinton would be a much bigger upset than Obama beating Clinton eight years ago. And so to some extent, if that happens, or if Donald Trump wins, we're in uncharted territory in terms of how the party establishments would react.

GROSS: Any other thoughts you have about what would happen if Donald Trump wins and how the Republican establishment would react?

SILVER: I mean, it's maybe the most fascinating nomination race that we've ever seen, or at least that I've ever seen. What's interesting is that in New Hampshire, even though Donald Trump has led basically every poll there since July, only about one-sixth - 18 percent - of the negative ads in New Hampshire were directed against Trump. All these candidates competing to be the moderate or, quote, unquote, "establishment" choice, are sniping at one another, and the front-runner is coasting by and just won 35 percent of the vote, more than doubling the second-place candidate. So ordinarily after a big win in New Hampshire, people would say, well, we were a little bit iffy after Iowa, but now we know that Trump is real. But now we know that everyone and their mother will be coming after Donald Trump. He would be in the spotlight. It might not hurt him. He has some very loyal voters. But at least it might prevent him from gaining further.

But instead, in South Carolina, you have the fourth-place candidate and the fifth-place candidate, Rubio and Bush, in a giant rivalry with one another instead of aiming at the front-runner Donald Trump instead. So by the time they have this figured out, Donald Trump is going to have accumulated more delegates. Ted Cruz will probably have a couple of states in the South. The Texas primary is a lot of delegates - that's coming up on March 1, I believe. And they might all be fighting for second place, or if not second place, a case where they can only win by going to the convention in Cleveland, which could be very, very damaging too.

GROSS: So how accurate are polls now? And here's what I'm thinking. In terms of how technology has changed, a lot of people don't have land lines. There are federal restrictions on how you can use cell phones for polling. You're not allowed to do those kind of automated robo calls. And a lot of people don't answer their phones. They're just going to wait to see, like, who called, and then decide if they want to call them back or not. I know there's some Internet polling. So would you say that the new technology on the whole is helping to make polls more or less accurate? What are the biggest flaws now?

SILVER: Oh no, I have to say that on balance, I'm worried about the state of polling right now. So traditional telephone polls - and let's take the best polls, like Pew Research, for example, who do call cell phones. Their response rates - so how many people actually respond to the poll, the ones they would like to respond - has gone from about 35 percent 20 years ago to I think it is 9 percent now. So they're kind of hoping the 9 percent of people they do reach are representative of the 100 percent of people they would like to reach. In some sense, it's worked remarkably well. I mean, the polls were pretty good in the past two presidential elections. But we've seen cases in other democracies, such as the United Kingdom, for example, Greece, Scotland, Israel, where the polls were pretty far off on election day. And I think that should worry people. Eventually, I think, like everything else, we'll lead our lives online, and online polling will be the standard. But people haven't really figured out what the kind of gold-standard methodology is for online polling.

GROSS: I've heard journalists and pundits counting Twitter followers of candidates and Facebook likes as if they were polls accurately measuring something. I've heard journalists and pundits reciting Google searches of candidates as if that was indicative of something meaningful. Are those accurate measures of anything?

SILVER: I think people should be wary. They're interesting data. We don't have enough history to know which of those tell you anything and which don't, although there was one useful indication that proved to be prescient both in Iowa and New Hampshire, which is that, which candidates are voters searching for on Google? So in Iowa, we saw in the last 24 hours before the caucus, a big surge in searches for Ted Cruz and Marco Rubio. In New Hampshire, we saw a big surge in searches for John Kasich. And all through, those candidates beat their polls on election day in Iowa and New Hampshire respectively. So when you see something going on on the ground or in the data in the states voting next and it's not happening in other states, that to me is pretty meaningful. If you see a big spike in searches for Trump because he said something funny on "Saturday Night Live," then that's a different story.

GROSS: My guest is Nate Silver, founder of the blog FiveThirtyEight, which analyzes polls, aggregates polls and predicts outcomes. We'll talk more after a break. This is FRESH AIR.

(SOUNDBITE OF MUSIC)

GROSS: This is FRESH AIR. My guest is Nate Silver, the founder and editor in chief of the website FiveThirtyEight, which is famous for rating and analyzing political polls and predicting election outcomes. You have also written that you think Donald Trump wouldn't do as well in general election as he's done so far in the primaries. Why are the primaries a better field for him than a general would be?

SILVER: Well, from what best we can tell, Donald Trump has very intense support with maybe a third or 35 percent of the American population. And people outside of that one third of the electorate don't like Donald Trump very much or in some cases, not at all. So right now, out of every candidate, Democratic or Republican, he has the worst favorability ratings with the population as a whole. Now, I think he would probably change and adapt. He's kind of said - warned people - warned Republicans, anyway, that I will probably change my positions and become more of a centrist if I win the nomination. And that could help him potentially. But Donald Trump's popularity is so far confined to an important segment of the Republican base. That might be enough for him to win the nomination if no one else is doing any better. But I'd put it like this - I think he's a very, very high-risk candidate. Having been one of many people to be dismissive of his chances early on, I certainly wouldn't rule out the chance that Donald Trump could become not just the GOP nominee but the next president. He could also be a candidate, though, who winds up losing by 12 or 15 percent in the worst landslide since 1984.

GROSS: So Jill Lepore in her New Yorker article about polling asked the question, are polls good for democracy? What do you think?

SILVER: I think good polls are good for democracy. And the reason why good polls are good for democracy is because it's sometimes, for better or worse, the only chance that regular people have to have their say. You can vote for president, you can vote for senator or for governor, but you can't vote on a trade deal. You can't vote on a treaty. You can't vote on a big tax cut, for example. And so, you know, I'd rather have people have their view represented in polls than misrepresented by reporting on the issue that might, frankly, report the views of elites and not regular people. When you have reporters going up to New Hampshire or Iowa and saying here's the feeling I have on the ground, well, you know, that's interesting local color but it can't possibly do as well as actually opening up a phonebook and randomly sampling hundreds of voters in one of those states.

GROSS: What about the horse race polls, the polls that just measure who's ahead? Are those good for democracy?

SILVER: You know, journalists long before there were polls were always diagnosing the horse race, and they always got behind front-runners. And when you're gaining, it looks like everything is going well for you. When you're slipping, you know, they think you're toast. So if you're going to have horse race coverage at all, I'd rather you take some modicum of polling to do that, you know, because again, the alternatives are just totally the echo chamber where there's no grounding in public opinion at all. But certainly, I'd agree with Jill and other people that maybe things have gotten a little bit echo-chambery with respect to the polls where polls can beget other polls. They can produce momentum. People pile onto the bandwagon, then it reverses all the sudden. Obviously, no one is perfect.

At FiveThirtyEight, though, we try and say, hey, we're going to report on polls but remember number one, that polls are not always accurate. In fact, they're often inaccurate in the primaries and the caucuses. And number two, to say some awareness of, hey, we're aware at FiveThirtyEight that we're a site a lot of people read and that can affect the narrative so-called about the race too. So being aware of how the media is a player and how people understand the horse race and how they understand politics instead of speaking in kind of the voice of God or the third person is really important to us too. So after the debate, for example, in New Hampshire last week, our headline was we thought Marco Rubio did really bad, but we're not sure yet what New Hampshire voters will think. People criticized us for that afterwards. They said, well, shouldn't it have been obvious that voters would think the same way and thought Rubio did badly? But the point is having some separation between what you as a reporter think and what the population thinks and being - having some self-awareness of that is important to the way that we tend to look at polls.

GROSS: Well, Nate Silver, thank you so much for talking with us.

SILVER: Thank you.

GROSS: Nate Silver is the founder and editor in chief of the website FiveThirtyEight.

Transcripts are created on a rush deadline, and accuracy and availability may vary. This text may not be in its final form and may be updated or revised in the future. Please be aware that the authoritative record of Fresh Air interviews and reviews are the audio recordings of each segment.