Pollsters blow another one--big time

I know we're laying off reporters and editors but we gotta spend hundreds of thousands on these polls--Here, again, is another unreported story. Same one. The pre-election polls in South Carolina were flat out wrong. Every one of them. It was, believe it or not, an even worse performance than New Hampshire. So far the polls have been wrong in every single primary or caucus. Bet you didn't know that.

[The Washington Post wins the award for being the first to pick up on this, the Sunday before Super Tuesday. Click here.] And you won't want to miss Opus here.

But they predicted the winner in South Carolina, you say. Indeed they did. And they got the victory margin so wrong as to make their results uncredible. Some polls even missed Barak Obama's rout by almost 20 points. They were clearly clueless. If you are a serious pollster, that's an erroneous result even if you got Obama winning. Once again, taking polls for a primary is fraught with danger. Why the media relies on them is an interesting question.

(The exit polling, to the contrary, was excellent. You knew it was going to be a blow-out when CNN announced the winner within seconds of the polls closing. When I was doing polling for the Philadelphia Inquirer, we usually knew the winner by late afternoon from exit polling and had the story all written and ready to go for the second edition as soon as voting ended. For CNN to go that fast meant someone had run away with the vote--as indeed happened.)

For a good technical explanation of just how wrong the polls were in South Carolina, see Pollster.com here, and Mark Blumenthal here. They screwed up big-time. And again, why does anyone pay attention to this train wreck? When they call the winner wrong, as they did in New Hampshire, that is big news. If the get the winner right but blow the results, everyone ignores the fact they were doing junk science. 

The answer says more about journalism than about polling. Reporters are fascinated by the competition, the horse race, and have been widely criticized for it. In part, it is because a political campaign is a horse race, but that doesn't excuse the emphasis. People do want to know who is ahead and who isn't. If you didn't have polling you couldn't tell because anything else would be anecdotal. But, as we have seen, even having the polls doesn't really tell you much useful. Even when they work they are a snapshot of a moment in time and nothing more. Stuff happens. To carry it out more than a day in advance is to misuse polling and even one day can be hazardous to your health as New Hampshire proved.

Part of it is simple laziness. It is easier to report a poll result than to report the issues and analyze what the voters are thinking. Just ask a pollster and there is your lede. It's a lot easier than breaking down somone's health care plan and discussing that.  Reporters on the trail will blame their editors back in their offices for forcing them to emphasize the polls, and that may be true in many cases. Also, there you are out on an expense account and you really need to justify all that money they are spending on you so you have to file every day even if you have nothing to report. And, you don't want to get lost back in the office so you are very anxious to make sure you have a story in the paper or on the air every day even if there really is nothing new to report. Been there; done that. But just how many times do the polls have to screw up before you see the danger in that and find something else to do?

Primaries are famous for being hard to poll because it is notoriously difficult to predict who is going to show up. This year it is particularly difficult because the Democrats are revved up (thank you George Bush) and are turning out in huge numbers, far more than the pollsters are anticipating. That makes this year even more difficult for a pollster to handle. 

They should be ignored. The media should drop them and invest in reporters, who might actually get the story. Or not.

Oh, John McCain and Mitt Romney are now in a dead heat in Florida. That's what the polls say. That's what the media is reporting. Why, for heaven's sakes.

And now the customary political blather:

• Barak Obama is the most inspirational political candidate since Jack Kennedy. I hope he is well -protected. 
• Hillary Clinton is probably the only Democratic candidate who can lose to a Republican this year, particularly if that Republican is John McCain.
  
• Hillary needs to tell Bill to shut up.
  
• If the Clintons don't get off the race card, a lot of African-Americans are going to stay home next November and we could get another Republican in the White House.
• Mitt Romney is still evil.
  

Butt forward with talking heads--STILL BLOGGING

I am a pundit on television. See how important I am--Two important defeats have gone unmarked in last week's Iowa primary and notice should be taken. It happened again in New Hampshire, although this time it was so bad everyone noticed. The polls picked the right winner in Michigan but got the margin of victory wrong.

With amazement and glee:

• The polls in all states, with only one exception in Iowa, were absolutely wrong and predictably so.
  
• The pundits, with no exceptions, were absolutely wrong in both states and predictably so.

Every poll the night before the Iowa caucuses, except the one in the Des Moines Register, showed a tight race, with Hillary Clinton the most likely winner. As we all know, Barack Obama won by 8 percent, a substantial victory. All the polls in New Hampshire were flat wrong, predicting Obama by double digits. He lost by 2 percent. I said the failures were predictable because it is exceptionally difficult to predict caucuses and not much easier predicting primaries. The key is measuring who is likely to show up.  In both states, more people voted than the pollsters thought would vote. And the two populations differed: In Iowa, women went for Obama; in New Hampshire they went for Clinton. Go figure.

After Iowa, the "experts" were talking about Hillary dropping out. Even her own campaign believed it and the in-fighting and panic in her campaign team was loud and open. They were wrong too.

What happened?

Caucuses require time,  effort and mental energy and not everyone is likely to spend any of the above unless they really, really care. Pollsters have always had problems formulating questions that accurately predict who they will be. In the case of Iowa last week, they failed. Far more people showed up than the pollsters had thought might and they went for Obama. In New Hampshire, the opposite was true: the newcomers went for Clinton.

The exception in Iowa was the Register poll, run by Ann Selzer, who got it right on the mark.

Mark Blumenthal, who runs the indispensable Pollster.com, says it was not blind luck. Selzer had a system in place that worked and she stuck too it. (She runs her own company and works on contract to the newspaper). When her poll showed Obama winning handily, she was attacked by the campaigns of other candidates and by other pollsters. She stuck to her guns. Other polls, Zogby in particular was wildly wrong. It had Clinton winning with a fair margin. Polls were wrong four years ago as well. Makes you wonder why media that are busy laying off reporters for financial reasons will still spend a fortune on polls that are often wrong.

Here's Blumenthal's take on New Hampshire. The best explanation is that the polls stopped too soon--they did not pick up the last-minute arrival of Mrs. Clinton's sisters. He also challenges Zogby to defend his claim he spotted the Clinton trend but had to stop polling too early.

The pundits are another problem, Rant alert!

They live in Washington, which may be the capital of the United States but isn't like the real

America at all, and they assume knowledge they do not have, like what those of us outside the beltway are thinking or live like. They run in herds. It's not just the politicians; the journalists are just as bad. If I owned a newspaper I would not let any of my reporters stay in the Washington bureau for more than five years. After that, they get intellectually corrupted. The press in the case of this campaign has been wrong repeated and infuriatingly, yet they blabber on.

Remember when Hillary Clinton was inevitable? Remember when John McCain was dead? Isn't Huckabee a character in a Twain novel? Nobody can win without having more money than anyone else? Obama was dead. Then Clinton was dead. I could go on. When Selzer's poll in Iowa came out, the chattering class jumped on it as contrary to the common wisdom. It had to be wrong. Even the reporters traveling in the state with the candidates missed an emotional surge that played itself out Thursday in Iowa and then they overestimated the surge in New Hampshire. They are clueless.

New Hampshire should have been another case. Primaries also are difficult to poll, and the talking heads still don't know what they are talking about. Some of them gave up only reluctantly. As the race in New Hampshire continued to be remarkably close and as it looked like Clinton might actually win, the pundits were incredulous. At PBS, David Brooks, the conservative columnist of the New York Times kept saying "the numbers don't add up." This can't be real, he said. Oh really? He finally conceded around midnight.

There is one more factor, perhaps. Obama is black. That did not seem to matter in Iowa but it is possible--and I don't know if it happened--that voters lied to pollsters in New Hampshire about their willingness to vote for a black. The race card. I have no idea if that happened.

And, again, if you want to know what's really going on with the polls, see Blumenthal here.

Several political asides that have nothing to do with science:

• Obama's victory speech, though it went on too long, was a classic in American political history. He changed my mind. It apparently was less impressive to some in New Hampshire.
  
• The Republicans are really in bad shape when all they can produce is John McCain and the Seven Dwarfs.
  
• Mitt Romney is evil. Fortunately, the more people know him, the less likely they are to like him. He has already lost twice and keeps going because he has no character, and that's scary.
  

The picture above, by the way, is pollster John Zogby, probably not explaining why he charges so much money for his polls and gets them wrong.