Sunday, January 27, 2008

Pollsters blow another one--big time

I know we're laying off reporters and editors but we gotta spend hundreds of thousands on these polls--Here, again, is another unreported story. Same one. The pre-election polls in South Carolina were flat out wrong. Every one of them. It was, believe it or not, an even worse performance than New Hampshire. So far the polls have been wrong in every single primary or caucus. Bet you didn't know that.

[The Washington Post wins the award for being the first to pick up on this, the Sunday before Super Tuesday. Click here.] And you won't want to miss Opus here.

But they predicted the winner in South Carolina, you say. Indeed they did. And they got the victory margin so wrong as to make their results uncredible. Some polls even missed Barak Obama's rout by almost 20 points. They were clearly clueless. If you are a serious pollster, that's an erroneous result even if you got Obama winning. Once again, taking polls for a primary is fraught with danger. Why the media relies on them is an interesting question.

(The exit polling, to the contrary, was excellent. You knew it was going to be a blow-out when CNN announced the winner within seconds of the polls closing. When I was doing polling for the Philadelphia Inquirer, we usually knew the winner by late afternoon from exit polling and had the story all written and ready to go for the second edition as soon as voting ended. For CNN to go that fast meant someone had run away with the vote--as indeed happened.)

For a good technical explanation of just how wrong the polls were in South Carolina, see here, and Mark Blumenthal here. They screwed up big-time. And again, why does anyone pay attention to this train wreck? When they call the winner wrong, as they did in New Hampshire, that is big news. If the get the winner right but blow the results, everyone ignores the fact they were doing junk science. 

The answer says more about journalism than about polling. Reporters are fascinated by the competition, the horse race, and have been widely criticized for it. In part, it is because a political campaign is a horse race, but that doesn't excuse the emphasis. People do want to know who is ahead and who isn't. If you didn't have polling you couldn't tell because anything else would be anecdotal. But, as we have seen, even having the polls doesn't really tell you much useful. Even when they work they are a snapshot of a moment in time and nothing more. Stuff happens. To carry it out more than a day in advance is to misuse polling and even one day can be hazardous to your health as New Hampshire proved.

Part of it is simple laziness. It is easier to report a poll result than to report the issues and analyze what the voters are thinking. Just ask a pollster and there is your lede. It's a lot easier than breaking down somone's health care plan and discussing that.  Reporters on the trail will blame their editors back in their offices for forcing them to emphasize the polls, and that may be true in many cases. Also, there you are out on an expense account and you really need to justify all that money they are spending on you so you have to file every day even if you have nothing to report. And, you don't want to get lost back in the office so you are very anxious to make sure you have a story in the paper or on the air every day even if there really is nothing new to report. Been there; done that. But just how many times do the polls have to screw up before you see the danger in that and find something else to do?

Primaries are famous for being hard to poll because it is notoriously difficult to predict who is going to show up. This year it is particularly difficult because the Democrats are revved up (thank you George Bush) and are turning out in huge numbers, far more than the pollsters are anticipating. That makes this year even more difficult for a pollster to handle. 

They should be ignored. The media should drop them and invest in reporters, who might actually get the story. Or not.

Oh, John McCain and Mitt Romney are now in a dead heat in Florida. That's what the polls say. That's what the media is reporting. Why, for heaven's sakes.

And now the customary political blather:
  • Barak Obama is the most inspirational political candidate since Jack Kennedy. I hope he is well -protected. 
  • Hillary Clinton is probably the only Democratic candidate who can lose to a Republican this year, particularly if that Republican is John McCain.
  • Hillary needs to tell Bill to shut up.
  • If the Clintons don't get off the race card, a lot of African-Americans are going to stay home next November and we could get another Republican in the White House.
  • Mitt Romney is still evil.


Anonymous said...

The media LOVES to be wrong.
Upset/ Dark Horse (I pun not)/ Unexpected/ Come from behind/etc. These narratives offer the hourly news cycle something to report. "Who could've expected it?". "wow". In short - news!
The only killer is "thoughtful" mode where they have to discuss intricate policy and wonkish stats. So Kucinich and Ron Paul got killed off early.

CW said...

Lately, several organizations have been releasing polls with margins of error over seven points. That's enormous in the survey world. It smacks of polls designed to produce the specific result.


CW said...

I forgot to add a question -

Why do you consider Mitt Romney to be evil?


Anonymous said...

If the Clintons don't get off the race card...

That might be a bit like getting off the bear trap after they sprung it...

Joel Shurkin said...

The media doesn't "love to be wrong." They love to be right. What they love is conflict because it makes for a good story. The polling problem is that they are also lazy.

I consider Romney to be evil because he will say whatever needs to be said to get elected, depending on where he is and what he is running for. Running for governor in liberal Massachusetts? Be liberal. Running for president in a conservative party? Be a conservative. And, he is trying to buy the election. He knows that if he relies on outside sources like everyone else, he'd be screwed. The more people hear him, the less they like him.


sam said...

Thanks for all the info. More people need to be made aware of this GOOD information.The information is very meaningful to

whom needed. Interesting!! tHANKS !!

Visit Pulau Tioman Malaysia

Daro said...

(1st Anonymous here) Sorry Joel - you're right, no one loves being wrong per se. It might be better to say being wrong-footed is one element of drama. And they are lazy. It's the only way to explain why people I meet who work in the media seem perfectly decent and genuine but the MSM industry on the whole produces such a poor result. The same as a lazy student who genuinely says "I wanted to do well. I didn't want to disappoint my elders." But they were too lazy to get round to doing the study. Maybe one silver lining to this (unfortunately) coming recession/depression will be to put the grit back into real investigative reporting.