Monday, January 12, 2009

Thoughts on the Predictions of Leading Online Sports' Websites "Experts

Exhibit A:
ESPN NFL Football Expert Picks for the Wild-Card Games
ESPN NFL Football Expert Picks for the Divisional Playoffs

Interesting items:

  • For the wild-card game between San Diego and Indianapolis, nine of the ten ESPN experts incorrectly chose the Colts to win.
  • For the wild-card game between Baltimore and Miami, all ten of the ESPN experts correctly chose the Ravens' win.
  • For the divisional playoff game between Carolina and Arizona, all ten of the ESPN experts incorrectly predicted the Panthers (who, incidentally, got their ass beat).
  • For the divisional playoff game between San Diego and Pittsburgh, nine of the ten ESPN experts correctly picked the Steelers over the Chargers.
Conclusions:
For these four games, the experts' unanimous and near-unanimous choices went a whopping ... 2-2.

Exhibit B:
Sportsline.com's Experts' Picks for the College Football Bowl Season

Interesting items:
  • On January 2, three bowl games were played, and the Sportsline experts went a combined 0-15 - not one of the five correctly predicted even so much as one Utah, Kentucky and Ole Miss wins.
  • Hawaii, Nevada, Boston College and Central Michigan, all unanimously predicted by the experts to win, managed to lose - and UH in blowout fashion.
  • Six teams who were unanimously chosen to win their games managed to actually win their games.
Conclusions:
Of all the bowls in which the experts chose a winner unanimously, the experts' choice went a rousing... 6-7.

The Moral Of The Story

Logically, you'd think that the games for which there was real consensus in picking a winner would represent a high chance for the experts to be correct. That's assuming that these "experts" are all independently evaluating the football teams and then predicting which team will win. The consensus, in this case, would represent a series of analyses that all come to the same conclusion.

Rather, I think these (admittedly small) samples suggest that there's a real tendency towards groupthink in these predictions. It's the sort of groupthink that is suggested by shit like the ESPN Bottom Line - which relentlessly reminded viewers that "Carolina is 8-0 at home, and Arizona is 0-5 when playing in the Eastern Time Zone" going into Saturday night's game - as though that would have a drastic impact on the game. Turns out, in an actual contest of American-style football, the time zone doesn't actually make much of a difference, and nobody bothered to mention that the teams played in the Eastern Time Zone were actually pretty good, including a 4-point loss to the Panthers. It's not like they played the Bengals or the Lions.

Or the same kind of groupthink that suggests universally that Notre Dame (they've lost a million straight bowls and didn't win road games this year!) or Ole Miss (Texas Tech is rated like really high! They scored a lot of touchdowns!) didn't merit even one choice of the five.

Actually, I suspect that these unanimous or near-unanimous decisions are rather the result of one analysis that gets passed around. Are there any statistical metrics which might have suggested that Arizona would win last Saturday (I mean, besides the fact that they only lost by four last time they played there)? I bet there are.

15 comments:

Jack M said...

Ignorance is Strength

Lou said...

If every analyst independently comes to the conclusion that team A has a 51% chance of winning the game over team B, you will see a unanimous selection of team A. I think it would be much better to at least let them rank their choices, or give projected scores, something to say that "team A's chances vs B are better than C's vs D, although I would pick both of them."

Chris W said...

Dan--

Interesting thought. I've had the same thought myself.

I wonder to what extent it's "groupthink" based around buzzwords or to what extent it's a symptom of the sorts of people they get to be analysts.

If you catch my meaning.

For instance, in 2006's playoffs, the Bears played the Saints in the NFC Championship game. Every single solitary ESPN analyst picked the Saints in a game that

a.) The Bears won easily

b.) Unlike the NFC Divisional games, they should have had the entirety of their faculties devoted to predicting (i.e. when the Panthers and Cardinals met, that was considered the least interesting playoff game of the weekend by most people)

Why would everyone pick the Saints to not get butt-pounded by the Bears, even though, by and large, the Bears were a much better team than the Saints in nearly every aspect of the game in 2006?

Well the possibility is that there was groupthink about momentum and offensive playmakers--that people passed around the idea of NEW ORLEANS and KATRINA and REGGIE BUSH and DREW BREES and SUPER ROOKIE MARQUES COLSTOM and repeated it ad nauseum until it became assumed fact.

OR maybe ESPN just happens to employ the kind of douchebags who think "momentum" and "the city behind you" is more important than the fact that the Bears had a superior running game and infinitely superior defense to the Saints AND were playing at home against a warm-weather dome team on a mucked up field. Because those are the kind of douchebags who appeal to the types of douchebags who ESPN caters to.

You know what I mean?

Angelo said...

thought this was hilarious: I followed those links and the first comment on the bowl picks website caught my eye.
"When will the media start giving the SEC some love. Let's face it, if the PAC 10, or Big 12, or even the Big 10 had won the BCS championship three years running, you'd never hear the end of it."

Passive Voice said...

As to bowl predictions:
Good fucking luck picking bowl games. Teams have like a month off. Players might sit around eating over the holidays. Alternatively, players who were out for parts of the regular season can get healthy. Coaches can spend way, way more time than normal preparing. By and large, teams travel decent distances, sit around in an unknown city, and may or may not really give a shit about the game. Lots of times, the teams are similar in talent, and usually don't have more than like one or two common opponents. Throw in the randomness inherent in any game and: good luck.

All that said, they unanimously picked Nevada? Can't defend that.

dan-bob said...

@LouinNJ

You're right - but my argument depends on the axoim that the closer that number gets to 50%, the more likely you are to have variance in what a panel of "experts" might say. Sure - if the panel had to predict the outcome of the Cards/Panthers game, you would expect much more variance in the choices than, say, if the Cardinals were facing the Little Sisters of the Poor.

@Chris W

I do know what you mean - and I think it's two sides to the same problem.

(a) the "buzz" around certain teams gets to deafening levels because of the lack of analysis that takes place on most sports networks - the groupthink is a substitute for actual thought, even when the "experts" actually played football.

and

(b) the "experts'" predictions themselves are influenced by the buzz their colleagues are generating. Even smart analysts are subject to these influences, but you'd hope they'd be able to see past that.

dan-bob said...

@ Jack:

Want to guest-lecture in my class when we discuss that in two weeks?

Bengoodfella said...

I think it is pretty obvious the experts are not actually experts. They are just guessing like the rest of us. I was amazed that the experts universally picked the Panthers to beat the Cardinals AND Vegas gave them 9.5 points. The exact same teams played at the exact same location two months ago and the Panthers won by 4.

In the bowl games, I saw the "experts" were mostly picking Texas Tech over Ol' Miss and Alabama over Utah. Of course the results were opposite. I don't think the experts do any more research than you or I do when it comes to predicting the winner of a game. You would think since they place themselves as experts they would know something we don't, but the truth is they know just as much as we do.

Unknown said...

I don't mind so much when most of teh experts end up wrong when the reasoning is well thought out. I can see most people picking the Panthers over the Cards (At 9.5 though, that was too much), who knew Jake was going to play so badly? The part that bothers me is that the experts rarely go "I think the Panthers should win, but it's going to be a very close game, remember, the Panthers squeaked by these Cards the first time." When predicting, I have to always use the idea of "All things being equal" which can't account for suddenly crappy games by a QB, or a punt bouncing off the head of a player who is blocking.

What is appalling is that you get near unanimous choices in situations like the Colts/Chargers game, that just seemed like bandwagonning. The Colts won more games then teh Chargers, but I never thought that they were in anyway beter then them. They have a recent history of playing poorly against them, and for 9-10 to choose the Colts is a joke. It was a pick 'em kind of game, and I would have thought half would have gone for the Chargers. That is when the "I made a good pick, yes you did, I made the same one" back slapping reinforcement of groupthink really rears it's ugly head.

dan-bob said...

@Bengoodfella:

That's exactly how I feel, and perhaps you're just a more placid person than me, but it makes me rage inside.

Anonymous said...

I remember the 2006 MLB Playoffs, when ESPN's panel of experts unanimously picked the Yankees to beat the Tigers in the first round. Out of something like 19 "experts", not one thought the Tigers had a chance. True, the Tigers had played poorly down the stretch that year, but they still had a much better pitching staff than New York, which is probably why they ended up winning that series. Of course, what's ESPN without a little East Coast bias?

Bengoodfella said...

I am not a placid person at all, it causes me great grief and pain inside. I want to yell, "At least pretend you know what the fuck you are talking about" almost any time I watch ESPN.

I don't expect anyone to be perfect but you hire these people because they are supposed to know something the average fan does not...and they don't! I guess I just need to accept this fact and leave it alone but I struggle doing that. I just think if you are on television and being paid to talk about sports, you should be able to pick a game better than me.

This past week it drove me crazy seeing the picks for the playoff games where everyone was picking the Giants and Panthers. Did they not watch the previous games the Giants-Eagles and Panthers-Cardinals played? It was very close. How can it be unanimous? Same thing with Ol' Miss and Utah, no one gave them a chance. Ol' Miss beat Florida (the same one playing in the National Championship) in Florida, what more do you need to know? The groupthink may be more fun to engage in but you can't hold yourself as an expert and then be average at analyzing games.

Unknown said...

Remember guys, we are also far better informed then fans from even 15 years ago were. We had to rely on guys like this for all of our information. Now we can get raw statistical data, other folks have created systems for analyzing them in a rational fashion. I think that unlike your typical fan or apparently, ESPN media members or Peter King, we can step back and go "I'm favoring the Giants, but really, the Eagles played them tough and I wouldn't be surprised at all to see them win." I mean hell, this is the NFL, even the Rams won a couple games this year, and they looked worse then the Lions in most of their games, allegedly.

Edward said...

Bengoodfella has an excellent observation that the "experts" are not really experts. Usually, they are just former players; and while players know how to play the game, and to analyze specific player-on-player matchups better than us fans, they don't necessarily know how to analyze it at the macro level. Hence, take their predictions with a grain of salt.

However, the Cardinals upset of Carolina is not a great example. What percent of objective fans would have told you the Cardinals were a better team? 5%? 1%? Throw in Carolina's home field advantage and that they were 8-0 at home, and you could think that any given person would have a 95% of picking Carolina to win.

Now, when you only survey 10 people as your "experts", the chances of everyone picking Carolina would be (.95)^10, or just under 60%. Therefore, you would expect all 10 "experts" pick a 95% favorite the majority of the time.

You could argue that 95% is a very high probability, and it is; however, "experts" tend to see things in more exaggerated terms, so a true 65% favorite could be morphed in to a 75% favorite.

Or I'm no better than these "experts" and I'm just talking out of my ass too.

Tonus said...

Ben: "I think it is pretty obvious the experts are not actually experts. They are just guessing like the rest of us."

I think that it's because, as much as many analysts hate the idea, luck and happenstance play a really big part in any sports contest. An expert can't guess that your kicker makes that tackle on a return that would've been a TD otherwise, and then a tipped pass leads to an interception on the next play. Or that your normally-reliable left tackle suddenly has an off-day and gets beaten for three sacks. Or that your all-pro cornerback trips and falls on a critical third and long. Etc etc.

I think most picks are based on a pretty understandable criteria-- which team is better, and are there any obvious factors (injuries, weather) that might affect the outcome? Aside from that, there's a whole lot of luck involved. So you're right, the experts are probably making picks the same way we do, and their track records shows it.