How to Completely Misunderstand the Use of Sabermetrics
A novel by Jonah Lehrer
The Math Problem
Sabermetrics can help teams identify hidden talent and turn regular sports fans into math nerds. But can the numbers lie?
No, not by themselves. They need the help of idiots to do that.
I have to paste this whole awful analogy. Bear with me here fellas.
Buying a car is a hard decision. There are just so many variables to think about. We've got to inspect the interior and analyze the engine, and research the reliability of the brand. And then, once we've amassed all these facts, we've got to compare different models.
How do we sift through this excess of information? When consumers are debating car alternatives, studies show that they tend to focus on variables they can quantify, such as horsepower and fuel economy. (Psychologists refer to this as the "anchoring effect," since we anchor our decision to a number.) We do this for predictable reasons. The amount of horsepower directly reflects the output of the engine, and the engine seems like something that should matter. (Nobody wants an underpowered car.) We also don't want to spend all our money at the gas station, which is why we get obsessed with very slight differences in miles per gallon ratings.
Furthermore, these numerical attributes are easy to compare across cars: All we have to do is glance at the digits and see which model performs the best. And so a difficult choice becomes a simple math problem.
Long and short of it: Jonah thinks that evaluating an athlete based solely on statistics can be misleading, much like relying on "metrics" like horsepower and fuel economy for buying a vehicle.
Unfortunately, this obsession with horsepower and fuel economy turns out to be a big mistake. The explanation is simple: The variables don't matter nearly as much as we think.1 Just look at horsepower: When a team of economists analyzed the features that are closely related to lifetime car satisfaction, the power of the engine was near the bottom of the list. (Fuel economy was only slightly higher.) That's because the typical driver rarely requires 300 horses or a turbocharged V-8. Although we like to imagine ourselves as Steve McQueen, accelerating into the curves, we actually spend most of our driving time stuck in traffic, idling at an intersection on the way to the supermarket.
Dude. You could not have picked something that more PROVES the necessity of sabermetrics. Here's the analogy you set up on accident. Substitute batting average and pitcher wins for horsepower and fuel economy. Stuff you might think on the surface is important, but when the economists (sabermetricians) analyze what really matters, it's surprising stuff like.....
This is why, according to surveys of car owners, the factors that are most important turn out to be things like the soundness of the car frame, the comfort of the front seats and the aesthetics of the dashboard. These variables are harder to quantify, of course. But that doesn't mean they don't matter.
And that is the point of sabermetrics. To identify which elements of a baseball player (car) are actually important, weed out those that aren't, and use that information to value them properly.
I truly apologize to my many, many fans for that incredibly dry chunk of text. But I think it's important to show how clueless this guy is before we get into the crap. His car analogy far more easily lends itself to explaining why sabermetrics are important than proving that they're overly relied upon.
But this is not a column about cars. My worry is that sports teams are starting to suffer from a version of the horsepower mistake. Like a confused car shopper, they are seeking out the safety of math, trying to make extremely complicated personnel decisions by fixating on statistics.
There is not a single sports franchise in the world, not even the Oakland Athletics, that is this hell-bent on ignoring non-numerical data. You clearly have no idea how sports teams operate (hey, a label!).
Instead of accepting the inherent mystery of athletic talent — or at least taking those intangibles into account — they are pretending that the numbers explain everything.
Who??? Who is doing this? Didn't Milton Bradley just get cut? Doesn't Mark Kotsay still play professional baseball for some reason?
And so we end up with teams that are like the worst kind of car. They look good on paper — so much horsepower! — but they fail to satisfy. The dashboard is ugly, the frame squeaks, and the front seats make our ass hurt.
Examples please. These are just general statements with no proof. Were the Houston Astros, Cleveland Cavaliers, Buffalo Bills, and whoever the fuck is bad in the NHL good teams on paper?
This is largely the fault of sabermetrics.
No no no no no no no. Wrong wrong wrong wrong wrong wrong. You need to provide me with one fucking example to get me to even listen to you.
Although the tool was designed to deal with the independent interactions of pitchers and batters, it's now being widely applied to team sports, such as football and basketball.
First of all, sabermetrics is not a word applicable to any sport besides baseball. So stop using it. It comes from the acronym for the Society for American Baseball Research. Stop fucking telling me that people are analyzing football with sabermetrics, because unless xFIP is somehow helpful in evaluating a wide receiver, you're making no sense.
I am now going to assume that instead of "sabermetrics", Jonah instead means to say "big scary numbers that I don't understand".
The goal of these new equations is to parse the complexity of people playing together, finding ways to measure quarterbacks while disregarding the quality of their offensive line, or assessing a point guard while discounting the poor shooting of his teammates.
Yes, and that's a smart thing to do. An intelligent GM in any sport considers the context of a player's ostensible success or failure and tries to evaluate how he would perform if extracted from his environment and placed in that of the team he runs. Why is this a problem?
The underlying assumption is that a team is just the sum of its players, and that the real world works a lot like a fantasy league.
No one, not even Ed Wade, is this stupid.
In many respects, sabermetrics has dramatically improved personnel decisions. By relying on unusual measurements of performance, such as base runs and plus-minus ratings, teams have been able to identify neglected talent, whom they can sign on the cheap. Sabermetrics has also helped sports executives double-check their instincts. Instead of blindly trusting some errant whim — and thus making a terrible trade or picking the wrong free agent — they can consult the math.
Correct.
And now we come to the paragraph where I became 100% convinced that Jonah is clueless, rather than someone bad at arguing.
If the Giants had trusted the numbers, for instance, they wouldn't be saddled with Aaron Rowand's five-year, $60 million contract. (He batted .230 last year.)
I'd think this is a pretty good point if not for the fact that Jonah clearly thinks he's pegged Rowand for useless (and duh, he is, but still...) because of his batting average in a single season of his deal. That, after all, is a sabermetrically sabermetric statistic that will convince your argument's opposition (sabermetricians) that you know what you are talking about.
They would have realized that his OBS
I don't know what that is.
and OPS is pretty mediocre, especially once his two outlier seasons are taken into account.
Rule: If John Kruk knows what it is, it's not a sabermetric statistic.
(Edit: Jonah since crossed out "OBS" since my initial read of this article. But on the other hand..."OBS"??@?!?!?!??!#?@#?R#$!!!#!$!???????????????? It's not a typo, of "OPS", because he distinctly wrote "OBS and OPS".)
But sabermetrics comes with an important drawback. Because it translates sports into a list of statistics, the tool can also lead coaches and executives to neglect those variables that can't be quantified. They become so obsessed with the power of base runs that they undervalue the importance of not being an asshole, or having playoff experience, or listening to the coach.
Incidentally....that Rowand guy......
[x] Not an asshole
[x] Has playoff experience
[x] Coachable
If Jonah Lehrer was the GM of a baseball team, he'd start Derrek Lee at first base and David Eckstein everywhere else.
This is the moral of the Dallas Mavericks.
Uh oh. You don't want to do that Jonah.
By nearly every statistical measure, the Mavs were outmanned by most of their playoff opponents.
Pretty tough to debate this. Few people could seriously argue they were favorites on paper against the Lakers or even the Heat.
(According to one statistical analysis, the Los Angeles Lakers had four of the top five players in the series. The Miami Heat had three of the top four.)
Wow. Yeah let's only look at the top 5 players on each team to prove a point. There isn't an NBA fan in the world that doesn't understand that the Miami Heat are largely pathetic once you drop below the Bosh node in the pecking order.
And yet, the Mavs managed to do what the best teams always do: They became more than the sum of their parts. They beat the talent.
They beat the Lakers and Heat in 7-game serieseses. The *best* teams always win a 7-game series!
Consider the case of J.J. Barea. During the regular season, the backup point guard had perfectly ordinary statistics, averaging 9.5 ppg and shooting 44 percent from the field. His plus/minus rating was slightly negative. There was no reason to expect big things from such a little player in the playoffs.
But CONTRARY TO THOSE EXPECTATIONS......he averaged 8.9 ppg and shot 42 percent from the field in the playoffs.
And yet, by Game 4 of the NBA Finals, Barea was in the starting lineup. (This promotion came despite the fact that he began the Finals with a 5-for-23 shooting slump and a minus-14 rating.)
Primary reason: DeShawn Stevenson was playing like absolute shit. Let's not mention that though.
What Dallas coach Rick Carlisle wisely realized is that Barea possessed something that couldn't be captured in a scorecard, that his speed and energy were virtues even when he missed his layups (and he missed a lot of layups),
Not helping your case.
and that when he made those driving floaters their value exceeded the point score.
Please please please please Jonah, if you could take a moment to re-read that sentence....
Because nothing messes with your head like seeing a guy that short score in the lane.
Wow, there you have it. This is why LeBron James played like crap in the 4th quarter. J.J. Barea's floaters were just ::sniff:: too demoralizing! This is why Earl Boykins has won so many championships!
Although Barea's statistics still look pretty ordinary — his scoring average fell in the Finals despite the fact that he started — the Mavs have declared that re-signing him is a priority. Because it doesn't matter what the numbers say. Barea won games.
Wow. Big slap in the face to Dirk Nowitzki and Jason Terry, who could not have withstood the tremendous 4th quarter play of LeBron James without J.J. Barea's floaters that caused Chris Bosh to start openly weeping after they went in.
What follows that paragraph is a depressing and poorly-placed story.
I'm thinking here of a Philip Roth metaphor. When asked by David Remnick, in a 2000 New Yorker profile, how he felt about a cramped literary interpretation of one of his novels, Roth busted out a sports analogy. He imagined going to a baseball game with a little boy for the very first time. The kid doesn't understand what's happening on the field, and so his dad tells him to watch the scoreboard, to keep track of all the changing numbers. When the boy gets home someone asks him if he had fun at the game:
"It was great!" he says. "The scoreboard changed thirty-two times and Daddy said last game it changed only fourteen times and the home team last time changed more times than the other team. It was really great! We had hot dogs and we stood up at one point to stretch and we went home."
If that little kid were around today, he'd be obsessed with sabermetrics.
"Scoreboard changes" being one of the pivotal cornerstones of advanced baseball metrics.
He'd almost certainly win his fantasy league, but he'd miss the point of the game.
Tsk tsk. If only you left out that part about the hot dogs and the 7th inning stretch, you might have a point. Sabermetricians have never attended baseball games, and if they did, they surely wouldn't waste their time eating hot dogs or stretching. WHAT IF THEY MISSED A SCOREBOARD CHANGE!?!??!?!
Sure, he wouldn't have squandered center field on Rowand, but he also wouldn't have started Barea or bet on the Mavs.
Are you telling me that you, Jonah Lehrer, thought the Mavs would win the NBA Finals at the outset of the playoffs? They were a 16:1 shot. Not impossible for them to win, but there isn't ANYONE who knows shit about the NBA who would have bet on the Mavs at the onset of the playoffs if betting on all teams payed out equally.
His car would have way too much horsepower and shitty seats.
Stop with this crap already. It doesn't make sense.
For reasons that remain mysterious, some teammates make each other much better and some backup point guards really piss off Ron Artest.
Well since the GM of Ron Artest's team has no idea whether a player he's going to trade for will piss off Ron Artest or not, he might as well just not even bother trying to evaluate anyone, right?
These are the qualities that often determine wins and losses, and yet they can't be found on the back of a trading card or translated into a short list of clever equations.
Seriously, no shit. Obviously it matters in basketball if players play well together, accept their roles, and complement each others' skillsets. You'd have to be a total asshat to think otherwise. The issue is that no one knows which players specifically fit best with Deron Williams before they actually play on the same court together. GMs do the best they can with the information they have at hand, and most of that information is statistics and scouting reports. It's pointless speculating who's going to become best friends with your fucking point guard. I've had it with this. Your Mavericks argument is shit, because it's based 100% in hindsight. Your definition of a "winning player", in essence, is "a player who won something". Try arguing to me that J.J. Barea is by definition a "winner" if LeBron goes off for double-digit points in every 4th quarter of the Finals like he's more than capable of and the Mavs lose. Because that's essentially foresight that you're holding sports teams to be responsible for knowing and acting upon rather than the data they actually have.
This is the paradox of sports statistics: What the math ends up teaching us that is that sports are not a math problem.
Putrid. Horrid. That sentence makes me want them to free Mariotti.
