Wednesday, July 8, 2015
Monday, June 2, 2014
Statistics, Racism, and Stereotyping
This article was originally published in ESPN the Mag in late March. It's an amazing conglomeration of foolishness. It was written by Jeff Phillips, a principal at The Parthenon Group, which is some kind of business consulting firm, so perhaps this isn't exactly sportswriting. But if ESPN the Mag is going to publish it, it qualifies.
FOR 150 YEARS, "clubhouse chemistry" has been impossible to quantify.
Have people been trying to quantify it for 150 years? I don't think there was anyone back in the early days of the American League trying to figure out some way to quantify the clubhouse chemistry. Plus, it would've been totally different back then. All the players were white guys from the good old USA, so the model would probably have to measure things like "willingness to pound moonshine or "ability to cover for me when I need an alibi for the manager's bed checks" or "sleeping ability on a train". I imagine that old time GMs probably had different models for chemistry than their current counterparts.
It's a good thing all the other baseball teams have the RED SOX around to show them examples of good chemistry. God knows the RED SOX are the most charismatic organization playing America's most charismatic sport. This is what makes everyone hate Boston's sports team.
I also find it interesting that the authors don't present the scores of the '04 or '13 Red Sox on their scale. This would've been a really good place to show that their super-duper chemistry metric matches the eye test of the obviously chemically superb Red Sox. But there's no mention.
Until now.
EMPHATIC SENTENCE. Jeff Phillips is following the advice of his 10th grade English teacher, Mrs. Higgenbotham, who reminded him to use syntax to reflect meaning. Way to go, Mrs. Higgenbotham!
Working with group dynamics experts Katerina Bezrukova, an assistant professor at Santa Clara, and Chester Spell, an associate professor at Rutgers, we built a proprietary team-chemistry regression model.
Jeff Phillips probably pitched his article something like this:
Jeff Phillips: We've got an educated baseball audience hungry for some new insights on baseball, and they seem to love statistics. But we also need to appeal to the good old boys who understand clubhouse chemistry. So we're trying to figure out how to quantify this. This is gonna be a killer article!
Katerina Berzrukova: Listen, I'm busy working on getting tenure, and working with you clowns isn't exactly going to boost my academic resume. Plus I am Russian. Call me if you want to quantify the chemistry on your gymnastics team.
Chester Spell: Yeah, publishing in ESPN the Mag is not exactly a peer-reviewed journal. If I'm trying to build my resume, I probably don't want to get published by the same organization that has employed both John Kruk and Joe Morgan as a baseball analyst.
JP: Wait, did I mention I'm writing an article for the World Wide Leader in Sports? The Biggest Sports Journalism Outlet around? You can get a lot of hits on our website.
[KB looks at CS]
CS: So?
KB: Yeah, so? Nobody reads it in Russia.
JP: Also we can pay you handsomely because our coffers are overflowing with money. I'll just grab a few grand from the Bristol Money Bin, where Simmons and the ESPN execs just swim in cash like Uncle Scrooge.
KB: Now that you mention it, I've got a PhD from Moscow State University, where I helped the vaunted Moscow State Dragoons hockey team run off a four-year undefeated streak by concocting a clubhouse chemistry model based on their loyalty to the motherland and their vodka tolerance. So this stuff is my wheelhouse.
CS: Well, I do a lot of research about organizations and businesses, and Major League Baseball is an organization and a business!
JP: Great! I'll throw some bills at you, you throw some random numbers and regressionspeak back at me, we bang out a few thousand words for the Mag, and we call it a day. Whew! I'm gonna hit the pool.
*************
Plus the model is proprietary. Which basically means that article could've been random numbers thrown out of nowhere. That might work if I thought ESPN had the ethos of being honest and trustworthy and devoted to making a contribution to how we understand the complex game of baseball... hahaha, I kill me.
Our algorithm combines three factors -- clubhouse demographics, trait isolation and stratification of performance to pay -- to discover how well MLB teams concoct positive chemistry.
It's a good thing that human relationships in the complex world of a baseball clubhouse can be reduced to three indicators. And it I'm still not buying the premise that MLB teams are trying to "concoct" positive chemistry. Sure, I bet now and again a team makes a decision perhaps not entirely based on baseball performance (the Reds did spend a fair amount of coin for an aging Scott Rolen in 2009 probably in part because of his clubhouse presence), but that particular acquisition wouldn't have done much at all for these threefactors.
Also, the word choice here is putrid. "Algorithm"? "Concoct"? C'mon, Mrs. Higgenbotham! I know you wanted to teach Jeff as much vocabulary as you possibly could, but you have to also teach him how to use it appropriately!
According to the regression model, teams that maximize these factors can produce a four-win swing during a season.
So by overhauling their entire 25-man roster in order to manipulate the overall chemistry of the team, they might be able to get four wins out of it. Sounds like a plan! Actually, maybe they should manipulate the overall quality of their baseball players instead.
According to our projections, such bonding will clinch this year's World Series: The Rays (third in chem) will prevail over the Cardinals (28th) in six games for Tampa Bay's first Series title. Happy players, happy fans.
At least the Cardinals suck at chemistry. I've heard all sorts of bullshit about the Cardinal Way over the years, so I'm glad to see that the BFIB won't latch on to this as evidence of their moral and personal superiority to the rest of the National League. But this model is still stupid. And if chemistry accounts for 4 wins over 162 games, its effect in a 7 game series must be miniscule. So I don't think it makes any sense at all to over-reach and use it to predict the winners of the World Series.
Demographic factor: The impact from diversity, measured by age, tenure with the team, nationality, race and position. Teams with the highest scores have several overlapping groups based on shared traits and experiences.
Does this mean teams should try to avoid racial integration? Or including foreigners? Or only bring them in if they're in groups? This is such an awful way to look at any kind of group construction - instead of actually working with your players, teaching them not to be assholes to people from other races and countries, and generally working to improve the clubhouse chemistry of the best players they can assemble, it suggests instead that GMs should view their human resources as fixed identities. What a terrible way to view human beings, and what a terrible way to generate organizational culture.
Also, why the hell is position in this factor? Are we really looking at the chemistry impact of the few positional decisions a team makes? OH SHIT IF WE CARRY A THIRD CATCHER THAT TIPS OUR CHEMISTRY FACTOR. CMON UP CORKY!
Isolation factor: The impact from players who are isolated because of a lack of subgroups from these shared demographic traits. Too much diversity can, in fact, produce clubhouse isolation for players who don't have teammates with similar backgrounds or experiences.
If they're assholes, yes, then you'll have a team full of players who don't want to talk to their teammates because they have different backgrounds. God forbid we have TOO MUCH DIVERSITY. I guess that's why the 1927 Yankees were so awesome. They didn't have too much diversity.
Maybe they're thinking of a team like the 1975 Reds - three Latinos, three black guys and too white guys - as their ideal team. That way, every player has his own buddy within his own racial group. I'd be interested to see how they treat race, for example. Perez was Cuban, Concepcion Venezuelan, and Geronimo Dominican; would they they have meaningful shared traits? As a Reds fan I am ashamed that I don't know the answer to that question. As a baseball fan I'm interested to see how they actually categorize demographic traits. Oh wait, this is an article written to appeal to the hordes of fans who just want to throw numbers around without actually thinking about how they were constructed. Shame.
Ego factor: The impact from individuals' differences in performance and monetary status. Too few All-Stars and highly paid players signal a lack of leadership; too many, however, creates conflict. The ideal level falls in the middle.
Actually, too few All-Stars signals a lack of good players. Too many All-Stars signals, generally, a good team. Leaving out the obvious point about All-Star selection being a popularity contest, it's still a stupid-ass thing to say that the IDEAL LEVEL of All Stars is "in the middle". Maybe in Russia an All-Star means something else.
Chris W. once reminded me that Bill James once said that if you find a stat that says some nobody was a better hitter than Babe Ruth, you should probably find a different stat. In this case, if you find a model that says a team should not try to have a maximum number of all star players, you should probably find a different model. Or at least a different proxy variable for "performance". All-Star appearances is a a terrible variable proxy for performance. The current leader in the AL shortstop voting is Derek Jeter, who has the 12th best WAR among AL shortstops.You'd think that with the tons of possible ways to quantify performance , the professors would pick something sensible.
Their point about the possible chemistry effects of income inequality in the clubhouse seems relevant. I can see how that might cause problems in a clubhouse. I'd actually like to see the information on that. BUT NO: the model is proprietary.
The article goes on to discuss every division and how chemistry will affect the standings. I'm not going to go through every division because it's boring, but here's a sample:
The Tigers' clubhouse diversity contributes 0.9 of a win by itself, but the strength of its subgroups is weak due to differences in age and years with the team, leading to an Isolation score of minus-1.6 wins. The Royals' Isolation score, on the other hand, is 4.7 wins higher, boosted by a pitching staff that includes eight Americans among the team's 10 core pitchers.
This is pure junk. We're supposed to draw a meaningful conclusion from the fact that the Royals, are doing their best to keep American jobs at home instead of outsourcing them like those anti-American Pirates? Or that Dave Dombrowski's careful assembling of Tigers' 25-man clubhouse diversity is almost worth one win? It's just so stupid.
Man, that article was bad. Faux academic knowledge? Statistics misused? Terrible writing? Subtly promoting racial stereotyping? Nice work, ESPN The Mag.
Thursday, March 1, 2012
As entertaining as the comment war going on in the next post down is
Tuesday, September 6, 2011
WMTMQR: There shall be no TMQR
What a bummer. How could this be? The guy writes an entire column centered around making predictions for all 32 teams in haiku format--yes, haiku format--and I can't find anything in it really worth picking on. Seems ridiculous but it's the case. I dunno, maybe I'm going soft in my old age. Maybe Gregg has written moronic drivel intentionally all these years and decided to cut his act this week. Maybe all this hoping to one day study law is giving me some kind of a mental block. Whatever the reason, Gregg gets a pass this week. I'm sure he'll be back to his old tricks next Tuesday, speculating that Team X kicked a 48 yard field goal on 4th and 12 that made the score 24-3 early in the third quarter because Team X's coach was trying to avoid having a shutout on his resume.
Arguments for: Leads NL with 18 wins, instrumental in bringing baseball relief to desert.
Arguments against: Haven't we learned enough from sabermetrics that pitchers just can't, ahem, control wins very much? And Kennedy's other numbers don't come close to stacking up.
AL MVP
Jacoby Ellsbury, Red Sox
Arguments against: Ellsbury is 27 runs behind Curtis Granderson.
Curtis Granderson, Yankees
Arguments: Granderson leads the majors with 125 runs scored, 27 more than the No. 2 run-scorer.Tuesday, June 14, 2011
You bet your sweet bippy I'm willing to bitch about FanGraphs
Why not? Sure, it's just a blog (that also maintains a ton of statistical information- here I'm just talking about their analysis). But we pick on Deadspin because Deadspin is a shit factory; and in its own way, FG is just as much of a shit factory. More importantly FG holds itself out to be a source for serious, objective baseball analysis. So I don't feel bad at all as I engage in this episode of blog-on-blog crime.
I "discovered" FanGraphs (by which I mean "was linked to" or "was told about"- don't you hate it when people try to act like they actually discovered some well-known site or blog before the general public? Like they got in on the ground floor or something? GRRRRRRRRRRR) sometime back in 2008. My initial reaction was "Wow! This is great! A whole blog dedicated to numbers-based analysis!" Within two or three weeks that had changed to "Wow! This is insane! What a pathetic collection of smartest dipshit in the room writers and sycophantic pathetic commenters! How could a concept this great be executed this poorly!"
It's sad and true. Despite offering itself as a high-minded source of objectivity, in reality it's often simply a bunch of pick-and-choose-which-stat-helps-your-predetermined-conclusion garbage. Frequently its analysis is no more objective that the kind of crap Dayn Perry or Scott Miller peddle. If a hitter the FG staff approves of (usually a Mariner, a Ray, or a backup catcher with a high OBP who is SO UNDERRATED IT'S CRIMINAL! THIS GUY WOULD BE A 5 WAR PLAYER IF THEY'D JUST LET HIM PLAY EVERY DAY! (Note: player does not play every day for a perfectly good reason, such as an obscene platoon split problem or the fact that he spent last year putting up slightly above average numbers as a 26 year old in AA ball)) is struggling, it's because of an "unsustainably low" BABIP or HR/FB or whatever. And if those metrics are down because that player's LD% is down and their GB% is up compared to their career numbers, no worries, it will certainly normalize itself in due time and you'd be a fool to think otherwise.
BUT- if a player who is not one of their chosen favorites is struggling, it's a whole new set of rules. The reduced BABIP or HR/FB which is a function of a tanking LD% is not going to change anytime soon. After all, the LD% was "unsustainably high" in the first place. People who say otherwise are selling snake oil. And of course, apply appropriate numbers and logic for pitchers FG loves/doesn't love. Long story short, the whole site is a celebration of the lies/damn lies/statistics thing. Decide what you want your conclusion to be, then build an argument around it based on cherry-picked stats. Just because batted ball splits tell us more about a player's skill level than RBI or batting average doesn't mean they can't be misused just as egregiously. Effecting the egregious misuse in question just takes a little more skill.
In any case, there's no more well-known or hilarious example of FG's groupthink stupidity than the 2010 "organizational rankings" which saw the Mariners, who just happen to be the team of choice for managing editor Dave Cameron, come in ranked shockingly high (#6org!) despite a crummy MLB team and a very questionable farm system. But there were TOTALLY LEGITIMATE AND OBJECTIVE REASONS behind that ranking- just ask any of those smartest dipshit in the room writers or sycophant commenters. Until June or July 2010, anyways, when the Mariners were on their way to a very foreseeable 101 loss season and everyone associated with the site started to get really annoyed with any commenters who made light of the whole situation.
But I'm not writing this post to generally dump on FG. I'm writing it to point out two atrocious posts they've published in the last two days. To be fair, neither is of the lies/damn lies/statistics ilk I was just talking about (although I'll keep an eye out for the next one of those they put out). Nope, each of these is flat out awful for different reasons. I think both would fall nicely under the label of "articles that didn't need to be written."
First, we have this peach, in which the author spends 5 paragraphs and 600ish words expressing the very controversial position that Paul Konerko is ABSOLUTELY NOT a hall of famer. No fucking shit, of course. But what really serves to douse the whole thing with dumbsauce is the reason the post was written in the first place: because White Sox manager Ozzie Guillen said that Konerko was a HOFer. So can anyone raise their hands and tell me why Ozzie might have made that argument? And further explain why saying this in response:
Over his career, Konerko has been worth only 26.7 WAR. Again, a far cry from Hall of Fame level. Just a quick glance at his stats reveals a good, but not exceptional, player. If that’s the case, why is Ozzie so convinced Konerko will make it into the Hall?
is kind of like using a hammer to open a can of soda? It's a completely ridiculous and pointless method for dealing with the situation. Ozzie is sticking up for his player (a conclusion deftly reached by the author in the post's final paragraph). A dumb Sox fan who thinks Konerko is bound for Cooperstown because he gave Jerry Reinsdorf the damn ball after the 2005 World Series isn't going to listen to someone talking about WAR. And you certainly don't need to explain that Konerko is not a HOFer to anyone who reads FG. Overall just a useless fucking article that takes the mindless bluster of one of the game's great mindless blusterers and turns it into an excuse to blandly state the completely fucking obvious. Awesome.
Better yet we have this, which is mind-bogglingly dumb enough to have rolled off the ESPN.com press.
What a tough question. What are the top rivalries in baseball?
Who gives a shit? Is there some kind of prize for the two teams involved? Is there any way to get past the insane subjectivity involved in this discussion? And in case I forgot to mention this, who gives a shit?
I’m sure Yankees fans will claim it’s Yanks-Boston,
I realize saying this makes me a hypocrite (to pick nits, I'm saying what's NOT the best rivalry in baseball rather than what IS), but this is almost certainly the most overrated rivalry in sports. It has negligible historical roots and is largely fueled by the fuckwits in the MLB front offices pushing it on everyone every goddamn second of every season. Anyways.
but if you went out on the West Coast, I bet you’d hear plenty of fans saying Giants-Dodgers deserves more consideration.
Actually, I bet most fans out here probably would say that it's Red Sox-Yankees anyways because they've been told over and over and over and over and over again for the past seven or eight years that that's the case. See above. Thanks Bud Selig! You make Roger Godell look downright sensible sometimes.
How exactly do you declare one rivalry “bigger” than another?
You can't, in any meaningful way.
How do you measure fan excitement, and compare one fanbase against another? Is it possible?
NO. Leave writing this article to Jayson Stark. Walk away right now while you still can.
Probably not, but regardless, I’m going to take a stab at it anyway. My methodology is very simple: I’m ranking rivalries based on the amount of Google hits returned for the search “(Team name) (Team name) rivalry”.
Your head should be buried in your hands right now. Tears optional. I'm not going to tell you what the results are, although they're fairly predictable. The point is this: FanGraphs is fucking hole. It's a disastrous combination of misguided analytical elitism and worthless poopstain articles like the two linked here. Fangraphs upsets me just as upset (if not more) as tales of Simmons and J-Hench's AL East-only 17 team keeper league.
Monday, January 18, 2010
Technology Could End This Industry

Thanks to reader A-Dave who sent along this NPR story about StatsMonkey, a program generated by Northwestern University which has the potential to computerize journalism. The program takes the input of a baseball game's boxscore and generates an article about the game - replete with stats, events, and even integrating quotations and pictures of the players.
Here's the project's website at the Intelligent Information Laboratory at Northwestern, where they explain how the program works.
The system is based on two underlying technologies. First, it uses baseball statistical models to figure out what the news is in the story: By analyzing changes in Win Probability and Game Scores, the system can pick out the key plays and players from any baseball game. Second, the system includes a library of narrative arcs that describe the main dynamics of baseball games (as well as many other competitions): Was it a come-from-behind win? Back-and-forth the whole way? Did one team jump out in front at the beginning and then sit on its lead? The system uses a decision tree to select the appropriate narrative arc. This then determines the main components of the game story and enables the system to put them together in a cohesive and compelling manner. The stories can be generated from the point of view of either team.A few thoughts on this:
1. I'd never thought of using Win Probability to do this - to determine the most significant plays in the game as the plays which change the WP the most. For example, it seems that the program can determine that before Larry B's broken-bat single down the third base line, Stanford Hall had a 4% chance of winning the game, but after that, his team had a 10% chance of winning the game. Therefore, Larry B is more clutch than David Eckstein.
2. If the code for this project ever goes public, you can imagine the Turing tests that will take place on this blog. I'm sure the computer generated stories will be better than HatGuy's sentimental diatribes on why his favorite teams need to win more.
3. Doesn't the quality of the output of these articles depend on the phrases coded into the machine's algorithms? For example, if the folks at Northwestern hire Jay to help them come up with a set number of phrases that describe a come-from-behind home run in the bottom of the seventh, that means every article that comes out of this program is stamped with Jay's inanity.
4. Someone needs to save sports journalism. Someone needs to walk into the compounds of sports news departments all over the country and deliver the news:
Listen, sportswriters, and understand. That StatsMonkey is out there. It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. It knows the clutch situations in the game without even listening to the roar of the crowd and the crack of the bat. And it absolutely will not stop, ever, until your industry is dead.I think this job falls to pnoles.
Thursday, July 30, 2009
Execs Love Stats, Thinking; Managers Hate Stats, Thinking
Reading Tim Kurkihjidsian's latest article (it's pretty good) about the magic number of the 100-pitch start, I came across an interesting paragraph that seems to corroborate the postulate I suggested in my last post about Jeter:
There is an growing discord in baseball between the managers and players in an organization and the management of an organization over the use of numbers/statistics/quantifiers to make baseball decisions.
Consider this paragraph, which attributes the slow but significant decline in pitches per start to:
Today's young general manager
Twenty years ago, nearly 90 percent of all GMs had played in the major leagues. Now there are three out of 30: Philadelphia's Ruben Amaro Jr., the White Sox's Kenny Williams and Billy Beane of the A's.An interesting stat, though it's not like any of those players had major league careers of major significance.
This decade has brought a new breed of GM, one who is highly educated, can run a spreadsheet and has mountains of data to support his theories.
It certainly has, suggesting that the Moneyball revolution has really shaken the management of the game to its core.
"We have a new wave of general managers who are deeply into mathematics, analysis, metrics -- I'm not saying it's wrong -- because that's what they charted in the minor leagues," said Rays pitching coach Jim Hickey.
I'm not really sure what he means here. Does he mean the GM started charting these things in the minor leagues of being a GM?
"I don't know the numbers, but the new wave of GMs are the ones who have charted that the chance of injury is, say, greater at 85 pitches than it is at 75. And with every five-pitch increment, there's a 22.8 percent more likely chance that someone gets hurt. With each 10 extra pitches, it goes up by five percent."
Interesting. So it seems like a basic calculation: after X number of pitches, you get a diminishing return on your investment in a starting pitcher. Obviously a pitcher isn't much more likely to get hurt between 50 and 55 pitches, but if a pitcher's chance of going on the DL rises significantly after, say, 105 pitches... is it really worth leaving them in for another inning when you have a reliever who can throw that inning almost as well?
The new GMs sometimes clash with the old-school manager about how the club should be run. Often, the GM wins.
I sure wish Wayne Krivsky would clash with Dusty Baker about having the player with the lowest VORP of any starting major leaguer hit leadoff and get the most at-bats for the Cincinnati Reds. Literally! He's 865th in the majors in VORP! Where is the stat-conscious GM when I need him?
"My GM used to load reams and reams of paper on my desk about that night's game," one former manager said. "Sometimes I'd read it; sometimes I just throw it in the trash.
... awesome.
But in the end, if it comes down to him or me, he's usually going to win. And if the discussion is about pitch counts, he is always going to win."
It seems as though the disconnection between the front office's love of player statistics and players' and managers' distaste for them is only going to increase in coming years. I wonder if there will be real consequence from it.