Technology Could End This Industry

Thanks to reader A-Dave who sent along this NPR story about StatsMonkey, a program generated by Northwestern University which has the potential to computerize journalism. The program takes the input of a baseball game's boxscore and generates an article about the game - replete with stats, events, and even integrating quotations and pictures of the players.

Here's the project's website at the Intelligent Information Laboratory at Northwestern, where they explain how the program works.

The system is based on two underlying technologies. First, it uses baseball statistical models to figure out what the news is in the story: By analyzing changes in Win Probability and Game Scores, the system can pick out the key plays and players from any baseball game. Second, the system includes a library of narrative arcs that describe the main dynamics of baseball games (as well as many other competitions): Was it a come-from-behind win? Back-and-forth the whole way? Did one team jump out in front at the beginning and then sit on its lead? The system uses a decision tree to select the appropriate narrative arc. This then determines the main components of the game story and enables the system to put them together in a cohesive and compelling manner. The stories can be generated from the point of view of either team.

A few thoughts on this:

1. I'd never thought of using Win Probability to do this - to determine the most significant plays in the game as the plays which change the WP the most. For example, it seems that the program can determine that before Larry B's broken-bat single down the third base line, Stanford Hall had a 4% chance of winning the game, but after that, his team had a 10% chance of winning the game. Therefore, Larry B is more clutch than David Eckstein.

2. If the code for this project ever goes public, you can imagine the Turing tests that will take place on this blog. I'm sure the computer generated stories will be better than HatGuy's sentimental diatribes on why his favorite teams need to win more.

3. Doesn't the quality of the output of these articles depend on the phrases coded into the machine's algorithms? For example, if the folks at Northwestern hire Jay to help them come up with a set number of phrases that describe a come-from-behind home run in the bottom of the seventh, that means every article that comes out of this program is stamped with Jay's inanity.

4. Someone needs to save sports journalism. Someone needs to walk into the compounds of sports news departments all over the country and deliver the news:

Listen, sportswriters, and understand. That StatsMonkey is out there. It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. It knows the clutch situations in the game without even listening to the roar of the crowd and the crack of the bat. And it absolutely will not stop, ever, until your industry is dead.

I think this job falls to pnoles.

Yes This Is Self-Indulgent and Yes This Will Piss Off Pnoles

Ok, so Nick Swisher was traded to the Yankees yesterday for Wilson...um...Betemit and some minor minor leaguers. And I'm sure steam is shooting out Pnoles ears. However, I'm not here to defend or attack this trade on any level.

What I am here to do is attack some bad commentary by the (normally decent) Mark Gonzales.

By trading Nick Swisher, the White Sox cleared $22.05 million off their books over the next three seasons

Or, another way of looking at it is, they cleared like 5 million for 2009. That's, you know, probably a better way of looking at it, since, you know, this trade wasn't a salary dump as much as it was a player dump.

and slightly alleviated what general manager Ken Williams described as "having a plethora of outfielders."

I really don't get the snarkiness here. Between Quentin, Dye, Konerko, Thome and Swisher, the White Sox had, like, 3 corner outfielders and 2 1B/DH types. Swisher had prior to 2008, played like 80 games at CF in his five year career. He described himself as uncomfortable at this position.

So the White Sox, depending on how you looked at it, had 3 corner outfielders, all of whom, for various reasons, should be considered as "starting" corner OFers. Now they have two. You know, the number of starting corner OFers a team is allotted. I don't know how this only "slightly" alleviates a problem, or how that problem needs to be qualified as "something Kenny Williams is describing as extant."

The Swisher trade worked only for the first two weeks of the regular season and when he filled in at first base when Paul Konerko was on the disabled list because of a ribcage strain in late June.

Oh ok. Gotcha.

Swisher was by no means great for the White Sox, but he produced slightly below league-average offense as an everyday player for a team that made the playoffs.

But the trade only worked for, like, three weeks of the season. Because...why? He had a lousy batting average and struck out a lot?

But It wouldn't be fair to put the entire blame on Swisher because he was moved to center field and batted leadoff only because of Jerry Owens' groin injuries in spring training.

I don't even have words for this. I really don't. Is Gonzales really suggesting that an OF, left to right, of Swisher/Owens/Dye would have helped this team more than Quentin/Swisher/Dye?

Maybe I just don't understand what his point is.

It's ironic that Swisher and Orlando Cabrera, who is leaving for free agency, are gone after being brought to the Sox to provide production at the top of the batting order as well as a spark to a stoic group of veterans that lost 90 games in 2007.

Why is this ironic? Cabrera was a one-year rental, acquired mostly for the improved draft picks he'd yield if he refused arbitration.

Swisher's departure is unexpected and disappointing, being, as it was, the result of a disappointing 2008 season. But it's not "ironic" that a young player fizzles out despite being penciled in as a cornerstone of a franchise.

No. That's not ironic. What would be ironic is if Nick Swisher were called "The Dirty 30" because he was a clean-nosed player.

Or if he, like, had sex with his mother or something. Or if it rained on his wedding day.

Semantics aside, Gonzales is leading into this point, which I totally disagree with:

But this signals the start of a new era in Sox history, with at least one more veteran likely to be traded.

Not sure how resolving the glut of corner OFers means another veteran will be traded. Dye, for instance, might be traded. But there's no reason to assume Kenny's going to create a hole in RF. Because that's what it would be now. A hole.

Further, Konerko, as a 10/5 guy coming off a lousy year is damned near untradeable, and Thome is a 9 million dollar contract holder who can't do anything but DH for like 130 games out of the year. Who's going to take that.

Not that a trade can't happen. I'm just confused as to how the ironical development of a one year rental leaving town and a disappointing young OFer being shipped before he is untradeable signals this "new era."

Oh well. Fuck it. I'll leave you with some hard hitting journalism. Fuck it:

Swisher will have to adapt to the Yankees' strict policies regarding facial hair.