The limits of baseball statistics

I love baseball, I love baseball writing and I love baseball statistics, so I’ve been mulling a subject that has bothered me for a long time: the assault on some baseball terminology as propagated by Joe Sheehan at Baseball Prospectus.

Baseball Prospectus has revolutionized baseball’s record-keeping over the last few years — or at least popularized a long-simmering underground revolution. Put simply, baseball’s most cherished statistics — batting average, runs batted in, pitcher wins and losses — tell us a lot more about what happened in a game than they do make an accurate measure of a player’s contribution. It sounds tricky, but the best way to look at it is that the statistics that Baseball Prospectus compiles from games have a much better track record of predicting what will happen in the future (hence, “Prospectus”) for any given player than the “traditional” stats, which are measures of things that are often beyond a player’s control. That is, the total number of runs batted in a player will accrue during the season relies heavily on the quality of a player’s teammates, and a pitcher may win a game in which he gave up 10 runs and lose one in which he gave up 1. It doesn’t even take a baseball fan to divine the dubious value of such a statistic.

As the BP crew has grown in stature and number over the past few years — primarily since the publication of the 2002 book Moneyball, which highlighted the “newfangled” methods — it has been under nearly constant attack from baseball lifers and “purists,” who argue, basically, that the number-crunchers are a bunch of dweebs who long to make passionate love to their computers. It’s was unfair even before the BP numbers turned out, on a macro level, to help teams to such a degree that it’s pretty much accepted that they were right; the criticism from the old guard now is fairly passive/aggressive and limited to veteran announcers and writers who make claims that the “stats don’t always tell the story” and that there are intangibles involved with winning baseball games. The numbers guys don’t deal with intangibles. If it can’t be measured, it is not important.

This dichotomy rears its ugly head, as it were, every October during the baseball playoffs. Inevitably an announcer will make a comment that a team’s “veteran leadership” will prove decisive, or that their “heart” will lead them to victory. Just as inevitably, Sheehan will write a column excoriating the mouthpiece. Here is an excerpt from this year’s column:

Post-season baseball is just baseball with more media credentials and fewer games between flights. Pressure? There may be more, but is it any more than that faced when you're trying to get drafted? Make a team? Win a playoff spot? Does this week really feel more pressure-packed for the Brewers or White Sox than last week, every game a must-win game, did?

The stock storylines don't add anything to our enjoyment of the game. Whether it's "post-season experience" or "veteran leadership" or "pitching and defense" or "small ball," all these attempts to fit the postseason into boxes limit our knowledge rather than expand it. If we're going to break down these games, and figure out why players do well and poorly, why teams win and lose, let's wipe the slate clean and focus on what's happening on the field.

After years of struggling to figure out exactly what my problem is with this line of reasoning, I think I’ve finally found it: it renders words meaningless. Sheehan’s problem is not that these terms are misapplied but that they are applied at all. This gets to the heart of what Baseball Prospectus is all about: predicting the next set of numbers. There is one set of numbers, a game happens, and then there is a new set of numbers, both for the game itself and one that incorporates all previous games. The numbers do a fairly good job of predicting what the results will be on a macro level, but as Sheehan notes above in reference to the playoffs, post-season baseball is fundamentally no different than regular-season baseball; that is to say, and I’m sort of quoting from memory from hundreds of other articles that he’s written, there is nothing about post-season baseball that makes the numbers any less capricious than they are in May. That is to say, October baseball is subject to the same forces as any other game, with respect to creating new sets of data. You can predict what might happen, and be correct a good percentage of the time, but the game itself — the number of outs, the rules, etc. — is no different in the playoffs than it is in September, in that it’s damn near impossible to predict anything with certainty. The otherwise dogshit Cardinals won the World Series two years ago due to a strong October run. Here’s what Joe wrote to crown them:

Fans, and the less-critical corners of the media, are welcome to embrace the Cardinals and create storylines about raising their level of play and coming up big when it counted and grit and guts and what have you. It might ring more true if it wasn’t the standard storyline for every single team that wins a championship: they’re better people than the guys who lost.

I think he means “better baseball players,” but that’s not my point. Look at sentence fragment that talks about how writers will “create storylines about raising their level of play and coming up big when it counted and grit and guts and what have you.”

Do you notice anything wrong?

If you do, awesome.

If you don’t, let’s start from the beginning. Of baseball. Baseball is a human construct. Or at least we assume it is (ha!), not knowing its precise origins. (The Cooperstown moment is a myth, but one that will do. Like a lot of history.) But let’s just be clear: there’s nothing inherently special about baseball any more than there is anything inherently special about anything: any meaning it has is what we give it. The pre-season, 162-game schedule and post-season are completely arbitrary, save for the meaning we give it. The “championship of baseball” is a construct that, like the sport itself, has no inherent meaning whatsoever. I suspect Joe would agree with me on this, and why the numbers don’t play any different in the post-season than they do the regular season. The numbers don’t know it’s the playoffs.

But the numbers don’t play the game.

This is an incredibly important distinction that has been made many times, by many people, the only difference between them and myself being that they are usually trying to discredit BP’s stat-heavy mission. I am doing no such thing. I love the numbers. I play in a fantasy baseball league that is entirely situation-neutral numbers heavy — that is, the numbers which are BP’s bread-and-butter — and wouldn’t trade the numbers for anything. But there’s a reason that the numbers can only predict what will happen in a given game, series, or season, I dunno, 60 percent of the time (to randomly choose a fairly generous number) — the game is played by people. Or, as Billy Beane, master of the numbers, said in Moneyball, “My shit doesn’t work in the playoffs.” People play the game, and sometimes the favorites win, and sometimes they don’t, like the Cardinals in 2006. And the people, unlike the numbers, know it’s the championship. While Joe is perfectly fine with making his own, completely subjective value judgments on how much “pressure” playing the playoffs actually brings (Despite his distaste of subjectivity, remember: “There may be more, but is it any more than that faced when you're trying to get drafted? Make a team? Win a playoff spot?”), he a) intentionally overlooks the fact that the World Series is, by acclimation if not definition, the most important baseball played each year and thus likely subject to the most pressure; and b) follows it up with, “If we're going to break down these games, and figure out why players do well and poorly, why teams win and lose, let's wipe the slate clean and focus on what's happening on the field,” which has nothing to do with his anti-“veteran leadership” et al. screed. When people are talking about “postseason experience” and “veteran leadership,” this is exactly what they are trying to do.

The numbers, with their gap in accuracy between predicted results and actual results, don’t do the trick. Observation closes the gap. In the Cardinals/Tigers series, Sheehan talks about how the Cardinals got lucky that the Tigers made so many errors, and that the Tigers lost the title more than the Tigers won it. This is likely because the Tigers made such a shockingly high number of errors (seven, I believe) in a short series, and errors are largely unpredictable, so the random sequence of events — the errors — tilted the series toward the Cardinals. All this talk of randomness and capriciousness, which creeps up every year, viz:

I keep coming back to the central theme of any baseball postseason. The champion isn’t necessarily the best team, but it is almost always the team that plays the best in the short series of October. The Rays aren’t getting "lucky" in any sense other than they’re playing well when playing well has some excellent rewards. The Red Sox aren’t getting "unlucky," other than that they’re playing poorly at the same time. The Rays are playing better baseball, and thanks to that, they’re one win away from something that would have seemed preposterous to all but one man and his trusty CPU seven months ago.

Just for a quick side-trip, let’s look up “champion” in Webster’s. It’ll be important:

2. One who by defeating all rivals, has obtained an acknowledged supremacy in any branch of athletics or game of skill, and is ready to contend with any rival; as, the champion of England.

Getting back to the Tigers/Cardinals, the question Joe posed is not how the Tigers lost but why they lost, for the how is obvious — it was the errors. Neither Joe’s statistics or observations begin to provide the why he seeks. Isn’t that something? In fact, the only two sources of why are the injury report and the work of writers, who try to use the tools at their disposal (words) to describe why what happens, you know, happens. Words like “heart” and “passion” are perfectly applicable in baseball because if they are not they would cease to exist. They would be meaningless. As a quick exercise, think about your day right now for one second. How many things are going through your mind? Now imagine a sport that takes, at the least, 18 people to complete one game. How many processes, spoken or unspoken, would contribute to the outcome? It would have to be infinite, right?

I think it’s time that Joe and the other hardcore number-crunchers realize that we have created baseball, but the numbers merely describe the numbers, and nothing else. The words we use have meaning, so when we call a team a “champion,” they are the best team because we say they are. Everyone knew what they were playing for when the season began, and only one team achieved it. It happens because of great players, veteran leadership, tactical decisions, experience, and features from across the spectrum of what it means to be human, some of which are quantifiable, some of which are not. Sometimes the words are wrong (Your best bet would be to ask for examples of veteran leadership). But sometimes the numbers are wrong, too. We’re trying to describe what makes our champions our champions. If the “champion” is merely a construct and doesn’t mean anything, you shouldn’t care. If it does mean something, then you’ve admitted defeat. The words don’t predict or describe as well as precisely as the numbers, but that doesn’t mean they’re less important. Baseball is one of the most dynamic games ever created, but it's not one-one hundredth as dynamic as the human brain and human emotion. Champions are champions for a reason; we made up the word the same way we made up the game. Let us tell the stories of why the champions became who they are. We'll use the numbers and use words. It's an imperfect exercise. But we're trying.

UPDATE: Wow. This now exists. It's all there.

Also: Follow-up emails for those that are interested.