Follow-up for those who care: The limits of baseball statistics

Some follow-up thoughts from the previous post on the limits of baseball statistics, as emailed back-and-forth between trusty argument partner cannatar and myself (before this came out, incidentally). I think it follows logically and enhances the discussion, probably clarifies some things that were not clear in my post. This also has a nice cast of characters including Steve the old-school baseball guy, Mike the mathematician and Andrea the cook (she's a great cook). Also, there's a next-level article at Edge magazine here that discusses the general limits of statistics and has some overlap, but you're going to need a fairly advanced degree to read it.

From: cannatar
To: Bryan

"I'm also mystified at the premise that the 'better' team doesn't always win the championship, when in fact the point of the championship is to determine the best team."

When did you turn into Steve? [ed's note: very anti-sabermetrically-inclined fellow, tends to turn quite disagreeable to the point of yelling at the mention of VORP. Sometimes threatens physical violence. Am not kidding.]

Look, it depends on what you want the word "better" to mean. Does it just mean who did a better job during the 7 games?

Or does "better" mean something more - that one group of men is inherently more skilled at the game of baseball than the other? We use the word "better" all the time in life. You're better than me at stickball, Andrea is a better cook than me, Mike is better at math than both of us.

When I say you're better at tennis than me, I mean that in my opinion (based on years of observation), on any given day, you have a better than 50% chance of beating me in a game of tennis.

If I beat you 10 times in a row, I'd probably change my opinion. If I beat you in one game of tennis, I won't. Even if we call this game a "playoff game" and both agree that this match is extra-important and attempt to play with more focus and determination than we would on another day, the outcome isn't going to change my opinion that if we played again the following week in such a match, you'd be more likely to win. And that's sort of how I choose to think about what baseball team is better. I think the Rays are a better team than the White Sox (particularly since the White Sox don't have Carlos Quentin). But, I realize that baseball is a somewhat random game and that the inferior White Sox had about a 1 in 3 chance of winning the 5-game series. If they had won the series, that would've meant to me that they played better for 5 games, but it wouldn't have changed my opinion about who would win if they played another 5 game series. In other words, I would still think the Rays are "better". I assume Steve would think that if the White Sox won the series, they would win if they played it again the following week. And again, and again. In other words, I think Steve believes that the series reveals the inherent truth about which team is better.

Basically, I think the whole argument is circular. There are 30 baseball teams. The commissioner decides to come up with a construct to determine which team is the best at playing baseball in a given year. Then, whichever team wins under that construct is in fact the best. That requires us to make two assumptions: (1) it is possible to come up with a way to determine which team is better in the absolute sense (Sheehan and I don't think it is) and (2) MLB has actually come up with the best method for doing so.

Even if I presuppose that #1 is true, all I'm doing is accepting that if the White Sox beat the Rays, that means they're better at this point in time in a series made up of 5 games with two days off in between. These distincions are important - the Dodgers probably were actually a better team than the Cubs in October even though they were worse during the 162 game regular season. They were different teams at different points in time - Manny Ramirez and Rafael Furcal weren't around in June, but were in October. Carlos Zambrano and Rich Harden were healthy for most of the regular season, but not in October. The Red Sox and Angels both used a 5-man starting rotation during the regular season. The playoff structure allowed them to use 3-man rotations in the first round of the playoffs. "5-man rotation Red Sox" is essentially a different team than "3-man rotation Red Sox."

But, even if I accept that it's possible for MLB to come up with a system to determine which team is better at a given point in time, I think we'd both agree that MLB has not in fact come up with the best method to determine such a thing. They're running a business - their primary goal is to come up with a season and playoff schedule that makes the most money for MLB. I think they're concerned with fairness in determining a champion, but that's clearly secondary. Even assuming that the 162, 5, 7, 7 format is the best, it's clearly not the case that the system is designed so that the best 8 teams advance to the playoffs. If the goal was to determine who's the best, all 30 teams would play a balanced schedule and the 8 teams with the best records would advance to the playoffs. Instead, some teams play much harder schedules than others, and teams with inferior records make it to the playoffs because they happen to be grouped with other crappy teams. The 2006 Cardinals had the 13th best record in baseball. In addition, their 83-78 record was inflated by the fact that they played a disproportionate number of games against teams in their own division. Not only did their division contain only one other team with a winning record, it also contained the two teams with the worst records in the National League. They also played the vast majority of their games against other National League teams, which as a group are inferior to American League teams. I think there's a compelling argument that the Rangers (80-82), Indians and Mariners (both 78-84) played better than the Cardinals in 2006, they just had much harder schedules. So, there's at least a reasonable argument that the Cardinals were the 16th best team during the regular season, and there's not much of an argument that they were better than 13th.

From: Bryan
To: cannatar

I'm not Steve. Whatever I am, I'm not Steve. That is to say, I understand (and respect) the angle from which you approach this argument, which I did NOT understand, in all its facets, three years ago. The discussion I've started now proceeds from our previous, similar discussion. I understand the statistical undercurrent of your objection and begrudge your opinion in no way, shape or form (To that end, this is the first time I've written something stats-y without the sort of implicit knowledge that I was writing it to you; I was just writing about things that were bothering me. I appreciate your taking the time to read and respond).

That said, let's take the Andrea-as-a-cook example. Andrea is a better cook than me. What if Andrea was on Top Chef? The only goal of Top Chef is to win Top Chef, not to win Top Chef if it were played again, and again, and again. Now I agree that being the better chef coming into Top Chef will give you a better chance to win, the same way I think a team full of Albert Pujii would be better than a team full of Justin Mornii. But Top Chef exists so someone can win it, the same way Major League Baseball exists so that someone can win the championship. The same way there are hundreds of thousands of independent chefs around the country who are NOT on the show, if Major League Baseball did NOT exist, there would be a surfeit of independent leagues and teams and no real place for them to come together. MLB brings them together to determine who is best. That is the entire idea.

And so, I am taking the not-all-that-revolutionary step of saying that the team that wins the championship is the best team, given that is the sole goal of the entire season. You say that you and Sheehan do not believe that there is a way to determine who the best team is, yet in this very email you say that you think the Rays are better than the White Sox, and Sheehan argues quite forcefully that the Cardinals were not the best team in 2006. But hold on, if you do not believe there is a way to determine who the best team is, how can you determine who it's not? It seems, actually, that you have a very good idea of who you believe the best team is and are frustrated with/dismissive and ambivalent of results that do not conform thusly. Sheehan has no problem saying the Red Sox were the best team last year. So I think there's no question that what you've written is not exactly true.

Whether or not MLB has come up with the best method is up for debate, but what's not up for debate is that it is the method. Teams plan ahead at the beginning of the season because they know what the playoffs will look like. When things change during the year, like when the NBA moved to a 7-game first round series in the middle of the season, or, you know, our tennis matches, that's what I find fundmentally unfair. As you know. That ain't what we're discussing. It's just that winning short series is known to be part of winning the championship. Do I hate the 5-game series? You bet. But it's like a Supreme Court Decision — it's the law of the land until it's changed. The law of the land is that if you want to be named champion, you win a five game series. Better have some good starters.

And I don't buy the MLB-as-a-business aspect of your argument. Yes, I understand that MLB is a business and these are business decisions. But trying to separate any of this from the games seems like a fruitless and ultimately foolish pursuit.

In short, I feel like your arguments are mostly political — taking on the established law of the land, so to speak — and I actually agree with some of them. What I'm saying to Joe (and, I guess, you) is that it's fine to take on the orthodoxy but be careful about tearing to shreds people who are merely accepting the orthodoxy, mostly in light of the fact that the numbers do not describe anything in "the fog" that Bill James warned about that lies between axiom and theory. There is a point, it seems, when the numbers guys will throw up their hands and say, "Well, what happens, happens, we can't predict it all." I agree. I don't think numbers or words or observations alone or even paired can describe baseball; that's what makes it great. It takes ALL of them, and more than that. I don't want to disqualify any terms outright. And when it comes down to determining the "best" team, either we can — by the numbers OR the championship — or we can't. If we can't, I don't see the point in playing the games. Ask Herm Edwards why. If we can, I'll take what actually happens. It may not be perfect, but it DID happen, each of us only get so many of these championships in my lifetime. The champions deserve their due. The history of anything isn't perfect; we know only what we really need to know. We know the champions of baseball because it is, even in shorthand, the easy way to tell who the best team was in a given year. It's fighting for that right that makes the playoffs so dynamic, and what makes them the playoffs even if the numbers think it's just another game.

To me, at least.

From: cannatar
To: Bryan

I think most of our argument (and part of your issue with Sheehan) boils down to a definition of the word "best." I am using the word to mean something inherent about the person/team that can really only be "known" by God, but can be determined to some level of certainty by looking at data. You're using the word to mean who did the best at winning the World Series. My take on it is that we already have a word for that: "World Series Champion." Now, if you want to also call the Champion, "the best," you can, but I think you need to acknowledge that you're using the word "best" in a particular way and that other people are entitled to use it in a different way. Stephanie is the "Top Chef Season 4" champion, but I don't think we have any basis to declare her the "best" chef of the group.

"when it comes down to determining the "best" team, either we can — by the numbers OR the championship — or we can't. If we can't, I don't see the point in playing the games."

I guess part of the difference between us is that if the Mets won the World Series, I would still be just as happy even if I didn't think they were the best. It does seem that if I think it's all pretty random, I shouldn't care who wins, but I still care for some reason. I'm more puzzled by the fact that I care about the success of a random group of 25 guys who are employed by Fred Wilpon.

minor point:

"You say that you and Sheehan do not believe that there is a way to determine who the best team is, yet in this very email you say that you think the Rays are better than the White Sox,"

I don't think I contradicted myself. I THINK the Rays are better based on all the evidence, but I don't have a method to determine it with 100% certainty, and I don't believe such a method exists. They're close enough that I probably can't say it with even a very high percentage of certainty (in contrast, I think there's sufficient evidence to say that Albert Pujols is a better baseball player than Andy Phillips with at least 99% certainty, but there's a minute possibility that Pujols's career has just been random chance).

From: Bryan
To: cannatar

My main point is that it's more about the quality of ideas on any side — the numbers, observations, or words — that it is about the ideas themselves. I don't think this is a radical notion, and I don't think Joe Sheehan or anyone at BP would think so either. So if idiots use the terms "veteran leadership" wrongly, they're still idiots. If they use the term correctly, good. Same thing with determining what team is the "best." I think there's a compelling argument — obviously — that mysticism w/r/t unplayed, nonexistent sets of baseball games does not automatically outweigh the results of the actual World Series, no matter how good the numbers are w/r/t these magical unplayed series. I don't think it's a silly thing to argue about, though, and I think that many, many people do. I think this is the main problem. Of course, if you really think that the "best" team can only be known by God, maybe that supports my theory — I'm just going with what us humans have come up with. God's almost certainly smarter than me. I'm just doing the best I can here.