Monday, November 28, 2016

Narrative stats versus evaluative stats

A few things crossed my path during my wanderings on the interwebs last week, and this tweet by Patrick Reusse on the yet-to-be-finalized Jason Castro contract is as good as any to get rolling:

Not to be snippy but .. yeah, it's been pretty obvious for a few years that Reusse, who was a darn good Twins beat reporter some 40 years ago, is out of touch with the advances in the game. He basically rejects any concept or stat that would have been unfamiliar to Gene Mauch and Earl Weaver.

In any other field, that would be silly. You'd be a terrible doctor if you knew nothing more than was known in the mid 70s, for example. In sports, cutting off your knowledge decades ago makes you "old school."

Joe Posnanski doesn't share Reusse's aversion to modernity. He had a series of blog posts this week examining the statistical differences between Rick Porcello (winner of the Cy Young Award) and Justin Verlander (who finshed second despite having more first place votes). The first post is here; the second, written after a response from Baseball Reference's Sean Forman, is here.

And in between came this piece by ESPN's Sam Miller on the same general topic: the ins-and-outs of Wins Above Replacement. Miller's piece, rather than being in the context of the Cy Young contest, is about Robbie Ray, who is either awesome or replaceable depending on which form of WAR one uses.

Read these pieces, and you might sympathize with Reusse. But we shouldn't.

The key question is: Why are we using the statistics?

You want to use stats to help tell the story of a game or the season? You want the traditional stats, what I think of as "narrative stats": wins and losses, RBIs, saves, etc, They get to the heart of what happened.

You want to use stats to help evaluate the talent and skills of a given player? You want the new age analytical stats, stuff you can't glean from the tables of the box score. They are designed for a different purpose, and they are necessarily less precise. And in some cases, such as Ray, they may not provide a clear answer.

The traditional stats are acts of accounting; the analytical stats are acts of interpretation.

If you are, as the Twins front office effectively did the past couple of weeks, deciding whether to invest $8 million in 2017 in Trevor Plouffe or Jason Castro, you're going to be more successful if you use the analytical stats (including pitch framing).

If you are, as the Cy Young voters were, trying to decide who had the better season between Porcello or Verlander, you're better off with the narrative stats. I have no problem with saying both that

  • Porcello had the better 2016 season and
  • Verlander is the more talented pitcher.

The analytical stats point you to Verlander. The narrative stats point you to Porcello, Different stats, different purposes. Nothing wrong with that.


  1. I would be extremely surprised if WAR had anything at all to do with the Twins FO deciding to sign Jason Castro. WAR and FIP as well, are stats invented by fans for other fans. They are built out of other stats and are basically used by fans to compare the value of different players, even those that don't play the same position.

    There is no set way to calculate WAR, and even the stats used to calculate WAR vary from different WAR to different WAR. You have to trust the maker of the particular WAR you are consulting, that he picked the "right" stats, that he weighted them correctly, that the positional adjustments make sense, and that his formula works the way he says it does.

    My guess is that while various internal analytics may have been used, mostly the decision to sign Castro probably came down to the following factors. Levine knows and likes him. He was about the best of the free agent catchers available who didn't carry with him extreme risk. The Twins FO don't believe Murphy or Garver are more than backups. Nobody in the Twins system is likely to be any better than Castro in at least the next 2 years. Acquiring a better catcher through trade would likely be way too expensive in prospects or money or both.

    I don't know how much analytics helped in those determinations, perhaps some. Mostly I suspect it was scouting and common sense.

  2. I make the same distinction but call them "analytic" and "predictive" -- I'm using "analytic" to refer the very accurate analysis possible with games already played.

    Predictive stats are... interesting. :)