So I was driving to the ballpark yesterday to score the Rockies 4-3 walk-off victory over the Diamondbacks when on ESPN I heard an exchange between the host of the program and ESPN's Peter Pastorelli. As they were running down the chances of various teams making the playoffs Pastrorelli noted that it would be tough for the Red Sox because their pitching is in disarray but also in losing Jason Varitek to a cartiledge tear in his left knee they probably lost (paraphrasing) "a half run per game" because of his handling of the staff.
Really?
The idea that there is some skill in game calling that can depress ERAs has been around for a long time as one of the unproven assumptions of baseball. Several years ago Keith Woolner at BP did some research on the topic and wrote about it in The 1999 Baseball Prospectus. The end result of his research (confirmed by others) was that:
"There is no statistical evidence for a large game-calling ability, but that doesn’t preclude that a small ability. For example, a genuine game-calling ability that reduces a pitcher’s ERA by 0.01, resulting in a savings of about 1.6 runs per year for the entire team and could be masked by the statistical variance in the sample size we have to work with. Players would need to play thousands more games than they actually do to have enough data to successfully detect such a skill statistically."
So while there may be some skill involved, the natural variation overwhelms the signal the skill may be giving off which means that for all intents and purposes you may as well make decisions as if there were no skill operating at all.
The reason the myth persists IMHO is that Catcher's ERA (CERA) can easily be found (for example in each team's game notes published each day for the media and made available in the press box) and the inherent variation does indeed sometimes show that the staff performed better under catcher A than catcher B. This difference is misinterpreted by writers and even teams as meaningful when in fact there is no evidence that you would expect the difference to remain given another equally large sample size. If it were the case that you always saw little variation in CERA among a team's backstops it wouldn't have the allure it does. But that variation is a mirage.
This dovetails nicely with what Rany Jazayerli had to say about differences in hitting given small sample sizes quoted yesterday.
125x125_10off+copy.jpg)
4 comments:
dan - is the variation in cera really a mirage? rather than disprove the cera 'myth', doesn't the lack of data keith wonder speaks of just tell us that we don't know. and in these circumstances (when the data are inconclusive), what do we do? i see three basic choices.
1. we can say the phenomenon does not exist (but we may very well be wrong since we're saying we need a larger sample size).
2. we can say it may exist and we need to wait for enough evidence to produce a verdict (the bill james non-answer).
3. we can use circumstantial evidence and feel for now.
i think 3 is a perfectly valid way to go. so when making a decision, wouldn't assuming there is no skill at play at all be foolish? right now we're just saying we can't prove the skill exists. but that doesn't mean it doesn't. and when all else fails and we need to make a decision with imperfect information, wouldn't your gut and eyes tell you that your pitchers will probably perform better with varitek behind the plate(even if you can't *prove* that fact) than the guy with the higher cera?
Tom Hanrahan has done some research on this and has found that catchers do have an impact. It was published in "By the Numbers." Go to
http://www.philbirnbaum.com/btn2004-11.pdf#search=%22hanrahan%20%22once%20more%22%20birnbaum%22
Thanks for the link Cyril but I think this study is addressing a different topic. Pitchers may indeed perform better over a period of years if they have a stable environment (including the catcher) but that's a different question than whether there is a "game calling ability" that we can measure for catchers.
Even so I have my doubts that the methodology of Tom's study is granular enough since it is based only on seasonal data and does not therefore sort out when the catcher in question actually caught the pitchers in the study. The data set is also relatively small including only 90 pairs of pitchers and catchers.
For Anonymous:
I agree that 3 is a perfectly valid approach but it is bounded by what the data is telling you.
So in this case the conclusion would be, "yes there may be a skill there but if there is it makes such a small difference that it is only measurable after multiple years and so at the working level we can assume there is no skill"
In fact what the data is telling you is that just because Varitek may have a lower CERA this year does not mean he'll have one next year. The analysis tells you that there is a wide variation there. Now, you may choose Varitek because pitchers like to throw to him or because the team is more confident with him behind the plate but those are reasons you would have used despite his CERA not because of it.
Post a Comment