Opening the Bullpen Gate

The gates to the HoME are now open to relief pitchers. Who will be the first to enter?

Want to start a scissor fight? Put a baseball traditionalist and a sabrmetrical dude in a room and tell them to discuss the role of closer. Guaranteed fun and frolicking.

Everyone believes that relief pitching is important. Well, everyone who doesn’t think that complete games mark the mettle of a man. Not everyone, however, believes relief pitchers are important. With Hoyt Wilhelm on our 1978 ballot, we need to figure out how important they are.

Those who venerate the Closer’s special powers of closingness believe that only a select but initially unknowable group of relievers can muster the courage to get three outs in the ninth. Those who poo poo baseball’s fetish with the ninth inning insist instead that pretty much all relievers are failed starters and can’t possibly be valuable enough to get the attention they do since they pitch neither often nor in great volume.

Baseball as an industry has a mixed view. On one hand, Ruben Amaro thinks that closers are magical, and he’s paying Jonathan Papelbon $60 million to pitch about 280 innings over four years. On the other hand, Papelbon, at $15 million is the highest paid closer in the game and makes about half the top-paid players elsewhere on the diamond. Relievers have never, ever made anywhere near the money that starters and position players make. Despite its unwavering belief in the modern bullpen hierarchy and the importance of the ninth inning, the industry as a whole doesn’t pay premium prices for relief stars. It pays utility infielder prices for most non-closer relievers.

Baseball doesn’t pay premium prices because, for example, last year nearly half of teams changed closers…during the season. Including the World Champion Red Sox and most of the playoff teams. And isn’t Mariano Rivera the only reliever we talk about as being a core player on a championship team? You don’t build a team around Rollie Fingers. You build around Reggie Jackson or Robin Yount. You don’t really build around Rivera either. You build around Derek Jeter because failed starters and live arms, the starter dough of great relievers, are easy to come by.

What do the numbers say?

If you are intimately familiar with how WAR and WPA treat relievers, feel free to skip to the next section. Otherwise, a quick refresher.

Wins Above Replacement (WAR) supports the idea that relief pitching is more important than relief pitchers. The most valuable career in relief pitching belongs to Mariano, hands down. In fact, he may be the most effective pitcher inning for inning in major league history. (Well, let’s not get too carried away, his numbers are the best per inning, but when you only face three guys at a time, it’s not so cut-and-dried.) However, among modern players, his WAR total is 207^th all time, and about the same as Orel Hershiser, Johnny Damon, and Will Clark. The best reliever anyone’s ever seen, who lasted nineteen years and maintained amazing peak performance in nearly all those seasons, could only get up to the level of three guys no one considers among the top fifteen at their positions. How about Wilhelm? 330^th and near Frank Viola. Goose Gossage? Tied for 440^th and next to Davey Lopes. Fingers? He’s 971^st, right there with Greg Luzinski. Sutter isn’t even one of the 1,000 most valuable players in history by WAR.

Yeah, relievers.

On the other hand…leverage. The thing that’s true about the magic-ninth-inning theory is that the ninth frequently does represent higher leverage than other innings. Sometimes it turns out to be earlier, but most closers are leveraged effectively. Close-and-late situations magnify the importance of every run and out.

Another stat, Win Probability Added (WPA), gives a little credence to the closer-magic theory. WPA not only describes the performance and the quantity of the performance (as WAR does), it factors in the impact of each batter faced by a pitcher. With each change in batter, it measures the change in the odds that a pitcher’s team will win. Sum it all up over a season to see how many wins he was worth. WPA requires play-by-play data, so we can only assess pitchers from World War Two onwards. Conveniently, the ace reliever is mostly a modern innovation.

• Rivera is 3^rd all time in WPA, between Greg Maddux and Tom Seaver.
• Gossage is 24^th all time in WPA near Sandy Koufax.
• Wilhelm is 26^th all time in WPA, right next to Robin Roberts.
• Sutter is 77^th all time in WPA, right beside Chuck Finley.
• Fingers is 94^th near Frank Viola.

WAR goes a bit further in neutralizing a pitcher’s context than WPA does. WAR attempts to separate a pitcher’s performance from his defensive support and his opponents’ strength. WPA only adjusts for the run environment (park and league) and the game situation.

What if we could combine these two approaches?…

WAR and Peace?

Combining WAR and WPA would give us two cool pieces of information: the quality and quantity of a totally context-neutral performance (from WAR) plus its impact (from WPA). Here’s my quick-and-dirty way to do it.

First off, I make no guarantees. Second, I only authorize this for use with seasons in relief (as in really obviously in relief). Third, I make no guarantees.

Both WAR and WPA spit out a number denominated in wins. No fancy footwork needed to push them together. Wins are wins. A second helpful thing is that they measure the same performances to arrive at those wins. It’s like using binoculars instead of a telescope. If there’s overlap in their assessments, we can use it to tease out the differences and make them work for us.

So let’s make a big assumption and proceed from it:

WAR measures most everything we want to know, and WPA supplements it with improved leveraging information.

In other words, WAR + WPA’s leveraging value will give us a more complete picture of a reliever’s value and impact together.

What a relief!

As it happens, BBREF WAR measures leverage, but it does so with less precision than WPA. It measures only the initial game-state when a pitcher’s outing begins. So we want to get rid of that. What BBREF does is take the pitcher’s Wins Above Average (WAA), which is a step on the way to WAR, and adjust it for the initial leverage a pitcher faced. Take away that adjustment (labeled by the helpful Mr. Forman as WAAadj) from WAR, and you’ve got the unleveraged performance.

In the Win Probability section of a pitcher’s stats, BBREF shows not only his WPA, but also something called WPA/LI, which is the pitcher’s performance without leverage. See where this is going yet?

[ WAR – WAAadj ] + [ WPA – WPA/LI ] =

WAR without leverage value + isolated win value of WPA’s leveraging =

Estimate of a reliever’s total value

Let’s take an example from the Mariano Rivera collection.

In 2008, Rivera had perhaps his best season as a closer. His ERA+ was 316, and it was an astounding 57 points higher than his total batters faced. Think about that one for a moment. Rivera faced 259 hitters in 70.67 innings, or only 47 hitters above the minimum in 64 appearances. His K/BB ratio was 13:1 (77 Ks and 6 walks). He walked a batter every four weeks.

The key numbers for Rivera in 2008:

• 4.3 WAR
• 1.0 WAAadj

BBREF implies that Rivera was worth about 3.3 Wins sans leveraging.

• 4.5 WPA
• 2.6 WPA/LI

That’s 1.9 runs of more precisely leveraged value.

Easy peasy lemong squeezy!

[ 4.3 WAR – 1.0 WAAadj ] + [ 4.5 WPA – 2.6 WPA/LI ] = 3.3 + 1.9 = 5.2 wins

If you do this for every season in which Mariano was a reliever and add them back to his one year as a starter, you get 69.7 wins. That figure is comparable to Derek Jeter’s own BBREF WAR total of 71.6. Once you attempt a full accounting of his leverage, Rivera goes from The Thrill to The Captain.

Here’s every reliever with 30 or more Wins by this method. Does not include hitting. We’ll call this relief pitcher WAR or rpWAR. And no, virtually no one else gets the same boost that Rivera does:

1. Mariano Rivera (1 season as starter):  57 WAR; 70 rpWAR
2. Dennis Eckersley (half a career as starter): 62.5 WAR; 64.8 rpWAR
3. Goose Gossage (1 season as starter): 42 WAR; 51 rpWAR
4. Hoyt Wilhelm (3 seasons as starter): 38 WAR; 49 rpWAR
5. Tom Gordon (half a career as a starter): 35 WAR; 43 rpWAR
6. Stu Miller (5 seasons as starter): 27 WAR; 40 rpWAR
7. Joe Nathan (2 seasons as a starter)*: 26 WAR; 39 rpWAR
8. Trevor Hoffman: 28 WAR; 38 rpWAR
9. Billy Wagner: 28 WAR; 35 rpWAR
10. Lee Smith: 29 WAR; 34 rpWAR
11. Ellis Kinder (half a career as a starter): 31 WAR; 32 rpWAR
12. Tug McGraw (1 season as starter): 21 WAR; 31 rpWAR
13. tied: John Franco: 24 WAR; 30 rpWAR, John Hiller (1 season as starter): 30 WAR; 30 rpWAR, Dan Quisenberry: 25 WAR; 30 rpWAR
    *Active

Again, I don’t claim this is perfect, but it draws the very best relievers a little closer to their competition among all players in history by providing a more comprehensive view of the one thing they bring that other player’s don’t: intentional situational value. It also shows that the perception of reliever value being less than the  value of relieving is accurate.

Anyway, with the top relievers starting to arrive on our HoME ballots, this at least buys us some time to think up something better! I’d love to hear your comments on this and whether you think it passes the sniff test.

—Eric