The value of a good starting goalie: a market calculation

Eric T.
October 11 2012 08:11AM

Mike Smith
Raise your hand if you saw this one coming
Photo by Mathew Cerasoli from the United States via Wikimedia Commons

A few days ago, a commenter had this to say:

Teams are allocating more $$ to goaltending than ever before. I'm fairly certain the only way teams have below $4M allocated to goaltending in the future will be if a goalie is on an ELC.

In 08/09, 3 teams had $7+M allocated to goaltending. Next season, 11 teams will have that amount. That's a significant increase in 4 years. In 12/13, there will be approx 15+ teams with $7+M allocated to goaltending. Why? Cause GMs are placing more importance on it, therefore placing more $$ on it. We saw in the last 2 summers that teams have to give goalies big contracts to retain them.

Let's use that as a jumping off point to talk about goalie contracts. I disagree with the claim that teams are spending more on goaltending, but perhaps more importantly, I will question a common assertion in the statistical community that teams should skimp on spending for goalies.

Rising salaries or flawed analysis?

First, let's look at whether spending on goaltending is really going up. The original claim was based on a threshold measure -- looking at how many teams were above some specific threshold at some specific point in time.

Threshold-based analysis is quite common, because it is easy to put together and describe. However, it can also be quite misleading, since the outcome often depends on exactly where you place the threshold. "I don't trust stats because you can make them say whatever you want" is a common rejoinder that is in most cases lazy and unfair, but it is often a valid concern with threshold-based analysis.

At times the convenience of a threshold is unavoidable, but the burden should then be on the person producing the stats to show that the outcome is not dependent on the choice of threshold. Here, the commenter hasn't done that, so I am immediately suspicious. Let's dig a little deeper.

Unfortunately, it seems that even the initial numbers are inaccurate. The commenter says only three teams spent $7M on goaltending in 2008-09, but I count four (the Rangers, Blackhawks, Canucks, and Ducks). He says 11 teams will spend $7M on goaltending in 2012-13, but I only see ten -- or nine, if you don't count Tim Thomas' cap hit as being allocated to goaltending.

That's not a good start, but let's move past that, since it only changes the magnitude and not the trend. The bigger problem is that if you move his thresholds a little bit, the trend changes completely. Suppose instead of setting the threshold at $7M, we asked how many teams spent $6M on goaltending, then here's how things look: 

Year Teams over $6M
2008-09 11
2009-10 12
2010-11 12
2011-12 11
2012-13 13 (incl BOS)

Or if instead we look at the individual level, 10 goalies made at least $5M in 2008-09, and 10 will make at least $5M in 2012-13. I don't think you'd look at any of that and conclude that GMs were placing more importance on goalies or allocating more money there.

Of course, the best way to approach this is to get away from the threshold analysis altogether. Here's what the average spend was for teams' #1 goalies (by cap hit, not necessarily usage) and top two goalies:

Year Top goalie Top two goalies % of cap spent on G
2008-09 $4.1M $5.3M 9.4%
2009-10 $4.1M $5.3M 9.4%
2010-11 $3.8M $5.0M 8.4%
2011-12 $3.8M $5.0M 7.7%
2012-13 $4.1M $5.5M 7.8%

It seems pretty clear that money allocated to goaltending has actually been pretty flat, and has therefore been a declining fraction of teams' salary cap. But we still have that second question to answer: what should goalies be paid nowadays?

How much is a goalie worth?

When I compare goalies across different teams, I prefer to use even strength save percentage so that a goalie won't look worse as a result of his teammates taking a lot of penalties. And because we're trying to distinguish minute differences between goaltenders, tenths of a percent, we need multiple years of data to try to assess a goaltender's talent level. Here is how career save percentages are distributed across the 42 active goalies with at least 150 starts:

The distribution of save percentages across the league

The top-end guys like Pekka Rinne and Henrik Lundqvist are right at .928. The 30th best established goalie was .918, and that list doesn't include a dozen or so younger goalies who have given some indication of an ability to perform at or above that level (Tuukka Rask, Cory Schneider, Anders Lindback, James Reimer, Semyon Varlamov, Braden Holtby, Josh Harding, etc). Add those players to the established goalies, and it becomes reasonable to suggest that a decent backup goalie is somewhere in the neighborhood of .918 these days.

That means we should expect Rinne and Lundqvist to save about 1% more shots than a reasonable backup. However, this estimate of their talent is more likely to be a bit high than low. Rinne benefits by a couple of tenths of a percent from playing for a team whose scorer overcounts shots; his career road save percentage is 1.3% lower than his home save percentage, a gap almost a full percent bigger than the average goalie. In addition, both goalies play for teams with a strong focus on defense -- team effects on save percentage are very small, but it's possible it contributes another one or two tenths of a percent here.

So it is reasonable to adjust the assessment downwards slightly to account for those factors. Let's say their actual innate skill results in stopping about 0.8% extra shots, overall; it's a guess, but something in that range seems reasonable. These elite goalies might play 70 games and face 1900-2000 shots per season, so we might expect them to save an extra 15-16 goals per season. That translates to about two and a half wins, and using Hawerchuk's figure that the UFA market pays $3M per win, that would make Rinne and Lundqvist worth about $7-8M per year. They're each getting about $7M, so this actually seems pretty reasonable, easily within the errors of our estimates.

As an aside, Driving Play came to a different conclusion about the likelihood of Rinne earning his salary. There are two things different here: 1) their regression had each win being worth about $2M, which meant Rinne needed more wins to justify his pay, and 2) he increased his workload this year -- they concluded that he couldn't do it without starting about 72 games per year, and then he started precisely 72.

The same calculus works as we move down the scale a bit. Ilya Bryzgalov or Kari Lehtonen at around 65 games and a .925 career even strength save percentage might save about 12 goals, which would be two wins and $6M per year; they actually get $5.7 and $5.9M respectively. (Of course, Bryzgalov's contract runs through the age of 40 and gives no discount for his expected decline, and acquiring him meant not making good use of their promising and underpaid young goalie, but for a 10,000-foot-level analysis, his cap hit is acceptable.) Overall, our estimates seem to be coming out very slightly higher than the market -- probably more due to the roughness in the $3M per win figure than anything else.

It seems that current spending on goaltending is reasonable -- consistently a bit lower than our estimates suggest, but not by a clear enough margin for me to argue that goalie salaries should be increasing.

Goalies are like a box of chocolates

This conclusion may surprise some readers, as many in the statistical community argue that goalies are not worth large salaries. This argument generally derives from the variability of goalies; why should you pay a lot for a goalie if that's no guarantee that you'll get strong goaltending?

It's true that goalie performance varies quite a bit from year to year, but that's not something you can avoid. You'll be stuck with that variability no matter which goalie you sign. However, a good goalie's good years will be better and his bad years will be less bad -- that doesn't mean his bad years will be better than someone else's good years, but the average will clearly be better.

Variability isn't a reason not to prefer a superior goalie. It is, however, a reason to be cautious about identifying a goalie as superior. This is where teams can get into trouble: an average goalie can have a good year early in his career as a result of natural variability. The mistake some teams seem to make is in misreading him as a superior goalie on the rise and signing him to a big-money deal, only to find out later that they have overpaid for an average goalie. It is possible that scouts can detect star goalies earlier and more reliably than stats can, but several recent goalie contracts suggest that their projections are prone to significant error as well. The truth appears to be that if just a few tenths of a percent separate decent from great, we need we need to see multiple years to get that refined an understanding of a goaltender's skill.

The other danger is that those big-money contracts often come with long term these days. As I argued a couple of days ago, it seems that the pool of goaltenders is getting deeper and stronger, which presents an added risk for a long-term deal -- the superior talent that you sign today may find the league catching up to him over the course of his multiyear deal. Every long-term deal confers some risk of injury or decline in performance, but goalies may be unique in having this added market risk.

Because top players seem to be able to command long-term deals these days, this may be a risk that teams have to accept if they want to land a top goalie, but it should be a risk they take knowingly. In essence, this argument means that teams that are building should focus on goalies on short-term contracts and hope to develop a goalie from within, that they should spend big money for an upper-tier goalie only if their prospects don't pan out and they find themselves with a team that is capable of competing for a championship in the near term.

Conclusion

In summary:

  • No, teams aren't spending more on goalies lately; the fraction of cap space going to goaltending is actually shrinking. (Be careful with threshold-based analysis.)
  • The amount that is being spent on goaltending is not terribly out of line with their average contribution.
  • Their actual contribution will vary considerably around that average, which should not dissuade a team from investing in goaltending.
  • However, teams should be extremely cautious about signing a goalie who does not have a track record of several years at the desired performance level
  • Teams should also understand that long-term deals carry a unique risk of the goalie being rendered obsolete by new and improved goalies hitting the market in the coming years.

Previously by Eric T.

2654ef2681c88bc3252431ec45e30590
Eric T. writes for NHL Numbers and Broad Street Hockey. His work generally focuses on analytical investigations and covers all phases of the game. You can find him on Twitter as @BSH_EricT.
Avatar
#1 Ben Wendorf
October 11 2012, 08:59AM
Trash it!
0
trashes
Props
0
props

A theoretical comment...so, by comparing a goaltender to replacement, we can assess their value in terms of contribution vis-a-vis a skater, and we can talk about goaltender variability, but we rarely talk about comparing the risk of variability in a goaltender versus a skater. A goaltender with higher variability moves little in percentage, but that movement directly affects goals. A skater's variability will primarily reside in their shooting percentages, which varies more greatly in percentage, though a loss in shooting percentage does not necessarily mean a loss in team goals. You can't really blame them for their own goaltender's SV% variability, nor the variability of their teammates SH%. So the question is this: would skater risk being greater/less than goalie risk shift our conception of goalie vs. skater pricing?

Avatar
#3 Les Comber
October 11 2012, 01:31PM
Trash it!
0
trashes
Props
0
props

I love your work Eric T. Question for you regarding the use of ES SV%:

I understand ES SV% is a better measure of a goalie's skill than overall SV% but are we sure it can be used to project the difference in number of goals allowed over the course of a season? In other words, by ignoring special teams SV%, is it possible that we're undervaluing (or overvaluing) the difference, in terms of goals prevented over the course of a season, between an elite goalie and a replacement level goalie?

In other, other words, is the special teams SV% gap between an elite goalie and a replacement goalie larger than the ES SV% gap? And if so, would the ES SV% method undervalue the differences between goalies somewhat?

Avatar
#5 Tach
October 12 2012, 04:28PM
Trash it!
0
trashes
Props
0
props

Re: Player Variability

I think one of the reasons that player variability is within a narrower band, is that while shooting percentage is only one facet of a player's ability to score (ability to drive possession including Eric's excellent work on driving play through the neutral zone), a goalie's save percentage is pretty much his whole contribution. Maybe there is a bit on puck handling for a Brodeur, but that's it. If you have Pavel Datsuyk, even if he isn't putting the puck in the net, he is still generating possession for you and helping you win.

That is another reason why I think you can allocate a lower overall cap space to goaltending. The returns overall on the other side of the puck should keep the puck out of your net (by increasing your share of possession) should outweigh the risk weighted average of spending it on stopping shots.

Comments are closed for this article.