How Efficient is the NHL Goaltending Market?

Hawerchuk
July 04 2012 04:30PM

 

 

Imagine, if you will, a professional hockey league where every season, one of the league's leading scorers came available as a free agent, looking for two-year contract at $2M per year, and could make your team 3-4 wins better.  Also imagine that every year, some of the league's other leading scorers got locked up to long contracts and proceeded to turn in net negative seasons despite not being injured.  Unthinkable?  Well, this describes the NHL goaltending market!

The last three seasons, you could have had Mike Smith, Antti Niemi or Craig Anderson - all of whom had put up overall good numbers during prior their careers - for next-to-nothing and then reaped the rewards.  At the same time, you could have made the decision to go long with Ilya Bryzgalov or Marc-Andre Fleury and ended up with the same performance you could have had from a number of minor-league goaltenders.

Why does this occur? Relative to skaters, we just don't get enough observations of goaltenders to figure out their true talent levels. But goalies are still evaluated on the basis of a year's or season's worth of data, just like forwards and defensemen, so their good (or bad) performance is over-valued.  And so we end up with big contracts for goalies with big error bars on their expected performance.

What I was wondering is whether teams have internalized these lessons in some way.  The percentage of total league salary going to goalies has dropped significantly over the last five seasons, suggesting some understanding of the potential inefficiency of goalie contracts:

Season Total $ Goalie $ Goalie %
2007-08 1404.8 134.2 9.56
2008-09 1615.7 140.1 8.67
2009-10 1648.3 143.3 8.70
2010-11 1704.7 144.8 8.49
2011-12 1776.3 145.7 8.20

But at the same time, teams continue to dole out huge long-term contracts to goaltenders, particularly their own goalies - all but one of the eleven five-plus-year contracts went to a team's own goaltender.  So are the teams that spend a lot of money on goaltending spending any more or less efficiently than the teams that don't?

  Save % GVT Salary above Repl $M/Win Wins
"Top 10" 916.2 163 59.1 2.18 27.1
"Bottom 10" 911.9 56 22.0 2.35 9.4

As far as I'm concerned, it's a wash.  The more you spend - in general - the better your team's save percentage.  (We expected that.)  But teams that spend more on goaltending are no more efficient than those that spend less.  Of course, just because a team spends less, it doesn't mean that it's better at allocating resources.  It's better to look at the top-paid goaltenders and see how they performed relative to their contracts:

  Save % Above Repl Salary above Repl $M/Win Wins
"Top 16" 916.7 190 73.7 2.33 31.6
"Next 16" 913.1 59 16.2 1.64 9.8

Just so that we don't get confused by RFA and ELC contracts, I looked at every goaltender age 28 and over who made $1M or more.  The top 16 goaltenders made, on average, $4.6M over replacement, while the next 16 made $1.0M.  (Interestingly, both groups had the same standard deviation on their save percentage, so it's not as though spending more money gave them more certainty.)  The bottom 16 goalies also played an average of 35 full games compared to 55 for the top 16.

Conclusion

I think we're finally at the answer here: there's huge value available in the second tier of goaltenders, though teams have difficulty evaluating them as they do with all other goalies.  Given that uncertainty, it is almost certainly better to spend less money on goalies and to give them the shortest contracts possible; forwards and defensemen are much more likely to be sure things, and you're better off paying for certainty.

F133ba2f9d7737aace589a3811b7b758
Gabriel Desjardins (aka Hawerchuk) grew up a huge Jets fan in Winnipeg. He runs the hockey stats website Behindthenet.ca and has written for the Wall St. Journal, the National Post, the Globe and Mail, ESPN.com and MLB.com. He has also worked as an analyst for a number of teams and agents in the four major sports. In real life, he attended Queen's University and UC Berkeley, and works in Silicon Valley. Previously, he briefly worked as a lumberjack in Manitoba. Contact: info at
Avatar
#1 antro
July 05 2012, 09:13AM
Trash it!
0
trashes
Props
0
props

This is great stuff, and certainly it seems there are teams who are employing the "next 16" strategy (e.g. Red Wings, Ottawa, Philly before Bryzgalov). But I wonder about the point on standard deviation and certainty. Does the top 16 have a smaller standard deviation than the next 16 in EV% from *season to season*, so that a team employing a top 16 goalie is taking a smaller risk that they will get Corey Crawford's year instead of Mike Smith's?

I also think that part of the inefficiency has to do with GM's hoping that a long enough contract will mean that the price relative to the future crop of "next 16" will be worth the difference. This might be rational enough. I'm wonder if it's possible to model this in the some way.

Avatar
#2 chuck
July 05 2012, 11:03AM
Trash it!
0
trashes
Props
0
props

Something that wasn't considered in your analysis is the importance of the Head Coach.

There is a reason why goalies that plays under Ken Hitchcock or Dave Tippett post incredible numbers no matter what their numbers were previously.

Just look at Steve Mason numbers "with or without" Hitchcock.

In 2010-11, Mike Smith was Tampa Bay 2nd goalie and he had horrible stat. Last year, he was in the top 5 in a lot of categories. Did he improve that much over 1 summer or is it the impact of Dave Tippett?

It's difficult to evaluate a goalie because unlike a skater, they can't push the play into the offensive zone.

They are also at the mercy of their teamates. A great/good goalie on a bad team will have average numbers. A replacement level goalie on a great team could post all-stars numbers.

Evaluating goalies is difficult/impossible because we haven't found the right metric to properly evaluate them.

Still looking at your post, it shows that GM at the NHL level clearly haven't found the way to see who's good and who's bad.

But then again, most of them can't even reconize the talent in forward or defensemen. Do you really expect them to get the goalie's right?

Comments are closed for this article.