Do NHL goaltenders age in a recognizable pattern?

Cam Charron
January 05 2013 02:21PM

There has been some work done on goaltender age recently, and the basic question surrounding the analysis has been "can we predict how Roberto Luongo will perform in his late 30s?" If we can, and if the indication is positive, it would imply a team may be wise to fork over a key asset or two for one of the best, consistent goalies of the era.

Luongo plays a very important role in goaltender analysis. He's the first elite goaltender to rise to prominence after the NHL began publishing even strength save percentage numbers in 1997-98. Luongo has played 12 NHL seasons, all of them coming after the switch. The only other goalie to play more games than Luongo since 1998 is Martin Brodeur, but some of his numbers are buried behind the impenetrable wall of simple save percentage as he established himself in New Jersey.

I downloaded 397 seasons of goaltender data, the criteria being that the goalie had to have faced 500 or more shots at even strength, and it had to come directly before another season where the same goalie played a season with 500 or more shots. The idea was to sort them out by age, calculate the year-to-year improvement of goaltenders, and see if there's any discernible age curve to be found (the method taken from this Hardball Times piece on hitters).

The key difference at this point between baseball and hockey statistics is that in-depth recording of baseball games goes back a century, whereas we only have about a decade to look at. I best fit some sort of a curve into the plot point, but there are so few goaltender seasons the qualified in the late-30s that it creates a real mess with the data:

Here's how Nate Silver would describe my overfit model, from his book The Signal and the Noise:

"The over fit model scores those extra points in essence by cheating—by fitting noise rather than signal. It actually does a much worse job of explaining the real world.

Overfitting represents a double whammy: it makes our model look better on paper but perform worse in the real world. Because of the latter trait, an over fit model eventually will get its comeuppance if and when it is used to make real predictions."

The curve I have shows a decline of goaltenders as they go into their late 30s, which is similar to what Eric T. found about a month ago. While it it probably true that goaltenders decline in their late 30s, and they most certainly lose their ability as they get older (to use an extreme example, picture Johnny Bower trying to stop Mike Ribeiro with his signature poke check today) it is also true that goaltenders who have poor seasons in their 30s are less likely to get another chance in the NHL than goaltenders who have poor seasons in their 20s. This creates the problem with survivorship bias, the ones who stick around are the Hall of Fame goalies or Dwayne Roloson.

Here are how a few well-known goalies performed in their late 30s and early 40s according to Wins Above Replacement, a rough metric I scraped together using backup and replacement-level save percentage for each season, divided by the average number of goals per game. I've normalized it for 1000 shots:

  Roloson Belfour Hasek Joseph
37 0.3 2.3    
38 -0.5 2.0 -1.3 0.6
39 2.0     -0.4
40 0.5 -0.9 5.0  
41 1.9 2.0 4.1 -3.8
42 -3.5   2.5  

Roloson had a lousy 38-year-old season, but all told, he was a very useful hockey player at 39, mediocre at 40, useful again at 41 before falling off a cliff last season, ne'er to be seen again. Dominik Hasek is a freak of nature who dominated with the Senators in 2006 and 2007 out of the lockout with Ottawa and Detroit, and had a strong 2008 at 42, the year he passed the torch to Chris Osgood and Jimmy Howard.

Osgood is one of a couple of goalies who simply ended after age 38, along with Olaf Kolzig and Mike Vernon, after posting negative-WAR seasons. I guess they didn't want to torch their legacy like Curtis Joseph practically did. Even after skipping a season he was clearly not fit to play at 41. Ed Belfour, however, was.

That's what's puzzling about old goaltenders, or any goaltender, really. They don't follow standard paths, and while my model above has ball parked the prime of goaltenders as between 26-31, the data also shows a massive uptick improvement for goalies between ages 32 and 33. Three excellent goalies entered the even strength save percentage at that era: Belfour, Hasek, and Patrick Roy. But beyond that, Ron Tugnutt, Sean Burke, and Miikka Kiprusoff had uncanny resurgences at that age. The safe thing to say is that there just isn't enough data.

There's unfortunately a sample size issue, and I'm not confident enough to put any money on how Luongo will perform in his mid-to-late-30s as his contract could become a burden on any franchise. I am confident to say he will probably be very good for four of the next six years, but there's too much noise after that to conclude anything further.

We may have to wait another 20 years or so before we have enough goalies to define a legitimate age curve. For now, I'm going to revert to my old standard: there's so much we don't know about goaltenders that it isn't worth sinking any money into trying to find the "right" one. You can play the market more effectively by having a cheap goalie play in a defensive system like Phoenix and St. Louis accomplished last season.

(h/t to Rob Pettapiece for helping me with math)

63811cbf517d2d685ea09e103488ea3a
Cam Charron is a BC hockey fan that writes about hockey on many different websites including this one.
Avatar
#1 Eric T.
January 07 2013, 10:22PM
Trash it!
0
trashes
Props
1
props

There's another kind of sample bias in this data that you need to watch out for.

Did you notice that almost every year, goalies get worse? Even from 18 to 24, their year-over-year change is below zero each year. The mathematician imported from planet Zoob would conclude that the best goalies in the world must be in juniors, or maybe mites.

So what's really going on? When you threshold it at a minimum number of 500 shots, you introduce a bias.

Imagine you have two 20-year-old goalies whose true talent is .915 goaltending, as measured by precision scientific equipment that only you possess. They both get called up to the NHL for a try-out stint.

One of them runs hot, stopping 93% of the shots over his first ten games. The other one runs cold, stopping 90%. Goalie number two gets sent back down to the AHL before he reaches 500 shots faced, and he doesn't show up in your data. Goalie number one stays in the NHL (though he regresses a bit over the rest of the season) and finishes the season with a .925 save percentage in 20-25 games and 500+ shots.

Next year, they both play a full season and finish at .915.

Your data set contains one point: goalie number one's -0.010 change. The goalie who improved doesn't show up, because his down year got filtered out as a result of coaches thinking he needed more seasoning. The result of coaches managing their goaltending like this is a negative skew in your data.

I'd suggest combating this by avoiding the threshold and using a weighted average instead. For example, you might weight by the lower number of shots faced, so if goalie number 1 saw 600 shots each year, that would produce a -0.010 data point with weight 600, and if goalie number 2 saw 250 shots in year 1 and 600 in year 2, that would produce a +0.015 data point with weight 250. Then you could take the weighted average of everyone instead of excluding based on a threshold; this might not eliminate the bias altogether, but it would go a long way towards reducing it.

Avatar
#2 Beta
January 05 2013, 03:50PM
Trash it!
0
trashes
Props
0
props

I agree with most of your comments regarding the sample issues.

What I see is greater variance when 'tenders enter their thirties. Which is sort of what your narrative may be implying.

Here is a thought...if you are interested in a goaltender's trajectory, why not fit some sort of growth curve model and examine the typical curve?

Avatar
#4 Ben Wendorf
January 06 2013, 12:09PM
Trash it!
0
trashes
Props
0
props

What are your data points on that graph? Are those "age 20-to-21," "age 21-to-22" data points?

All told, I agree with your point that you can't really tell for a goaltender in their late-late 30s, and it's unlikely that we'll get enough data for that until well into the future. Regardless, you can look from about 19 to 37 or so and identify a pretty good curve there that's trending down after 33 or so. I wouldn't be scared of the noise if it's not bumping above 0.0; you can safely assume that your players are declining at that point.

Noise may make your prediction less definitive, but the nature of the noise is important. If your noise is dipping into the positive yet, you have reason to believe that a goaltender might not decline.

Avatar
#6 ubermiguel
January 06 2013, 09:35PM
Trash it!
0
trashes
Props
0
props

What are the sample sizes for each age? The variability starts at 33, did the sample size also drop there?

Comments are closed for this article.