Shot quality matters, but how much?

Eric T.
July 03 2012 12:18PM

 

 

Last week, I delved into the always-controversial realm of shot quality, looking at how tight the link between scoring chances and shot differential is. In the comments there and in some subsequent conversations, I was reminded that the current understanding of shot quality is often oversimplified and misrepresented.

I don't think anyone believes that there is no such thing as shot quality, that all shots are equal. The argument is actually that most shot quality effects are smaller than people think, and that over the sample sizes we normally work with, differences in shot quality tend to be dominated more by noise than talent. In this article, I'll walk through some examples and try to emphasize the subtle difference between saying shot quality is meaningless and saying it is often negligible.

Team-level shooting percentage

Even after 82 games, about 2/3 of the shooting percentage differences between teams comes from simple variance. The result of this is that when a team leads the league with a 8.86% shooting percentage at 5-on-5 (0.74% above average), the best guess at their long-run true talent is only 8.32% (0.20% above average). Most of the difference you see between teams in a season is just chance.

  • The point is not: "All teams shoot for the same percentage, so ignore shooting percentages completely."
  • The point is: "Differences in shooting percentage are small and require a very large data set to overcome noise, so you won't be wrong by much if you ignore them."

Individual shooting percentage

Some players definitely shoot for a higher percentage than others. This appears to be driven as much by where they shoot as from how well they shoot, but shooting locations are much more reproducible. In fact, people's shooting skill is so transient that only Ilya Kovalchuk and Alex Tanguay can be unambiguously identified as being good shooters. The result is that players' shooting percentages vary a lot from year to year, and you should look at several years' stats to estimate how a player will shoot going forwards.

  • The point is not: "There's no such thing as a good shooter, so ignore shooting talent completely."
  • The point is: "You can tell shot location skill quickly, but differences in shooting talent are significant and require a very large data set, so look at several years of shooting percentages to make your predictions."

Linemate shooting percentage

Some players have a higher on-ice shooting percentage than others. (On-ice shooting percentage is the team's shooting percentage with that player on the ice.) Even including the player's own shooting percentage, four years of data only identifies 10% of players as having unambiguously high or low on-ice shooting percentages (see comments section). We already said that some players have higher shooting percentages than others, so remove the players' own shots and the spread shrinks. Remove the tendency of good shooters to play together, and it looks like over multiple seasons we see only a few top playmakers improving their teammates' shooting.

  • The point is not: "There's no such thing as a good passer, so ignore linemates completely."
  • The point is: "The ability to improve teammates' shooting is small and requires a very large data set to overcome noise, so you won't be wrong by much if you ignore it."

Save percentage

The average change in a goalie's save percentage from year to year when he stays with the same team is just 0.0005 larger than simple random chance would predict, and when a goalie changes teams the sv% difference is just 0.0011 larger than random chance. The best shot-quality-influencing system of this era (Jacques Lemaire's) reduced Fenwick shooting percentages by about 0.0015. The result is that any team effect on a goalie's save percentage doesn't add up to more than a goal or two per season.

  • The point is not: "All teams face the same shots, so ignore shot quality completely."
  • The point is: "Differences in shot locations are small and require a very large data set to overcome noise, so you won't be wrong by much if you ignore them."

Conclusion

Not everyone agrees with what I've laid out here, but I've tried to capture what the majority of the leading analysts believe, emphasizing the nuanced distinctions that we usually skip for the convenience of writing readable prose. David Johnson's work is a good place to start if you're looking for the dissenting viewpoint, focusing much more on goals than shots. But as Hawerchuk has summarized:

Together, Fenwick/Corsi and Luck account for around 3/4 of team winning percentage.  What's the remainder?  Goaltending talent - which Tom Awad estimates at about 5% - and special teams, along with a very small sliver that's due to shooting talent and the oft-mentioned "shot quality."

In general, shot quality factors tend to be small enough that they don't grossly alter our understanding of the game, and they tend to be swamped by noise during in-season analysis. The best possible understanding obviously requires more than a cursory glance at shot totals, but shot-based analysis has consistently proven to be a strong approach to identifying talent and predicting outcomes.

Recently by Eric T.:

2654ef2681c88bc3252431ec45e30590
Eric T. writes for NHL Numbers and Broad Street Hockey. His work generally focuses on analytical investigations and covers all phases of the game. You can find him on Twitter as @BSH_EricT.