Stats are ruining hockey

Eric T.
October 29 2012 07:23AM

Sabermetrics! Fun!
"It's a triumph of number-crunching over the human spirit -- and it's about time."

It's something I think we've all heard, that stats are ruining the game. It always struck me as nonsense.

After all, a fan could easily watch every game on television, read his hometown newspaper every day, talk to the guys at the bar all the time, and never once encounter a non-traditional statistic. If you really hate hearing about non-traditional hockey stats, it seems like your strongest rational claim is that stats are ruining the arguments you like having in certain circles on Twitter and blog comment sections.

Surely they aren't actually hurting the product on the ice, right?

Driven by incentives

Incentives

There's a passage in Nate Silver's book The Signal and the Noise in which he talks about a basketball bettor who got an edge by observing the behavior of Ricky Davis. With his Cavaliers out of the playoff hunt in 2002, Davis's incentives to maximize the team's chances of winning were reduced, and his play was increasingly reflecting a focus on his impending free agency.

Knowing that players are often evaluated in part on statistics such as points per game and assists per game, he was pushing the offense to run at a much higher pace. Regardless of what effect that might have on the team, getting extra possessions would lead to him getting more points and assists in each game, which in turn would presumably make him a more valuable commodity in the upcoming free agency market.

Some might see this as selfish, and I can't say that they're wrong. A player in a team sport has a variety of incentives, and sometimes the best action for the team is not the best action for the player; a reasonable definition of a selfish player would be one who chooses the option that benefits him more often than most people would.

However, I am also hard-pressed to criticize every individualistic play. Certainly, some individuals are more selfish than others, but the system of incentives is complex and ever-changing.

There are a lot of things that can reduce the incentive for a player to help his team. Maybe his choice becomes less important to the team because they are out of playoff contention or because the game has gotten out of hand. Maybe he has become less interested in helping them because he wants to see a coach he dislikes fail or because he's not worried about letting down teammates who have treated him badly. A veteran who knows he is retiring at the end of the year and has never won a championship might be more team-focused than the player whose team has made it clear that they won't re-sign him and have made him available before the trade deadline.

So let's take it for granted that many players will sometimes make a play that suits his needs more than the team's -- some might do so much more easily, egregiously, or often, but it is probably unavoidable that it will happen on occasion. What does that have to do with the argument that stats are ruining the game?

Misaligned incentives

Selfish

Sometimes those selfish plays will have nothing to do with how a player is evaluated. A player gets out of position to deliver a big hit because he just enjoys delivering big hits. Another player takes an untimely retaliatory penalty because the opponent gets under his skin and he loses sight of what his team needs from him. But sometimes the connection to glory and money is strong, and a player, like Davis, takes actions directly aimed at boosting his market value.

Right now, the stats with strongest influence on a player's perceived value in hockey are his goals and assists. As a result, a selfish player who seeks individual glory might wait for a breakout pass to set up an offensive chance instead of backchecking hard to help his team recover the puck. This kind of play can unquestionably hurt the team -- he might not be making the optimal play -- but at least when he succeeds in scoring a goal, it is clearly a positive for the team.

One of the primary revolutions in recent statistical analysis has been the discovery that shot differential bears a strong correlation to winning and is one of our best predictors for future wins. As a result, the shot differential with a given player on the ice is widely seen as a measure of his ability to drive play and an important component of his value.

But unlike with goals and assists, nobody thinks shot volume alone is the cause of winning. Shot differential is valuable only because we've observed that differences in shot quality from team to team are small, and so the number of shots is a good indicator of how many goals a team is likely to get. That doesn't always have to be true.

If in those selfish moments, players start taking shots from anywhere on the ice just to boost their shot differential, then this will no longer be a matter of simply focusing on offense over defense. It's not a big deal if individual incentives lead players to sometimes make choices that are slightly less than optimal, like a point guard pushing the offense a little faster. But when our value metric is not a very direct cause of success, the incentives shift from promoting slightly less optimal sugestions to promoting an active undermining of the team's goals.

A shifting landscape

A growing number of teams are adopting the idea that shot differential is an important factor in rating a player. Obviously the notion that selfish players would be shooting from the outside to pump up their shot differential for the Corsi nerds is a bit tongue in cheek. But the possibility that new views of the game will result in new behaviors by players is real, and that possibility underscores the need for us as analysts to constantly update our views on the game. After all, the idea that shot quality differences are small from team to team is by no means universal or immutable; it wasn't that way a generation ago and won't necessarily be that way a generation from now. 

In fact, the progress of statistical analysis may even propel such changes. In a world where half of the teams adopted a heavy focus on shot differential and half didn't, we would likely see a growing disparity in shot quality skills as one group targeted players who drive shot differential and the other group would then happily see the shot quality players fall to them. The very act of convincing teams not to focus on shot quality could serve to make shot quality a more important factor in evaluating teams than it had been before.

So what are we as analysts to do? We can continue working to break things down from the high-level results-oriented analysis into components that more directly relate to specific skills, as with the zone entry project. But most importantly, we need to continually update our studies and re-evaluate our understanding. The community of statistical analysts has been quite good so far at adapting to new information uncovered by novel analysis, but we need to make a point of also updating or reconfirming the old information.

Previously by Eric T.

2654ef2681c88bc3252431ec45e30590
Eric T. writes for NHL Numbers and Broad Street Hockey. His work generally focuses on analytical investigations and covers all phases of the game. You can find him on Twitter as @BSH_EricT.
Avatar
#2 AronV
October 29 2012, 07:51AM
Trash it!
0
trashes
+1
0
props

@Eric T.

A player that could fit that description would be Tyler Kennedy on the Penguins, I don't think even he believes some of his shots have any chance of going in.

Avatar
#3 David Johnson
October 29 2012, 08:14AM
Trash it!
0
trashes
+1
0
props

"In a world where half of the teams adopted a heavy focus on shot differential and half didn't, we would likely see a growing disparity in shot quality skills as one group targeted players who drive shot differential and the other group would then happily see the shot quality players fall to them."

Which is why we shouldn't evaluate players by just corsi or by just shot quality but by a combination of both. In fact, I would go a step further and suggest that the more important question to answer is how to maximize the skills of both. Is it best to build a team (or a line on a team) that is all one type or the other, or do the skill sets complement each other such that the best way to build a team/line is to have a combination of the two.

For me, maximizing team and line building is an area I'd like to research more.

Avatar
#4 Kent Wilson
October 29 2012, 09:36AM
Trash it!
0
trashes
+1
0
props

It's something I think we've all heard, that stats are ruining the game.

It's funny this charge is never leveled at traditional stats, which are included on the back of baseball cards and poured over by just about everyone about the most casual of casual fans.

As you imply, Eric, it's more about the deflating of precious myths than the "ruination of the sport". People prefer to think in grand narratives rather than regression to the mean.

Avatar
#5 Matt Pfeffer
October 29 2012, 09:51AM
Trash it!
0
trashes
+1
0
props

Are we certain a higher density of shots to the net is a bad thing? Maybe instead of adjusting to make their stats look good players will adjust because they think it's an inefficiency.

When I explained corsi to some people inside hockey the first conclusion they jumped to was that players who took more shots (as in, on a per opportunity basis,) were more effective.

Avatar
#7 gongshow
October 29 2012, 11:32AM
Trash it!
0
trashes
+1
0
props

Thanks Eric.

I look at baseball and see that some of the old school stats could definitely lead to misaligned incentives, but as statistical analysis of the game has matured, it is giving a truer sense of what contributions hold value.

Similarly, I can see how, when misinterpreted or mis-used, hockey stats can lead to problems (ie: your shot quantity argument).

Subjectively, on the topic of shot quality vs quantity, I'll prefer fewer high quality shots over higher quantity but lower quality shots any day. (exhibit A: Ethan Moreau). I'd love to see save %age for goalies evolve to some hybrid of this as well.

Avatar
#9 Ben Wendorf
October 29 2012, 03:54PM
Trash it!
0
trashes
+1
0
props

@Kent Wilson

I'd go even further than that, Kent...the most annoying thing I can think of are pseudo-advanced stats that pick apart data and attached arbitrary importance. For example, the records of teams against one another the past 10 years ("the Sabres have NEVER defeated the Kings after January 1st at home"), or a player's boxcars in away games that year. We dislike them because they have all the substance of farts in the wind, but people who dislike fancy stats make those kinds of stats emblematic of the things that we do.

Avatar
#10 Corey Pronman
October 29 2012, 08:35PM
Trash it!
0
trashes
+1
0
props

The only serious stats debate happens in the public, not in the industry. Advanced analysis has been part of hockey for a while, and I'd say from what I've heard a majority if not a significant majority of teams use Corsi, Qual Comp etc.

Comments are closed for this article.