March 18 2014 12:06PM
If you've ever seen the movie The Social Network you will remember the scene early on where Mark Zuckerberg creates a website which allows the students of Harvard to compare and rate the hotness of two randomly selected females from the university and over time this allowed the campus to see who the collective student body thought was the "hottest" looking.
The very algorithm that was explained to Zuckerberg by Edwardo Saverin was the Elo rating system and lately I've been playing with it to see how it can work in the context of rating hockey players.
Read on past the jump to learn the method and results.
Elo Rating System
Elo, like PDO, is not an acronym but rather the name of it's creator (analytics people seem to not think about their naming conventions even pre-internet!). The Elo rating system is "a method for calculating the relative skill levels of players in such competitor-versus-competitor games as chess".
The basic idea is that every player starts with a pre-determined amount of points (usually 1000 or 1500 in chess) and after every win or loss that number gets updated.
What is most interesting in the math is by how much your score changes after each match. If you beat someone who is much weaker than you you will only increase slightly compared to if you had beaten someone much stronger. If you lose to someone who is much weaker then you your score will drastically drop whereas a weak opponent losing to a strong opponent will not decrease very much. The system is self correcting so if a strong player starts off with a few losses he will start moving quickly in the right direction when he starts winning.
What is also beneficial is that the difference in scores between two players serves a predictor in the outcome. Two players with equal scores are expected to have an equal number of wins against each other. A player with a score greater than 100 than his opponent is expected to win 64% of the time while a differential of 200 would suggest 76% of victories for the stronger player.
The full mathematics can be found on the Wikipedia entry.
The more I've been thinking about this method the more I've been wondering how it would work with hockey.
NHL Elo Rating
In the NHL players work in groups of 3, 4, or 5 on the ice (plus the goalie) facing a similar sized squad. Those who you are playing with greatly effect how well you're likely to play. Playing with strong players will likely cause you to have an inflated production compared to normal. Conversely, playing with weak teammates will make it harder for you to score compared to normal. We already have Quality of Teammates (QoT) to reflect this.
Just like QoT, if you're always being matched up against top lines it's going to become difficult to score than if you were going head to head against the opposing team's bottom lines they would be easier to score against (a factor we usually call quality of compeition - QoC).
In theory, Elo should be able to reflect this. If every goal is a "win" between the two on-ice squads you should be able to capture the QoC and QoT into a single variable and those players who are on the ice for lots of goals, despite weak teammates and in the face of strong opponents will have higher a higher Elo rating than players who are scoring with strong teammates against weak opposition.
So this is what I did: I ran the Elo rating on every on ice goal in the NHL so far this season (to 16 March, 2014). The Elo of both on-ice squads were averaged and compared and then all players had their score updated. Every player in the NHL started with an Elo rating of 1000 and it was updated by there (with a k of 32).
It is too long of list to give you the results of all 826 players on the ice for a goal so far this NHL season so here is a small subset.
The top 20 players and their scores have been:
And the bottom 20 players so far have been:
The full list can be found here. The Python code can be made available upon request if you want to play with it yourself.
It is too early to say if Elo has any value but worst case it is interesting to play with. We can use just the eyeball test and see many of the players in the top and bottom of the list are players many would consider to have performed well/weak this year.
Players like Kunitz are going to be in the top of the list because they are on the ice for many goals against tougher competition (give he is playing with Crosby). On the bottom of the list we can see lots of defencemen, which makes sense given how much ice team they see. Alex Edler is on the low end which matches how he's been doing this year given he's been on the ice for many goals against.
Further works is most likely needed to see if these numbers are repeatable.
As any hockey analytics guy will tell you, you will never have a single stat that is a catch-all for a player. Despite that we have stats like GVT, THoR, and variants of WAR. I see a player's Elo rating more like those than a single category stat such as a players Corsi.
More work is needed to see if Elo is even useful but the initial eye tests suggest it does a decent job at representing players who are on the ice for a lot of goals while trying to adjust for their teammates and opponents.
Thoughts? Opinions? Ideas? Feel free to share them.