Elo for Evaluating Players

Josh W
March 18 2014 12:06PM

equations

If you've ever seen the movie The Social Network you will remember the scene early on where Mark Zuckerberg creates a website which allows the students of Harvard to compare and rate the hotness of two randomly selected females from the university and over time this allowed the campus to see who the collective student body thought was the "hottest" looking.

The very algorithm that was explained to Zuckerberg by Edwardo Saverin was the Elo rating system and lately I've been playing with it to see how it can work in the context of rating hockey players.

Read on past the jump to learn the method and results.

Elo Rating System

Elo, like PDO, is not an acronym but rather the name of it's creator (analytics people seem to not think about their naming conventions even pre-internet!).  The Elo rating system is "a method for calculating the relative skill levels of players in such competitor-versus-competitor games as chess".  

The basic idea is that every player starts with a pre-determined amount of points (usually 1000 or 1500 in chess) and after every win or loss that number gets updated.

What is most interesting in the math is by how much your score changes after each match. If you beat someone who is much weaker than you you will only increase slightly compared to if you had beaten someone much stronger. If you lose to someone who is much weaker then you your score will drastically drop whereas a weak opponent losing to a strong opponent will not decrease very much. The system is self correcting so if a strong player starts off with a few losses he will start moving quickly in the right direction when he starts winning.  

What is also beneficial is that the difference in scores between two players serves a predictor in the outcome.  Two players with equal scores are expected to have an equal number of wins against each other. A player with a score greater than 100 than his opponent is expected to win 64% of the time while a differential of 200 would suggest 76% of victories for the stronger player.

The full mathematics can be found on the Wikipedia entry.

The more I've been thinking about this method the more I've been wondering how it would work with hockey.

NHL Elo Rating

In the NHL players work in groups of 3, 4, or 5 on the ice (plus the goalie) facing a similar sized squad.  Those who you are playing with greatly effect how well you're likely to play. Playing with strong players will likely cause you to have an inflated production compared to normal. Conversely, playing with weak teammates will make it harder for you to score compared to normal. We already have Quality of Teammates (QoT) to reflect this.

Just like QoT, if you're always being matched up against top lines it's going to become difficult to score than if you were going head to head against the opposing team's bottom lines they would be easier to score against (a factor we usually call quality of compeition - QoC).

In theory, Elo should be able to reflect this. If every goal is a "win" between the two on-ice squads you should be able to capture the QoC and QoT into a single variable and those players who are on the ice for lots of goals, despite weak teammates and in the face of strong opponents will have higher a higher Elo rating than players who are scoring with strong teammates against weak opposition.

So this is what I did: I ran the Elo rating on every on ice goal in the NHL so far this season (to 16 March, 2014). The Elo of both on-ice squads were averaged and compared and then all players had their score updated. Every player in the NHL started with an Elo rating of 1000 and it was updated by there (with a k of 32).

Results

It is too long of list to give you the results of all 826 players on the ice for a goal so far this NHL season so here is a small subset.  

The top 20 players and their scores have been:

Name Elo
PATRICK SHARP 1257.654286
BRAD RICHARDS 1193.100984
BRYAN LITTLE 1184.808848
JAROMIR JAGR 1183.532442
JONATHAN TOEWS 1182.401575
CLAUDE GIROUX 1178.685406
DEREK STEPAN 1176.780315
CHRIS KREIDER 1174.052787
BRADEN HOLTBY 1172.018125
MATS ZUCCARELLO 1167.575772
JIRI HUDLER 1165.549261
CHRIS KUNITZ 1163.020994
COREY CRAWFORD 1158.968555
DANIEL ALFREDSSON 1155.887982
LOGAN COUTURE 1154.15938
MATT NISKANEN 1153.909063
ANDREW LADD 1152.976379
JAKUB VORACEK 1145.21001
ERIC STAAL 1139.540773
BLAKE WHEELER 1138.247828


And the bottom 20 players so far have been:

Player Elo
BRIAN BOYLE 898.5037608
MARC-EDOUARD VLASIC 897.2985421
ED JOVANOVSKI 896.9982169
ADAM HALL 894.1102713
JAN HEJDA 890.1714008
DAN ELLIS 886.7437428
TAYLOR PYATT 885.7621945
ARTEM ANISIMOV 882.6942826
DREW MILLER 877.6910266
PAUL GAUSTAD 876.4984082
CHRIS HIGGINS 871.6601917
TREVOR DALEY 866.7753016
ERIC GRYBA 866.657868
BROOKS ORPIK 863.9218216
SERGEI BOBROVSKY 862.7468304
ALEXANDER EDLER 843.7521203
DAN GIRARDI 836.2345469
BARRET JACKMAN 833.6014587
CASEY CIZIKAS 832.2938684
KARL ALZNER 830.5719978


The full list can be found here.  The Python code can be made available upon request if you want to play with it yourself.

Summary

It is too early to say if Elo has any value but worst case it is interesting to play with.  We can use just the eyeball test and see many of the players in the top and bottom of the list are players many would consider to have performed well/weak this year. 

Players like Kunitz are going to be in the top of the list because they are on the ice for many goals against tougher competition (give he is playing with Crosby). On the bottom of the list we can see lots of defencemen, which makes sense given how much ice team they see. Alex Edler is on the low end which matches how he's been doing this year given he's been on the ice for many goals against. 

Further works is most likely needed to see if these numbers are repeatable. 

As any hockey analytics guy will tell you, you will never have a single stat that is a catch-all for a player.  Despite that we have stats like GVT, THoR, and variants of WAR. I see a player's Elo rating more like those than a single category stat such as a players Corsi.

More work is needed to see if Elo is even useful but the initial eye tests suggest it does a decent job at representing players who are on the ice for a lot of goals while trying to adjust for their teammates and opponents.

Thoughts?  Opinions?  Ideas?  Feel free to share them.

172ff756e336b4deef407cc7fc644369
I am a Van Fan in Bytown. Living in Ottawa for work, I research Sports Analytics and Machine Learning at the University of Ottawa. I play hockey as well as a timbit but I compete in rowing with hopes of 2016 Olympic Gold. Follow me on twitter at @joshweissbock and feel free to give me a shout.
Avatar
#1 mk
March 18 2014, 03:49PM
Trash it!
0
trashes
Props
6
props

Very interesting method - I was skeptical this method could be used effectively here (hockey) given the difficulty of matching up players 1v1. This is the best attempt I've seen anywhere. Any thoughts on doing this type of analysis using shot data rather than goal data?

Looking through the list, these are interesting (picking some "big name" players a bit randomly): Crosby - 156 / Ovechkin - 24 / Bergeron - 252 / Doughty - 262 / Malkin - 713 / Pietrangelo - 638 / Bouwmeester - 256 / D. Sedin - 22 / H. Sedin - 57

Avatar
#2 Ricardo
March 19 2014, 01:06AM
Trash it!
0
trashes
Props
1
props

Also very interested in seeing it applied to shot date. Obviously much more computationally intensive but likely more telling. Also would be interesting would be to see the average for a team and the average for a team based on average ice time.

Avatar
#3 Phil
March 20 2014, 08:57AM
Trash it!
0
trashes
Props
0
props

I think that this is fairly accurate. Was their anything taken into account for powerplay and shorthanded goals, or was this strictly 5 on 5? Every player has different line mates throughout the night, where did you get the numbers to rate the strength of these line mates?

Avatar
#4 Baalzamon
March 21 2014, 09:54AM
Trash it!
0
trashes
Props
0
props

Why are there so many NYRangers players (FOUR!) in the top 10?

Avatar
#5 Vais Mahendran
March 23 2014, 11:43AM
Trash it!
0
trashes
Props
0
props

This is definitely interesting. I would love to see the code for this to play around with it myself.

Comments are closed for this article.