May 18 2013 11:23AM
You probably don't know me, my name is Josh Weissbock and I am a Masters student at the University of Ottawa up in Canada and I study Machine Learning. Many of the other authors on the Nations Network, who use advanced stats, typically use statistical models to analyze hockey where as I use algorithmic models. What I can do is use Machine Learning techniques which is an application of these same statistical ideas. Machine Learning is simply the ability for a computer to take a whole bunch of data, learn from it and be able to make decisions.
Part of my research is combining Sports Analytics and Machine Learning; can we use these algorithms to learn from hockey data and try and make accurate decisions or predictions? This area actually has very little academic research; in fact I could find no peer reviewed published works on predicting anything in hockey in any field. Basketball, American football and soccer all have plenty of research in machine learning but none have been found in hockey.
This post is the first of many on applying Machine Learning to hockey and the NHL. In this simple experiment to see if it is possible to make simple predictions on who will win the game, using known stats on each team. The goal is to beat 50%, or a coin flip. This type of predicting model is valuable both to team managers and for betting.
I've been collecting data daily on every game that happened over the NHL season. This experiment used games over a ten week period fro February to April 2013, a total of 386 games. Before each game I collected advanced stats and traditional stats on the teams. After the game I collected the result of who won, the score and the shots for and against. This gives me the flexibility to calculate more stats in the future.
The traditional stats that were collected were: Home/Away Status, Goals For, Goals Against, Goal Differential, Power Play Success Rate, Power Kill Success Rate, Shot Percentage, Save Percentage, Winning Streak and Conference standings. For the advanced stats I collected the Fenwick Close %, 5-on-5 Goals For/Against ratio and the season PDO of each team. These stats have been shown to be much more correlated with wins than traditional stats.
Classification & Results
After pre-processing the data I plugged the vectors of game data into the algorithms to let the algorithms do their learning. The algorithms I used were Neural Networks (NN) because they are good with noisy data; Support Vector machines (SVM) because it does well in machine learning; Naive Bayes (NB) to learn on past data to predict the future; and Decision trees (J48) as they create a human readable output. I won't go into the math of how these algorithms work but I have linked them to their Wikipedia page which gives further detail.
Testing of the algorithms was done using 10-fold cross-validation. The results of the learning with these algorithms are presented in the table below. I ran the algorithms on just the traditional stats, just the advanced stats and a mixture of both.
Conclusion & Future Work
60% accuracy, that's pretty decent for a very simple experiment. That would probably make profit in the long run if used in betting. I can't confirm which algorithm is statistically best without running a t-test. What I find most interesting is that we see this advanced stats on the Nation Network are the best for forecasting future success but here they didn't seem to make any improvement in the forecasting rate. That doesn't mean they aren't valuable in machine learning, it most likely stems from the fact I only used three of them. Also interesting is if you look at the decision tree generated ML finds the most value in the teams Goals For, Power Play % and Conference standing in predicting a game winner. I don't think PDO is that useful in predicting over the long run as it regresses to 100% but it would be much more useful in predicting the short term.
There is much future work to build off this, first I would start with using a different variation of PDO. Machine Learning can predict more than just who will win a game, it has been used to predict the scores of games, who will win a tournament, which prospects are likely to work out, and there has been plenty of ML research in baseball on predicting award winners and hall of fame nominees.
If you are interested in my data for your own work let me know. If you have any questions or discussion feel free to send me a line. This experiment is a positive result in making predictions in hockey using Machine Learning and future posts will look at it in more depth.