Predicting the Playoffs: Why Nobody is an Expert

Ben Wendorf
May 25 2012 09:34AM

I'm going to admit my bias: I don't like experts. I mean "experts," really, the people who posit themselves as the go-to person for hockey advice. There are a lot of intelligent and/or informed people out there who can provide equally apt insight into the game and its machinations (and sometimes, its psyche), but I guess it makes for some pretty milquetoast TV to just poll them all the time when you want to make a statement.  

When it comes to the playoffs and predictions, though, people like to pump their own tires a bit. Every group of analysts at all the levels of exposure, pomp, and circumstance truck out their predictions, sometimes all together, sometimes series by series. We scrutinize and agonize the picks, but for any "expert" the assumption is that their pick has elevation above the analysis of a bloke at the bar.

I'm here to tell you that, when it comes to playoff predictions, nobody's an expert.

Monkeys Tossing Darts

Remember when that octopus was picking everything correctly?  That should have been your tip-off.  Or maybe it should have been the fact that, to this point, only one NHL team hasn't had a four-game losing streak this year (the New York Rangers, though they have lost four of five twice). And that's the worst-case scenario, of course...you can be considered successful even if your winning percentage in a 7-game series would have barely put you in the playoffs in the first place (57.14%).

So, let's say you aren't willing to trust your gut (we've yet to prove this is the case with our "experts", but whatever), and wanted to select your teams by the conference rankings. Some years, that's a really good idea, just like it's sometimes a good idea for March Madness, so rock out. If you did that to predict each of the playoff series the last four years, your record would have been 42-30...or 58%.  

Keep in mind a coin toss is 50%.

Then, you decide that the 3 seed is throwing you off, so you shift over to points percentage (similar to win percentage, but remember the Bettman point). Happy that you are reaching a higher level of analysis, your reward over the last four years was 38-29 (in five series, the points percentage was identical)...or 57%. Well what the hell?

Okay, screw points percentage, sometimes the best teams don't always win; now we're going to move to what really matters, what the game is all about: goals. I think that it's really the defence that wins the game, so I'm going to say that the difference in goals allowed between the two teams will determine the winner. They had a hot goalie this season, so surely...ah shit, 58%.

Y'know, it's offence that wins the game, goals-for differential that's right, Gretzky Gretzky Crosby Crosby and all that...59%.

Hmm...well it's about holism here, we're moving into our John Lennon, Hindu-consciousness era and must consider the offence and the defence. Goal-differential...differential. Two become one.  68%, booyah baby!  Just as you're about to order Rosetta Hindi, you check the four years before the previous four years, just in case...

53%, or a full ten percent lower than if you chose by rank during those years.  Crap.

Beaten, bruised, waking up in the gutter, you've got no place left to turn except your gut (currently turning in a different way, so scratch that) or the...gulp...stats people. They're like the mole people only mole people come up from underground every once in a while. Probably to watch the games.

They tell you to use "Fenwick close,"  which you guess is a distant Kentucky cousin of Glenn Close, and you diligently record the numbers (they use decimals, jeebus) and run the results. 56%.

Fetal position, thumb-sucking...there there. It's just the playoffs.

Future Babble

You see, there's a certain level of dissonance between what even the last approach uses and the NHL rewards. Play 30 games against the same opponent and Fenwick Close is your strongest predictor, but put those two teams in a format where a singular result (winning a series) comes from a slightly positive effort (57% winning percentage) over a small sample, and you have a recipe for prediction disasters.

It's not a complete crapshoot, you're likely slightly better than a coin flip if you know the game, but in any year an octopus can put you to shame. There's only one exception, and that's when a big-time Fenwick Close team (55+) matches up against, well, pretty much anyone. Chances are, though, that same team is a 1 seed with high goal differential and is visibly better, so everybody picks that one. Not what you'd call the expert edge.

There are greater implications at work here. For one, teams sometimes use playoff results to shell out greater contracts and fans sometimes attach great significance to "playoff performers".  If any team has a pretty good chance at winning, any skater (and especially any goalie) has a pretty good chance at looking like a quality player. At the team level, this means that your roster decisions should be focused on making the playoffs, not agonizing over playoff results. I don't give a crap if you think Roberto Luongo isn't a playoff performer, he still turned in a regular season that would make the Philadelphia Flyers jealous.

But back to my initial point, to the people that feel they are (or are being presented as) "experts": you are not an expert, none of us are, not even myself.  

I'm over it.  Deal.

417ea714751a4580f06253a09a79bab0
Benjamin Wendorf was co-manager of the SB Nation Winnipeg Jets blog Arctic Ice Hockey (formerly Behind the Net); he is currently in the PhD program in Africology at the University of Wisconsin at Milwaukee. You can follow his graph work on Tumblr or his tweets @BenjaminWendorf. He can also be reached at bwendorf84 AT gmail DOT com.
Avatar
#1 Eric T.
May 25 2012, 10:17AM
Trash it!
0
trashes
+1
0
props

Agree wholeheartedly.

Put another way: in a league where the very best teams only win 60% of their games, a short series is never going to be terribly predictable.

A seven seed beating a two seed might sound like a big upset, but am I really supposed to be shocked that the Caps beat the Bruins in a seven-game series when the difference between them this year was only a win every 12 games or so?

In the NBA, the best teams win upwards of 70% of their games and the playoffs are for the most part fairly predictable. In the NHL and MLB, there is a lot more parity, and parity plus short series equals champion octopus.

Avatar
#2 Josh L.
May 25 2012, 10:40AM
Trash it!
0
trashes
+1
0
props

I hate having to make predicitions, and I hate having to defend predicitions I don't care to make even more.

Avatar
#3 Robert Vollman
May 25 2012, 11:08AM
Trash it!
0
trashes
+1
0
props

Agreed. A good analyst will tell you what to look for in a series, and the most probable outcome if the series were played 100 times, but it's up to you to remember that it is played but once.

Avatar
#4 Jonathan Willis
May 25 2012, 11:09AM
Trash it!
0
trashes
+1
0
props
Beaten, bruised, waking up in the gutter, you've got no place left to turn except your gut (currently turning in a different way, so scratch that) or the...gulp...stats people. They're like the mole people only mole people come up from underground every once in a while. Probably to watch the games.

This was a great post but that put it over the top.

Avatar
#5 Jerod
May 25 2012, 11:37AM
Trash it!
0
trashes
+1
0
props

It so hard to predict playoff hockey given the number of

concussions and injuries. Bigger faster hockey will eventually change they will have to increase the size of the rink, and make a few rule changes. Unfortunately the NHL is reactive and not proactive, so we wil l have to wait for some serious injuries from the superstars

But it will happen.

Avatar
#6 Eric T.
May 25 2012, 12:41PM
Trash it!
0
trashes
+1
0
props

@Jerod

I think the point here is that concussions and injuries aren't what make playoff hockey hard to predict -- it's just that the series are so short and the teams are so closely matched.

As an aside, I think Sidney Crosby and Chris Pronger would say we've already had some serious injuries from the superstars.

Avatar
#7 Danny Gray
May 25 2012, 02:19PM
Trash it!
0
trashes
+1
0
props

Excellent stuff here. Dan Gardner would be proud. It'd be great to go back and look at some of the past pre-season predictions to see how well the "experts" did. I'd imagine their record over a few seasons is no better than chance.

Avatar
#8 dan
May 25 2012, 03:49PM
Trash it!
0
trashes
+1
0
props

Ben;

Interesting post. IMO something that seems overlooked is that in all accurate evaluations of performance we need to first set the appropriate range.

Fact: as you pointed out we know the sport results have a large luck factor.

For case of argument let's use ~ 60% (a fair approx. for this years playoffs?) using formula from B. Burke's work at (advancednflstats.com)this means that a team with most skill wins 40% of time & 20% by luck or 60%. why? because luck means either team can get it so .5*40% =20%

Now imagine the perfect predictor of skill what we are all trying to do? he would get 6 out of 10. And, the predictor using just luck 5 out of 10?

So the range between expert and novice is only 1 out of 10

Seen in this light 57% predicting ALL playoff series and IF sustainable would be very good

It's all about quantifying expectations. Of course, if your buddy goes 5 out of 10 & you go 6 out of 10 doesn't really feel like intellectual domination but if repeatable IMO it is difference btwn mansion on the hill and 9-5 job..

Comments are closed for this article.