Putting error bars on NHL equivalencies

Eric T.
November 09 2012 07:40AM


After I posted the prospects wing of our reference library, I got a couple of interesting replies on Twitter:

I love what the #fancystats community have done, lots of useful stuff, but confidence in their goalie and prospect analysis a bit too high.

@steffeG @Sens_Army_ @NHLnumbers not in that library are any columns pointing to the limits of prospect analysis.

I asked both of them to explain further. (I also defended myself -- my Tweets announcing both of those articles made light of how iffy the stats are in those areas, so I think it was clear that I know there are limitations.) The essence of their feedback, as I understood it, was that by not actively discussing the uncertainty, we as a community have implied that we know more than we do.

This strikes me as a fair criticism, at least in part. It seems like half the articles in the goalie section talk about how unpredictable goalies seem to be, so I'm not sure I'd accept the critique there. But it's pretty rare for us to put actual error bars on our projections. For prospects in particular, there have been a lot of articles written where we give league translation factors to two decimal places; I am pretty certain that the authors did not mean to imply that we can project results to within 1%, but we haven't explicitly laid that out for people.

Can we actually estimate the uncertainty on those projections?

Read Article | 5 Comments

Reference library: Prospects

Eric T.
November 07 2012 04:05PM

This article is part of the NHL Numbers reference library, which seeks to collect articles from around the web that have contributed to our understanding of the game.

This page is devoted to articles that touch on prospects. This will include league translations, historic trends, and other things that relate to evaluating prospects.

We need your help to keep the library complete and up to date -- contact me on Twitter (@BSH_EricT) or via email (bsh.erict -at- gmail) with suggestions of articles you think we should consider adding.

Return to the library main index.

Read Article | 0 Comments

Review: zone start adjustments to shot differential

Eric T.
November 05 2012 07:26AM

Malhotra faceoff
Little known fact: Manny Malhotra personally funds 60% of the research into zone start adjustments
Photo by Matt Boulton

Note: In conjunction with the NHL Numbers reference library, we will occasionally be publishing review articles aimed at establishing the contemporary understanding of an area and highlighting opportunities for additional research.

Today is the five-year anniversary of what I believe is the first published list of how often players took offensive or defensive zone faceoffs, and with it in the comments came the question of how zone starts might affect their on-ice shot differential.

There have been a lot of attempts to answer that question over the last five years. In this review article, we will touch on some of them and then make our own recommendation.

Read Article | 5 Comments

Reference library: Goalies

Eric T.
November 02 2012 04:12PM

This article is part of the NHL Numbers reference library, which seeks to collect articles from around the web that have contributed to our understanding of the game.

This page is devoted to articles that touch on goaltending. This will include save percentage, team effects, aging profiles, and other things that relate to evaluating goalies.

We need your help to keep the library complete and up to date -- contact me on Twitter (@BSH_EricT) or via email (bsh.erict -at- gmail) with suggestions of articles you think we should consider adding.

Return to the library main index.

Read Article | 5 Comments

Reference library: Player usage

Eric T.
November 01 2012 08:08AM

This article is part of the NHL Numbers reference library, which seeks to collect articles from around the web that have contributed to our understanding of the game.

This page is devoted to articles that touch on player usage. This will include zone start adjustments, quality of competition metrics, teammate analysis, and other factors that might impact a player's results.

We need your help to keep the library complete and up to date -- contact me on Twitter (@BSH_EricT) or via email (bsh.erict -at- gmail) with suggestions of articles you think we should consider adding.

Return to the library main index.

Read Article | 3 Comments