July 31 2013 10:44PM
This is the third in a series of articles looking at the history of the NHL Entry Draft from 1979 to 2008. I have tabulated every pick made by every team during that time, selecting out the players who managed 200+ NHL games as draft successes. The initial look is a simple study wherein I credit a team with finding an NHL player, regardless of their draft position or impact on the NHL. The notable weaknesses of this approach are that it evens the playing field between the 3rd line journeyman winger and the franchise defenseman.
In this article I will be examining the draft years from 2000 to 2008. For the purposes of development I prefer to leave a five-year window before judging a draft year, hence the 2008 cut off. Also for the purposes of inclusion I have had to extrapolate some results for the years 2006 through 2008. I have tried to choose conservatively and ignore any bias; however being aware of this alteration to the methodology, the final two years can be viewed with some skepticism.
To begin with, a note of clarification. The writers at the Nations pride themselves on getting their ducks in a row, specifically when it comes to terminology. Those of us who are pushing the “advanced analytics” line especially so, because it doesn't lend credence to the cause when we flub basic statistical terms. In the two previous articles I made mention of a qualitative and quantitative study. This is a misnomer. What I have undertaken is a quantitative study, with separate thresholds (the first being marginal NHL players, the other impact NHL players). A true qualitative study of the draft, while likely fruitful and fascinating, is well beyond my resources.
A sincere thank you to alert reader Luke Thomas Mergner for bringing this to my attention, and a sincere apology for the error.
Luke sums up the difference between the two systems in an email to me, the following excerpt of which is published with his permission:
“Qualitative studies are not less exact discussions of mathematical data. They are, or hope to be, a distinct and no less valuable approach to the study of social phenomena. Thus the first premiss to reject is that all social objects lend themselves to mathmatical analysis. Often, this implies a deeper interest by the researcher in the meanings that social actors attribute to their actions. In the case of a draft, we might ask why a General Manager drafted a particular player. This is a different question than how successful were their drafts over time, though related and certainly of interest. You may also note that the qualitative study does not necessarily maximize the number of cases. Instead, it might focus on a few key, representative cases.
“...qualitative studies generally rely on different sources of data...[a]ssuming we are interested in draft success, but also care about organizational strategy and judgment, the study might interview the crucial decision makers in each organization. Why did they do what they did? What constraints did they perceive; What opportunities? What were their strategies? Aside from interviews, qualitative studies can rely on archives, journalism, etc.
“The crucial difference between quantitative and qualitative is how the understand the process of validation. Qualitative studies prefer deep analysis that offers a plausible causal narrative over statistical correlation. [sic]
So this is a quantitative study, one that looks to define what draft success looks like, draw a reasonable line in the sand as to what to expect from future drafts, and hopefully illustrate the incredibly difficult task that amateur scouts face each season as they try to suss out the music from the noise.
My references in earlier articles to qualitative results should read as a modification of the threshold from replacement-level NHL player to impact player. I'll delve further into that in subsequent articles.
To date we have seen a league average draft success rate of between 23% and 21%, depending on the decade, and extending from the phenomenal years of the 80s (1979 had a success rate of 53%) to the disastrous draft years that ended the 90s (1999 had a success rate of only 15%). The fluctuation between draft years is statistically notable, even if it translates in real numbers to only a body or two. The crucible that is the draft-and-development model in the NHL means that every body counts, and a shortage of prospects in-house usually results in a team having to take resources away from somewhere else to make roster ends meet.
We left off our previous article looking over the rubble of the 1997 and 1999 drafts. Financially the league was in poor condition and the relief that many franchises had hoped to find in acquiring cheap developing talent through the Entry Draft came largely to naught.
As such, there were several teams that crawled into the new millennium in very poor condition after years of sub par drafting and/or financial hardship. Most Canadian teams, the Maple Leafs aside, were in difficult financial circumstances, having endured the late-90s recession, the collapse of the Canadian dollar and a financial landscape in the NHL that could be generously described as asymmetrical. Some were also hamstrung by extended periods of draft futility.
For those teams the 90s were not so much a time to regroup and rebuild as they were a time to simply stay alive. The Canadian landscape lost two of its franchises during this period, and I think it is important to mention that both of those teams were strong teams at the draft, indicating that they were undone by forces outside of the control of Hockey Operations.
Then came the 2000 NHL Entry Draft. This draft welcomed two new teams to the NHL cartel and managed, somehow, to surpass even the mind-numbingly bad draft year that was 1999 in terms of success rates, with a meager return of only 14% out of the 293 player selected managing to play even 200+ NHL games. In terms of higher-end talent, it only slightly edged out 1999, managing to produce 13 impactful NHL players, to the previous year's 9. Details on this process will follow in later installments.
Suffice to say, 2000 was not a good year. In fact, overall, the 2000s held some very poor draft years in quick succession. A look at the graph below will speak volumes about the boom-bust nature that the draft begins to take on during this decade, with the years 2000, 2002, 2004, 2005, and 2007 all falling below the 19%-20% historical baseline of league-average draft success. While the others during that time period, 2001, 2003, 2006 and the early indications on 2008 are all around league-average.
Reviewing the 2000s at the Draft
So what can be gleaned from the emerging pattern? Did the draft suddenly become a more risky enterprise? Has there been a significant change in the draft strategy of most NHL teams since 2001 when we see this new pattern emerge?
On the first point, we need to take an historical approach. The draft appeared during its first decade as a resource with steadily declining value. From an initial rush of talented players to an eventual trickle of talent. The end of the 1980s ushered in a period of stagflation for the draft that came to dominate the 1990s. That is to say, a combination of circumstances where the best teams retained their dominance at the expense of the weaker teams and there was a stalling of the recycling process of franchise renewal which the draft was meant to address. Poorer teams became bad, and the draft, which was designed to deliver to them elite-level talent failed which meant that they continued to stay at the bottom of the standings while other teams were forced due to finances to avoid falling too far down the standings and were thus trapped in a seemingly never-ending cycle of mediocrity. Meanwhile the top-tier teams engaged in a continual race against each other to acquire the best free-agent talent forcing player salaries in an upwards spiral that only served to further separate them from the weaker majority of the league.
In other words, if a team had top-end talent and the financial resources to retain them in the 1990s they could expect to remain at the top of the standings for the better part of the decade as the talent recycling mechanism built into the NHL was defeated by the severe financial imbalances between teams.
The final decade in our exercise would seem to suggest that the market has entered a period of instability and fluctuation. That being said, the book hasn't been closed on many of the players selected during this final decade and there may yet be some changes to the data set that alter our perception of relative successes and failures – Brian Bickell was drafted in 2004 for instance and only this year passed the 200+ game threshold.
When trying to evaluate if the draft has become a riskier proposition for teams the numbers can only get us so far. This is something I'd like to explore further down the road.
To the second point, the draft strategy of NHL teams is a closely guarded secret that they often end up wearing on their shirtsleeves. If other factors remain unchanged at management and hockey operations, one need only study a team's historical selections to garner some insight into the types of players that have been selected and thus determine what attributes are most highly valued by the organization.
That being said, there are some external factors that have had an influence on draft selections, some of which I have mentioned previously – the trend amongst many teams during the late 80s and into the early 2000s was to draft size when it was available alongside, but independent of, skill. The exceptions were teams like the Buffalo Sabres who selected skill ahead of size. Once the rules were altered following the 2005 lockout to favour skill and speed, the draft priorities of some teams changed slightly although size still ranks quite high as an important factor in any prospect’s assessment.
So has there been a change in philosophy amongst NHL teams? The data gathered doesn't contain any intrinsic information that could be used to determine that, and for the most part we are not privy to the information that could be used to determine this on a large enough scale to apply across the league.
When most people think of the draft during this decade they think of one year in particular.
The 2003 Draft Mythology
So how good was 2003? Based on my definition of what constitutes an impactful player this draft year was the best since 1983. It delivered to the NHL a large collection of higher-level players who continue to influence the league today. When viewed from the perspective of the overall depth of the draft as a whole, including the more pedestrian players who are available from year to year, it is in the range of the better draft years but is essentially on par with years like 1990, 1993, 1994, and 1995. Those same years we have just recently described as being devoid of elite-level talent and contributing to the polarization of power within a few larger-market franchises. So 2003 delivered a larger proportion of higher-level NHL players than those drafts mentioned above, but the overall rate of NHL players delivered to the league was in the range of other draft years. This is a crucial distinction that needs to be observed when we discuss the depth of a draft.
It is important to note that the while 2003 draft did bring in a large collection of household names to the NHL, there were many teams who found only one or two players in this now-legendary year, including Carolina (E. Staal), Calgary (Phaneuf), New Jersey (Parise), the Rangers (Dawes), Tampa Bay (Tarnasky), Toronto (John Mitchell), Vancouver (Kesler) and Washington (Fehr). Phoenix failed utterly to find a prospect in their entire draft cohort of eight selections.
This was a very intriguing year but it is an outlier not unlike the 1979 and 1980 drafts.
What 2003 did bring to the table was a collection of talented players at key positions - defense and center - who have dominated the league since the end of the lockout in 2005. That so many of the names selected in that year are at or just past their peak career years no doubt informs the view many have on this draft class.
Interestingly, while it is somewhat premature, early indications are that the 2008 draft may have been the best draft by sheer numbers since 2003, and with the smaller draft pool has a success rate by percentage not seen since 1983. This is based on some extrapolations of current trajectories, but the early data suggests that it may at least equal 2003 in some areas, and as I mentioned earlier skepticism of the data is called for at this stage of the process. I'll undertake a more in-depth review of this in the coming weeks.
Looking at the Big Picture
Getting back to the timeline from 2000 to 2008 one can spot a slight upwards trend in the draft results beginning in 2001. This becomes more clearly visible when we look at the graph of the entire draft history. We can't know anything about the drafts between 2008 and present outside of the emergence of the most quickly established elite-level talents so the numbers could continue onwards or plummet again in our current decade.
The oughts brought a different dimension to the draft-day experience: something of a boom-bust scenario as one strong year was usually followed with a weak one and back and forth.
In 2005 the NHL lockout signified a massive realignment of franchise power bases. When one looks at the teams that were the most powerful and financially capable during the pre-lockout years and compares it to those today it is, in some cases, a complete reversal. In the ten years leading up to the 2005 lockout three teams accounted for eight of the Stanley Cup Championships. In the eight years since only one of those teams has won (Detroit), with the other two suffering significant setbacks (Colorado and New Jersey).
The effect of the lockout was also felt on the draft such that it was now limited to seven rounds and the rule changes emphasized speed and skill, thus altering how some teams would draw up their draft board. The lockout bisects our final draft decade and thus the results are likely to be affected although there is not yet enough data in this exercise to fully determine to what extent.
In the end what a fan can take away from this review is that the draft is not a science. There is no guarantee of anything going into a draft year that would allow one to know with certainty whether it is strong or weak, as even the best scouts can be wrong more often than not. One can make educated guesses, but history has a way of making fools out of even the wisest men.
The league-wide average rate of successfully selecting a player sits just north of 20%. In a team's draft cohort of roughly seven to eight players that means that history suggests one NHL player will emerge. This also means that most NHL amateur scouts have a failure rate somewhere in the range of 80% - a rate that is not entirely theirs to control given the many factors that influence a prospect’s development including physical growth, professional development, maturity, injury, developmental league politics, organizational competence and roster redundancies, to name a few.
They key factors one can take away from this exercise is that scouts have one of the most difficult talent assessment jobs in the industry, and generally can perform at a reasonable rate. They often receive less credit than is perhaps due and are due only a portion of the blame that is often directed their way. When one reviews a draft cohort for a specific team, a reasonable rate of success can fall anywhere between zero and 30% and this should temper expectations during the post-draft glow that surrounds most organizations.
In the next installments I will delve more into some particular issues that have come out from this analysis including a look at some of the best and worst draft performers and, time and data permitting, whether there is a link between management or place in the standings and a franchise's draft cycles.