October 03 2012 08:03AM
Clutch play is a common source of debate in sports. We've all seen players come up big in big moments, but does that mean they are clutch, that we should expect them to do it again next time?
Variance is a part of life; everyone has good days and bad days, or even good years and bad years. We know intuitively that a single year's worth of games doesn't tell us everything we need to know -- people knew that Nikolai Kulemin was unlikely to repeat as a 30-goal scorer, and hopefully nobody is counting on Max Talbot for 19 goals next year. Yet players like Talbot and Johan Franzen get the clutch label after much fewer than 80 playoff games.
In this article, we will compare players' playoff performances to their career rates and look at whether the results we see in playoff performance are consistent with typical variance or whether the number of people at the extreme high or low ends exceeds what we would expect by simple random chance.
There's no question that some players do better in the playoffs than others, of course. For the most part, that's because they are better players -- you don't have to invoke clutch skills to explain why Craig Adams doesn't score as much in the playoffs as Sidney Crosby.
If we want to call a player clutch, what we need to show is that he scores more in the playoffs than you would normally expect from someone with his talent. And in this article, the goal is to take sample size into account as well, so we know how likely it is that he'd be that far above his typical production level.
To do that, we'll start with an estimate of his typical production rate. We'll take his regular season point total since the last lockout and divide by his minutes played to get a rate stat -- Danny Briere had 420 points in 8292 minutes, for 0.051 points per minute. Over that span, he has played 1950 playoff minutes, so we might expect him to have 1950 * 0.051 = 99 playoff points.
However, when you work out the expected points for everyone in the league, you find that the mix of better defenses and fewer penalties actually suppresses scoring slightly in the playoffs; after we take that into account, we would expect Briere to have had about 87 playoff points since the last lockout. He's actually had 106 points, so he's done better than we'd expect.
106 points is definitely better than 87, so people are right to think of Briere as someone who's done well in the playoffs. But remember, results fluctuate up and down all the time; what we want to know is whether 106 points is higher than we should reasonably expect by random chance.
To assess that, we can use a statistical tool called the standard error, which is like a standard deviation for a counting statistic. For 87 points, the standard error would be sqrt(87), or a little more than 9. So at 106 points, he is almost exactly two standard errors above our expectations. We expect a little more than 95% of the league to fall within two standard errors of their expected results, so about one in 40 would be that high by random chance (and another one in 40 would be two standard errors below expectations). If we see more players at the extremes than that, then we might conclude that some players are clutch and some are chokers.
We have 215 forwards whose talent and usage lead to an expectation of at least 10 playoff points. If variance were the only factor in play, we would expect about 5% of them -- about 11 players -- to be more than two standard errors away from their regular season scoring rate. We actually see 13 people outside that envelope, which is in quite good agreement.
Similarly, if we calculate how many standard errors above or below expectation each player is, we would expect the standard deviation of this distribution to be 1.0 if the differences in playoff performance were determined purely by variance; instead it is 1.1. So both methods of assessing the data suggest that we see very slightly more spread than we would expect by random chance.
After we allow for random chance, what's left for clutch tendencies is at most 10% of the difference we see from player to player.
In fact, clutch skill probably accounts for even less than that; some of the spread will be the result of imperfections in our estimation of expected points. For example, Cory Stillman's playoff results almost all came in the 2006 playoffs, but his expected scoring in this model is based on the six years he played from '05-06 to '10-11. Since his scoring dropped sharply over those years as he grew older and league scoring went down, it is not surprising that in those 2006 playoffs he markedly outscored the expectation calculated based on the six-year span. Such factors will tend to increase the number of outliers we observe in this model, so the actual room for clutch talent is likely even less than 10% of the observed differences between players.
In other words, we don't need to invoke clutch skill to explain our distributions -- they fit almost perfectly with what we'd expect from natural variation.
We see a similar result when we look at goalies.
There are 52 goalies whom we would expect to have allowed at least 10 goals, so we might expect two or three to be outside the 95% envelope, and we actually see three. So like with forwards, we see that some players have outperformed expectations, but the number of clutch performers is just what we would expect through simple random chance. The implications for Chris Osgood's worthiness for the Hall of Fame will be left as an exercise for the reader.
The number of people whose performance improves or declines in the playoffs is almost exactly what we would expect from simple variance over the small playoff sample sizes. It is thus hard to argue that clutch talent is a significant factor in playoff performance, or that people who have had improved outcomes in the playoffs should be expected to continue to do so.
Moreover, even if there were some players who genuinely performed better in the playoffs than in the regular season, I am not sure this would be something we should celebrate. If a player really does improve his scoring skill in the playoffs, there are only two possible conclusions:
- Everyone else chokes under pressure and performs worse, but the pressure doesn't bother him and he maintains his normal skill level. Since his opponents' play is declining, he scores more points. However, if this were the case, we would almost certainly not see the results above -- we would expect to see a lot more players outperforming expectations if all it took was not choking.
- He performs at less than his full ability during the regular season, then turns it on for the playoffs. Is coasting during the regular season really something to celebrate? Aren't the players who give their all in every game more noble than the ones who wait until they think a game is worthy of their full effort?
We don't see evidence that players really do elevate their performance in the playoffs, but even if we did, I would argue that those players might be more deserving of criticism than praise.
Previously from Eric T.
- Assessing performance in all three zones: Zone entry data for the Wild and Flyers
- A competition metric based on ice time
- Time on ice competition plots for all 30 teams
- Projections for Rick Nash on the Rangers