Do some players elevate their game in the playoffs?

Eric T.
October 03 2012 08:03AM

Clutch play is a common source of debate in sports. We've all seen players come up big in big moments, but does that mean they are clutch, that we should expect them to do it again next time?

Variance is a part of life; everyone has good days and bad days, or even good years and bad years. We know intuitively that a single year's worth of games doesn't tell us everything we need to know -- people knew that Nikolai Kulemin was unlikely to repeat as a 30-goal scorer, and hopefully nobody is counting on Max Talbot for 19 goals next year. Yet players like Talbot and Johan Franzen get the clutch label after much fewer than 80 playoff games.

In this article, we will compare players' playoff performances to their career rates and look at whether the results we see in playoff performance are consistent with typical variance or whether the number of people at the extreme high or low ends exceeds what we would expect by simple random chance.

Methodology

There's no question that some players do better in the playoffs than others, of course. For the most part, that's because they are better players -- you don't have to invoke clutch skills to explain why Craig Adams doesn't score as much in the playoffs as Sidney Crosby.

If we want to call a player clutch, what we need to show is that he scores more in the playoffs than you would normally expect from someone with his talent. And in this article, the goal is to take sample size into account as well, so we know how likely it is that he'd be that far above his typical production level.

To do that, we'll start with an estimate of his typical production rate. We'll take his regular season point total since the last lockout and divide by his minutes played to get a rate stat -- Danny Briere had 420 points in 8292 minutes, for 0.051 points per minute. Over that span, he has played 1950 playoff minutes, so we might expect him to have 1950 * 0.051 = 99 playoff points.

However, when you work out the expected points for everyone in the league, you find that the mix of better defenses and fewer penalties actually suppresses scoring slightly in the playoffs; after we take that into account, we would expect Briere to have had about 87 playoff points since the last lockout. He's actually had 106 points, so he's done better than we'd expect.

106 points is definitely better than 87, so people are right to think of Briere as someone who's done well in the playoffs. But remember, results fluctuate up and down all the time; what we want to know is whether 106 points is higher than we should reasonably expect by random chance.

To assess that, we can use a statistical tool called the standard error, which is like a standard deviation for a counting statistic. For 87 points, the standard error would be sqrt(87), or a little more than 9. So at 106 points, he is almost exactly two standard errors above our expectations. We expect a little more than 95% of the league to fall within two standard errors of their expected results, so about one in 40 would be that high by random chance (and another one in 40 would be two standard errors below expectations). If we see more players at the extremes than that, then we might conclude that some players are clutch and some are chokers.

Results

Looking at which players increase scoring in playoff games

We have 215 forwards whose talent and usage lead to an expectation of at least 10 playoff points. If variance were the only factor in play, we would expect about 5% of them -- about 11 players -- to be more than two standard errors away from their regular season scoring rate. We actually see 13 people outside that envelope, which is in quite good agreement.

Similarly, if we calculate how many standard errors above or below expectation each player is, we would expect the standard deviation of this distribution to be 1.0 if the differences in playoff performance were determined purely by variance; instead it is 1.1. So both methods of assessing the data suggest that we see very slightly more spread than we would expect by random chance.

After we allow for random chance, what's left for clutch tendencies is at most 10% of the difference we see from player to player.

In fact, clutch skill probably accounts for even less than that; some of the spread will be the result of imperfections in our estimation of expected points. For example, Cory Stillman's playoff results almost all came in the 2006 playoffs, but his expected scoring in this model is based on the six years he played from '05-06 to '10-11. Since his scoring dropped sharply over those years as he grew older and league scoring went down, it is not surprising that in those 2006 playoffs he markedly outscored the expectation calculated based on the six-year span. Such factors will tend to increase the number of outliers we observe in this model, so the actual room for clutch talent is likely even less than 10% of the observed differences between players.

In other words, we don't need to invoke clutch skill to explain our distributions -- they fit almost perfectly with what we'd expect from natural variation.

We see a similar result when we look at goalies.

A comparison of goalie playoff and regular season performance

There are 52 goalies whom we would expect to have allowed at least 10 goals, so we might expect two or three to be outside the 95% envelope, and we actually see three. So like with forwards, we see that some players have outperformed expectations, but the number of clutch performers is just what we would expect through simple random chance. The implications for Chris Osgood's worthiness for the Hall of Fame will be left as an exercise for the reader.

Conclusion

The number of people whose performance improves or declines in the playoffs is almost exactly what we would expect from simple variance over the small playoff sample sizes. It is thus hard to argue that clutch talent is a significant factor in playoff performance, or that people who have had improved outcomes in the playoffs should be expected to continue to do so.

Moreover, even if there were some players who genuinely performed better in the playoffs than in the regular season, I am not sure this would be something we should celebrate. If a player really does improve his scoring skill in the playoffs, there are only two possible conclusions:

  • Everyone else chokes under pressure and performs worse, but the pressure doesn't bother him and he maintains his normal skill level. Since his opponents' play is declining, he scores more points. However, if this were the case, we would almost certainly not see the results above -- we would expect to see a lot more players outperforming expectations if all it took was not choking.
  • He performs at less than his full ability during the regular season, then turns it on for the playoffs. Is coasting during the regular season really something to celebrate? Aren't the players who give their all in every game more noble than the ones who wait until they think a game is worthy of their full effort?

We don't see evidence that players really do elevate their performance in the playoffs, but even if we did, I would argue that those players might be more deserving of criticism than praise.

Previously from Eric T.

Recently on NHLnumbers

2654ef2681c88bc3252431ec45e30590
Eric T. writes for NHL Numbers and Broad Street Hockey. His work generally focuses on analytical investigations and covers all phases of the game. You can find him on Twitter as @BSH_EricT.
Avatar
#1 Kent Wilson
October 03 2012, 09:12AM
Trash it!
0
trashes
Props
0
props

So what you're saying is the Washington Captials are a bunch of chokers, right?

Avatar
#2 Jared Lunsford
October 03 2012, 12:14PM
Trash it!
0
trashes
Props
0
props
Kent Wilson wrote:

So what you're saying is the Washington Captials are a bunch of chokers, right?

No, he's saying they're due!

Avatar
#4 Kent Wilson
October 03 2012, 01:12PM
Trash it!
0
trashes
Props
0
props

@Jared Lunsford

Touche.

Avatar
#5 Peterman
October 03 2012, 01:13PM
Trash it!
0
trashes
Props
0
props

Claude Lemieux

Avatar
#7 Lolz
October 03 2012, 11:03PM
Trash it!
0
trashes
Props
0
props

Greatest clutch player of all time?

Paul Henderson. Lets leave it at that

Avatar
#8 Pardini36
October 04 2012, 07:41AM
Trash it!
0
trashes
Props
0
props

"He performs at less than his full ability during the regular season, then turns it on for the playoffs. Is coasting during the regular season really something to celebrate? Aren't the players who give their all in every game more noble than the ones who wait until they think a game is worthy of their full effort?"

Not sure how to quantify this, but couldn't there be an argument for a smaller guy like Briere that he's not "coasting" in the regular season, but if he went to the same "dirty" areas all season long he would be injured or worn out by the post season, so he protects himself until the games matter the most?

Avatar
#9 not norm ullman
October 04 2012, 07:47AM
Trash it!
0
trashes
Props
0
props

Great stuff, Eric, though some of the conclusions are overstated or perhaps misstated, imho.

1. The NUMBER of players out-performing in the playoffs may match up with your estimate of variance, but this in no way explains their IDENTITIES. That is, the concept of a player being "clutch" is in no way eliminated simply because only a limited number of players have it.

2. That business about how we should be angry at players who "coast" during the regular season is just plain nonsense, and I hope it's just your attempt to brushback the idiots who ONLY go on about "clutch" and "European" and similar nonsense. Look. A RATIONAL player will exert their limited amount of energy/effort when it produces the most benefits, right? We see this literally every single hockey game we watch. Players do not sprint back to the bench, they do not chase every puck - in short, every fight is not a fight to the death. The same goes for a rational approach to a season. Some games are well in hand early on, and most players adjust within those games. If you're up 3, you probably don't throw as many punishing checks, don't dive straight into a heap of skates and the melee in front, the same way as you would in a bigger game. In particular, players of limited physical size could ONLY be considered rational if they did this. Guys like Briere and Giroux. Any NHL body is always carrying wounds and damage, and guys with slighter builds, or who are outweighed, who be irrational to throw themselves into the fray every game and every situation. So I think you perhaps need to adjust the wording of that last point.

There are also bigger players, physically, who coaches do not ask to take the full beating in front of the net every night, again and again, no matter the situation. This isn't about "coasting," any more than a baseball coach pulling his big horse starter after just 6 innings is. A guy like Frantzen takes an awful beating, but you probably MOST want him to do it in the more important games.

4. There is also the clear phenomenon of players with the skill to score against worse players, but not against better ones. Guys who can tear up the NHL, but, for whatever reason, not in the NHL is an obvious version. But it holds for any sport. Players who can dominate smaller golf courses, but who can't drive long enough for the best courses. Baseball players who can smash an 85 mph fastball, but not come close to a 90 mph one. And hockey players who have fabulous moves, but only a limited subset.

There are also the quite high-talent players who FEED off these weaker players. They've got moves that work great on Komisarek, but not so much on Chara. And some players just plain ARE more interested in racking up goals against these guys, even in runaway games, than other players are.

Applied to this scenario, the playoffs are effectively removing more of these bad players - with all their weaknesses - from the game. There are still weaknesses and mistakes being made on the better teams, and with the NHL including SO MANY teams in the playoffs, the "weak player" filter is more round-by-round, I would suspect.

But from that, what you would expect is that certain players, some who were just too weak to score on the best quality opposition - as well as some high-talent guys who feed on the weaker ones - would show a sharper decline in performance as we entered the playoffs, and the rounds moved along, and the games were tighter.

5. Now, does this argument I'm making in any way prove "clutchness?" No, it doesn't. Does it make a rational case that some players may - rationally and for completely logical reasons - score what is expected form their regular season performances? Yes, actually.

And might there actually just be a bit of "clutchness" to it all? Sure. Any study of war or the human response to stress will show a curve. As it goes up, some people don't do as well, while others perform very well indeed. You might be able to tell me how many players will fall apart, or how many will soar, but doing so in no way destroys the reality that they may have authentic differences. It's just that our statistical methods can't detect WHO it is.

Thanks for the piece, and the chance to respond, Eric.

Avatar
#10 not norm ullman
October 04 2012, 07:49AM
Trash it!
0
trashes
Props
0
props

4. Should read, "Guys who can tear up the AHL, but, for whatever reason, not in the NHL...."

Avatar
#13 not norm ullman
October 04 2012, 09:57AM
Trash it!
0
trashes
Props
0
props
Eric T. wrote:
A RATIONAL player will exert their limited amount of energy/effort when it produces the most benefits, right? We see this literally every single hockey game we watch. Players do not sprint back to the bench, they do not chase every puck - in short, every fight is not a fight to the death. The same goes for a rational approach to a season.

I'm not sure your analogy works, because I don't think physiological recovery is uniform across timescales. Sprinting to the bench is a lot more likely to weaken your performance on the next shift two minutes later than playing hard in an October game is to weaken your performance in May.

But even if I let you have that, and assume that somehow working hard in October does make you a worse player in May, remember that we're not comparing NHL players to an ideal who goes all-out all the time; we're comparing them to each other. Presumably the league as a whole rationally distributes its energies and efforts. Out of all of the players who expend their energies in various ways, why should we particularly celebrate the ones who put in the least effort in October, who force their teammates to expend more energy to carry them to the playoffs so that they can shine?

The league as a whole may be "rational" in how it distributes its energies, without requiring that the players within that league make the same distribution. e.g. A guy trying to make the league for the first time can usually be counted on to exert their utmost coming right out of the gate, right? Whereas a veteran may take a more measured approach to their season. Plus, you may see players turn it down a notch after clinching playoff spots, that sort of thing.

What surprises me here Eric, and makes me feel that you're reaching, is that this phenomenon is pretty much explicitly articulated, and then visibly and measurably acted upon each season. Namely, when coaches sit star players, or players with lingering injuries, during games that are less important. We see this across every sport, and pretty much in every game. We see pitchers rested, sluggers sat, 2nd string QB's come in when the game's out of reach, 4th liners getting ice-time, rookies brought up to "give them a look."

These are just very simple facts on the ground. The coaches say what they're going to do, then they do it, and you can measure it.

Why is there any less doubt when an athlete skips a tournament, e.g. a golfer or track athlete or tennis player, and says it's because it's less important, or they want an injury to heal? There isn't even a debate here. We all know it happens.

But extend it again, to an NHL player, and suddenly it's to be debated? Seems to me that pretty much all the bios of pro athletes say that they then mentally or physically took a break to prepare for "the big game," or rested an injury or passed up on a fight or held back from pushing for that 8th goal or whatever. But suddenly this is treated as some weird delving into psychology to be mashed down by odd comments about guys coasting, and dumping the load on their teammates? Not sure why.

i.e. Why do you feel you need to argue that every player gives exactly 100% every night of every season. This quite simply strikes me as silly, and more importantly... not necessary to any serious case you're making.

Avatar
#14 not norm ullman
October 04 2012, 10:08AM
Trash it!
0
trashes
Props
0
props
Eric T. wrote:
1. The NUMBER of players out-performing in the playoffs may match up with your estimate of variance, but this in no way explains their IDENTITIES. That is, the concept of a player being "clutch" is in no way eliminated simply because only a limited number of players have it.

/FLYERROB'd

Variance is an unavoidable fact of nature. If you had variance plus a distribution of clutch talent, you would see larger extremes -- people who were clutch _and_ ran hot, people who choke _and_ ran cold.

It's true that I can't explain or predict the identities of the people who are at the high end of the curve. But the point is that if the distribution almost exactly fits what you would predict by variance alone, then it's unlikely that anyone can, because any clutch talents that the players have seems to be swamped by variance.

If you take a hundred coins, flip them all ten times, and get a distribution that exactly matches what the probabilities would suggest, you probably shouldn't conclude that the ones that came up heads 8/10 times are special head-seeking coins.

Except, and this is a broader point, you are also seeing a distribution on defence, as well as in nets. i.e. If there are also clutch defenceman, and clutch goalies, in roughly the same proportions as amongst forwards, then we're going to see the extremes reeled back in. Same would hold for clutch defensive forwards, but you get my drift - that "clutch" may also apply to defensive play. Which makes the whole argument a bit weaker, if I may say so.

As for the coin-tossing example, can we maybe not do that? It's a tad patronizing. For example, coin-flipping probabilities maybe don't explain why the Germans were 0 for 2 in world wars last century. That is, real-world behaviours, motivations, skills and performances - even if they fall within a particular pattern - may not be adequately explained, or explained away simply by fitting a statistical pattern.

Avatar
#17 speeds
October 04 2012, 10:18PM
Trash it!
0
trashes
Props
0
props

If it's true that the game isn't quite the same in the regular season vs. post-season, perhaps that would explain a portion of what some call "clutch"?

More hooking and holding as the refs tend to call less, is it impossible that a player isn't "clutch" or "not clutch", but just a somewhat better player the way the game is called in the regular season NHL vs. the post-season NHL?

Avatar
#19 not norm ullman
October 05 2012, 07:47AM
Trash it!
0
trashes
Props
0
props

@Eric T.

To take one point at a time.

Your response on the "not 100%" argument is to repeat the word "coasting," argue that all players must be doing it equally, and then to conclude that if a player does "coast," it's not noble. I guess I'm not sure what to do with an argument like that.

On the other side, I laid out various individual sports where the principals stated that they "dialled it back" or took a break, etc. I also gave the example of coaches doing the same thing to players. And then, mentioned player bios, where they say they did it.

Now, there is no rational backing to your case that this would need to be done equally across players. A Toronto Maple Leaf (sadly) rarely finds himself in a position where his team is a playoff lock. Other teams do. So we have situational differences for starters. We also have the case where players pick up an injury, and try to rest it through part of the season so they can be as strong as possible for the playoffs. This too is rational. It is also repeatedly stated, as coaches make their decisions.

Now. Why would they do this? Because, in many cases, playoff games are more important than regular season. Not always, but on the whole. In fact, I think you're building this into your argument in the first place. - recognition that playoff games are more important.

Which means, there are multiple perfectly rational motivations for dialling it down at points during the regular season, arguments which are then articulated by the players and coaches.

Which also means, it is entirely possible, rational and in fact, even likely, that a player can perform at a higher level than they do across an average of regular season games.

Avatar
#20 not norm ullman
October 05 2012, 08:37AM
Trash it!
0
trashes
Props
0
props

You said, "Sorry, I didn't mean to be patronizing. I was under the impression that you were missing the point and so I was trying to clarify -- I didn't realize that you understood that the results are almost entirely explained by inexorable variance when you tautologically pointed out that variance wouldn't tell us _which_ players landed at the high end of the curve."

Okkkkkk. Not entirely sure how to take that, but overall the comment seems to me to be slightly too clever for its own good - especially after the coin-flipping thingie. Therefore, some "clarifying" talk from my end.

1. We'd all like to raise the quality of discussion around hockey. Some quality statistical work would help. You do a lot of that. Thanks.

2. But I've noted a bit of a tendency amongst some stats people, when applying themselves to a field, to start dismissing things. First this, then that, and soon, many many things, as "not existing." And why? Well, because they ran a study which couldn't find the thing in question. And which, in their minds, meant that it could all be explained by randomness, or - how did you put it? - "inexorable variance."

Except that, during my years at Oxford, I saw a fair chunk of statistical studies. And an awful lot of them revealed, "nothing beyond the workings of mere chance." Except, what was often found later on was that there were multiple variables at work, cancelling and reinforcing and making things difficult to get at. i.e. Things can be complex. And especially when it concerns human motivations and their links to behaviours and other outcomes. Which is kindof what we have here, eh?

Which meant that, yes, some of science involves demolishing myths built up by humans to explain the workings of other forces. There's lots of science like that, in fact.

But there's lots of another kind of science, as well. Science which takes things that appear to have no other explanation than chance, and discovering that, oh my, there were actually a number of things going on beneath the hood.

3. Which... is what I was trying to open up. And why I proposed no single answer. I argued that there are multiple reasons for a player to vary their performance, depending on the game and situation - whether it's size, energy levels, injury, team position, etc. I think that's a fairly strong argument. It's complex, however, because you'd have to tease some of these things apart. Which is a big job.

I also mentioned the idea that there are talent levels across players, and how the worst may be unlikely to score much at all when placed against better players, how some of the better will feed off these opportunities in the regular season, no matter the score, and how the playoffs are likely to reduce the talent differentials. I'm not sure there's anything too too controversial in this either. Except that, it's going to make it harder to tease out again.

We then have the questions of there possibly being something "clutch" in some goalies and defencemen. Not sure how this plays through. But for a brief starter serving, a number of complexities there.

4. And then, to make it all worse, we have the fact that you never really did much to define what you were after to begin with, eh? I mean, "clutch?" Is a guy who scores when his team is up 5-1 in the 3rd period of a playoff series they lead 3-0 as "clutch" as a 7th game goal? Not sure this gets at the "clutch" thing people actually talk about.

5. Then there's what might actually make up "clutch" behaviour. Part of it strikes me as involving a response to high pressure - stressful - situations. I gave some examples from numerous other fields, and other sports, where stress clearly knocked some people out of the fray. Somehow, I would expect that this exists in hockey as well.

6. So, take just that little list of things. You're arguing that there pure chance - sorry, "inexorable" chance - explains the distribution. And thus, that there is no room left for these other things. Oh, other than "coasting." It could be that.

Or, it could be that some players vary their effort and focus across different games and situations... and that some respond quite poorly to stress... while some are forced to expend too much energy in certain games and series... and some get beat down by the physical grind... while some don't have the talent to play against tougher competition, while for others it doesn't matter much... and that some moments, even within the playoffs are more important than others.

Taken all together, I would entirely expect that some players would outperform their expected results, while others will do more poorly. Which is kindof what a good coach is looking for, preparing for, as he goes through the season and into the playoffs. Who's tired or hurt, who can't handle the stress, who doesn't have the skill to beat their man, and who's ready for more.... in any particular moment.

And damn it all, when there are multiple variables at work like that, I'm just not sure what the distribution would look like.

Shorter? When you say "the results are almost entirely explained by inexorable variance" you're moving from an attempt at useful stats work into the realm of bad science. Reel it back a notch. You can easily say that, at this level of analysis... there wasn't much in the way of an an obvious "clutch" factor at work.... blah blah blah. What's wrong with that?

Avatar
#22 garret9
October 12 2012, 05:24AM
Trash it!
0
trashes
Props
0
props

This discussion was brought up over at HFBoards... The guy didn't believe me; he argued and gave "examples". I showed him this; he said it was stupid and gave "examples". I laughed at him; he made a poll http://hfboards.hockeysfuture.com/showthread.php?t=1270251

Comments are closed for this article.