Hot Goalies?

A couple days ago, on FiveThirtyEight.com, Neil Payne posted an article called   "Who’s the ‘Hottest’ Goalie in the NHL Playoffs?" Neil pointed to Henrik Lundqvist and Corey Crawford as being hot. A few hours later, Crawford decided that the third period would be a great time to demonstrate regression to the mean. As a result, Neil had to issue an apologia of sorts. Unfortunately for Neil, the initial article has a number of significant problems.

Neil says "save percentage is more important than shots per game, shooting percentage or shots allowed". If you think about it, save percentage and shooting percentage are mirror images, as are shots per game and shots allowed. The only way for this statement to make sense is if he means that, coming into a series, the teams' previous performance on these metrics influences the outcome of the series. When I looked at Regular Season PDO and Playoff Success we can see that Neil's statement is wrong. "Even though the goalie (skill) part of PDO persists somewhat into the playoffs, it only explains about 2.5% of the total variability seen. "

"The only thing that matters is shot differential." Over a small sample (like a 7 game series), randomness might allow save percentage to outweigh shot differential at times, but that's not the way I would bet.

Accepting the Alternative Hypothesis

To prove something statistically, you need to set up a Null Hypothesis and an Alternative Hypothesis. Here the Null Hypothesis would be "All goalies ARE the same" and the Alternative would be "All goalies ARE NOT the same". Before you can say that A <  B or C > D, you have to disprove the Null Hypothesis.

Neil says "We can measure how "hot" an NHL goalie has been in the playoffs by comparing his postseason performance (measured by save percentage) to what we would have expected from his previous statistics and the strength of the teams he’s faced"

Several problems here. One, Neil has jumped to accepting the Alternative Hypothesis without disproving the Null. He just assumes that the "difference" he sees is meaningful. It isn't.

Call:

glm(formula = savegoal ~ name, family = binomial(logit), data = Playoffs14)

Analysis of Variance Table

Response: savegoal

Variable Df Sum Sq Mean Sq F value Pr(>F)

name 23 1.117 0.048545 0.6755 0.8739

Residuals 3732 268.183 0.071860

So if nobody is different from normal, Lundqvist and Crawford can't be.  At the point Neil wrote the article (prior to the second Hawks-Kings game) goalies had faced 4662 shots and made 4258 saves for a 0.913 save percentage. Lundqvist had faced 452 shots, Crawford 403. At 452 shots, a 0.913 goalie has a 95% CI of 0.884 to 0.937. Lundqvist was at 0.934. He was within randomness of average. At 403 shots, the 95% CI is 0.881 to 0.939. At 0.933, Crawford was also within randomness of average.

At ES, Lunqvist had 355 saves on 377 shots. 0.942 Average is 0.9224. 95% CI for 377 shots is 0.891 to 0.947. Crawford had 306 on 329. 0.930 95% CI for 377 shots is 0.886 to 0.947.

Now, If there are goaltenders who have above-average talent, Lundqvist is one of them. For his career, he is 0.928 ES and 0.920 overall. Plug those numbers into the Binomial Confidence Interval generator and you get 0.898 to 0.952 at ES and 0.891 to 0.944 Overall. There's a roughly 18% chance that Lundqvist would have 355 or more saves on 377 shots just by being Lundqvist. Calling his play "hot" is a stretch.

Shooting Percentage and Team "Strength"

"Strength of opponent" is an inaccurate way to describe team shooting percentage. The notion that any team has a shooting percentage that is better than average is wrong. Lots of people have looked at this over the years. I looked at this back in the fall. (This obviously didn't include the 2013-14 season.) The Penguins were maybe a little above average. Maybe. A couple teams were maybe a little below average. Most teams were just average.

This season the Blues had an overall shooting percentage of 9.9% and an ES shooting percentage of 8.6%. They weren't "better" than average, they were a little luckier than average. Adjusting a Crawford's expected save percentage on the basis of the Blues' prior shooting percentage is unwarranted.

Another problem with using Overall shooting percentage (as with using Overall save percentage) is shot mix. Two teams might have identical ES and PP shooting percentages but if one team has more PP shots they will have (slightly) different Overall shooting percentages.

Seeing hot goalies

There's no question that goalies get hot (or cold). It's hard to see "hot" because a normal goalie on a normal day is going to save 92.0% of the ES shots he sees. Add in small sample sizes and a dash of randomness and it's really hard to see many performances that fall above the confidence interval. But there certainly are some. Ben Scrivens with a 59 save shutout. Check. Jaroslav Halak closing out the Capitals in 2010 by giving up 3 goals in 3 games on 134 shots. Check. Lundqvist and Crawford in the 2014 Playoffs? Not so much.