Hot Goalies?

Warm, not hot. - Andy Marlin-USA TODAY Sports

There's no question that goalies get hot (or cold). It's just hard to see it in small samples.

A couple days ago, on FiveThirtyEight.com, Neil Payne posted an article called   "Who’s the ‘Hottest’ Goalie in the NHL Playoffs?" Neil pointed to Henrik Lundqvist and Corey Crawford as being hot. A few hours later, Crawford decided that the third period would be a great time to demonstrate regression to the mean. As a result, Neil had to issue an apologia of sorts. Unfortunately for Neil, the initial article has a number of significant problems.

Neil says "save percentage is more important than shots per game, shooting percentage or shots allowed". If you think about it, save percentage and shooting percentage are mirror images, as are shots per game and shots allowed. The only way for this statement to make sense is if he means that, coming into a series, the teams' previous performance on these metrics influences the outcome of the series. When I looked at Regular Season PDO and Playoff Success we can see that Neil's statement is wrong. "Even though the goalie (skill) part of PDO persists somewhat into the playoffs, it only explains about 2.5% of the total variability seen. "

"The only thing that matters is shot differential." Over a small sample (like a 7 game series), randomness might allow save percentage to outweigh shot differential at times, but that's not the way I would bet.


Accepting the Alternative Hypothesis

To prove something statistically, you need to set up a Null Hypothesis and an Alternative Hypothesis. Here the Null Hypothesis would be "All goalies ARE the same" and the Alternative would be "All goalies ARE NOT the same". Before you can say that A <  B or C > D, you have to disprove the Null Hypothesis.

Neil says "We can measure how "hot" an NHL goalie has been in the playoffs by comparing his postseason performance (measured by save percentage) to what we would have expected from his previous statistics and the strength of the teams he’s faced"

Several problems here. One, Neil has jumped to accepting the Alternative Hypothesis without disproving the Null. He just assumes that the "difference" he sees is meaningful. It isn't.

Call:

glm(formula = savegoal ~ name, family = binomial(logit), data = Playoffs14)

Analysis of Variance Table

Response: savegoal

Variable Df Sum Sq Mean Sq F value Pr(>F)

name 23 1.117 0.048545 0.6755 0.8739

Residuals 3732 268.183 0.071860

So if nobody is different from normal, Lundqvist and Crawford can't be.  At the point Neil wrote the article (prior to the second Hawks-Kings game) goalies had faced 4662 shots and made 4258 saves for a 0.913 save percentage. Lundqvist had faced 452 shots, Crawford 403. At 452 shots, a 0.913 goalie has a 95% CI of 0.884 to 0.937. Lundqvist was at 0.934. He was within randomness of average. At 403 shots, the 95% CI is 0.881 to 0.939. At 0.933, Crawford was also within randomness of average.

At ES, Lunqvist had 355 saves on 377 shots. 0.942 Average is 0.9224. 95% CI for 377 shots is 0.891 to 0.947. Crawford had 306 on 329. 0.930 95% CI for 377 shots is 0.886 to 0.947.

Now, If there are goaltenders who have above-average talent, Lundqvist is one of them. For his career, he is 0.928 ES and 0.920 overall. Plug those numbers into the Binomial Confidence Interval generator and you get 0.898 to 0.952 at ES and 0.891 to 0.944 Overall. There's a roughly 18% chance that Lundqvist would have 355 or more saves on 377 shots just by being Lundqvist. Calling his play "hot" is a stretch.


Shooting Percentage and Team "Strength"

"Strength of opponent" is an inaccurate way to describe team shooting percentage. The notion that any team has a shooting percentage that is better than average is wrong. Lots of people have looked at this over the years. I looked at this back in the fall. (This obviously didn't include the 2013-14 season.) The Penguins were maybe a little above average. Maybe. A couple teams were maybe a little below average. Most teams were just average.

This season the Blues had an overall shooting percentage of 9.9% and an ES shooting percentage of 8.6%. They weren't "better" than average, they were a little luckier than average. Adjusting a Crawford's expected save percentage on the basis of the Blues' prior shooting percentage is unwarranted.

Another problem with using Overall shooting percentage (as with using Overall save percentage) is shot mix. Two teams might have identical ES and PP shooting percentages but if one team has more PP shots they will have (slightly) different Overall shooting percentages.

Seeing hot goalies

There's no question that goalies get hot (or cold). It's hard to see "hot" because a normal goalie on a normal day is going to save 92.0% of the ES shots he sees. Add in small sample sizes and a dash of randomness and it's really hard to see many performances that fall above the confidence interval. But there certainly are some. Ben Scrivens with a 59 save shutout. Check. Jaroslav Halak closing out the Capitals in 2010 by giving up 3 goals in 3 games on 134 shots. Check. Lundqvist and Crawford in the 2014 Playoffs? Not so much.

X
Log In Sign Up

forgot?
Log In Sign Up

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to SB Nation going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to SB Nation going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join St. Louis Game Time

You must be a member of St. Louis Game Time to participate.

We have our own Community Guidelines at St. Louis Game Time. You should read them.

Join St. Louis Game Time

You must be a member of St. Louis Game Time to participate.

We have our own Community Guidelines at St. Louis Game Time. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9355_tracker