Tired Goalies and Rested Goalies

When I wrote "On: Why an NHL team should never play a tired goalie", my first draft ended with the phrase:

There are problems with the data that suggest it is unreasonable to pool tired goalies from 2013 with tired goalies from 2011-12.

I should have gone with my first instinct and just ended there. But I wanted to make a point. So I went ahead and pooled the goalies. Heads exploded. Twitter erupted. So I felt like I needed to take a broader look at the data and clarify what is going on.

I took all the data from 2007-08 through 2013-14. I pulled all the back-to-back games. I looked at the second game of the pair. I noted whether the goalie was at home or away, and whether he was rested or tired. I used the RTSS files and pulled the 5v5 shots. This eliminates the ES/PK/PP shot mix issue.

Again, these are 5v5. The weird data for "2013" (here 2012-13) we saw in the previous article is gone.

Year	Status	Saves	Goals	Save Percentage
2007	rested	4336	346	0.926
2007	tired	3730	310	0.923
2008	rested	4279	364	0.922
2008	tired	3873	323	0.923
2009	rested	5311	474	0.918
2009	tired	3469	286	0.924
2010	rested	5101	441	0.920
2010	tired	3829	333	0.920
2011	rested	4863	418	0.921
2011	tired	3326	269	0.925
2012	rested	2928	254	0.920
2012	tired	2225	190	0.921
2013	rested	6079	556	0.916
2013	tired	2723	226	0.923
Total	rested	33028	2868	0.920
Total	tired	23458	1957	0.923

There is no meaningful difference between rested goalies and tired goalies. Tired goalies certainly are not worse than rested goalies.

Statistical Significance?

Nope.

> prop.test(goalies)

2-sample test for equality of proportions with continuity correction

data: goalies

X-squared = 1.6812, df = 1, p-value = 0.1948

alternative hypothesis: two.sided

95 percent confidence interval:

-0.007243215 0.001451793

sample estimates:

prop 1 prop 2

0.9201025 0.9229982

Practical Significance?

Nope. Play either goalie, whenever, wherever. Doesn't matter.

Let me reinforce this: if there is not statistical significance, you should not say there is a difference. Saying there is a difference is a "Type 1 Error".

Type 2 Error?

Sure, maybe. But who cares?

Maybe there really is a small effect but 61,000 or so shots just doesn't give you enough power to see it. If so, so what? If you can't see an effect in pooled data of 30 teams over 7 seasons, you aren't going to see it in 20 games, and you sure as hell aren't going to see it in a single game.

Home and Road

Nothing there either.

Where	Saves	Goals	Save Percentage
Home	18760	1660	0.919
Away	37726	3165	0.923

> prop.test(goalies)

2-sample test for equality of proportions with continuity correction

data: goalies

X-squared = 2.7918, df = 1, p-value = 0.09475

alternative hypothesis: two.sided

95 percent confidence interval:

-0.0084847988 0.0007008886

sample estimates:

prop 1 prop 2

0.9187071 0.9225991

What went wrong in 2012-13?

No idea. I'm not seeing it at 5v5. In the previous article we saw tired goalies gave up 284 goals on 2624 shots, a 0.892 save percentage. They should have given up about 231. That's 53 extra goals.  Either that's a one-in-a-zillion outlier or something went wrong in the collection of the data.

Conclusions

There is no statistical or practical difference between the play of tired goaltender and the play of rested goaltenders.

The data at 5v5 in 2012-13 is similar to the other 6 years in this data. The data in the original article has to reflect a mammoth statistical outlier in situations other than 5v5 or there was a problem in the collection of the data.