On "Why NHL teams should never play a tired goalie"

*Unless you are a NHL team, in which case it doesn't really matter what you do.

James Mirtle had an article in the Globe and Mail last Monday titled  "Why NHL teams should never play a tired goalie" He concluded that "Statistically speaking, it’s almost never a good decision to start a goalie two nights in a row." I suppose most people read it and thought "Wow, that makes a lot of sense". I, being a contrarian, thought, "Geez! I wonder where that analysis went off the rails?"

Mirtle bases his argument on an article by Eric T. from 2013 (Whoomp! There it is!)   Eric concludes "we can say with reasonable confidence that a rested goalie stops about 1 percent more shots than a tired goalie, even when both of them are playing in front of a team that played on the previous day." The data for this is summarized in a couple tables.

Team playing back-to-back	Games started	Saves made	Shots faced	Save percentage
Goalie played previous game	97	2340	2624	.892
Goalie rested previous game	106	2719	2982	.912

Team playing back-to-back	Games started	Saves made	Shots faced	Save percentage
2013 tired goalies	97	2340	2624	.892
2013 rested goalies	106	2719	2982	.912
2011-12 tired goalies	186	4843	5344	.907
2011-12 rested goalies	229	6291	6894	.913
Total for tired goalies	283	7183	7968	.901
Total for rested goalies	335	9010	9876	.912

Statistical significance?

Not everybody worries about things like statistical significance, but I do, so let's rewrite the first table, adding in the lower and upper ends of the 95% Confidence Intervals.

Situation	Games	Saves	Shots	Save Pct	lower95	upper95
2013 tired goalies	97	2340	2624	0.892	0.880	0.904
2013 rested goalies	106	2719	2982	0.912	0.902	0.922

The 0.912 for rested goalies is outside the Confidence Interval for tired goalies (and vice versa) so the difference is statistically significant.

A huge problem

Let's look at the data from 2011-12

Situation	Games	Saves	Shots	Save Pct	lower95	upper95
2011-12 tired goalies	186	4843	5344	0.907	0.899	0.915
2011-12 rested goalies	229	6291	6894	0.913	0.906	0.920

The Confidence Interval for tired goalies overlaps the 0.913 of rested goalies. Tired goalies and rested goalies don't differ significantly! In fact

Situation	Games	Saves	Shots	Save Pct	lower95	upper95
2013 tired goalies	97	2340	2624	0.892	0.880	0.904
2011-12 tired goalies	186	4843	5344	0.907	0.899	0.915

Tired goalies in 2013 are significantly different from tired goalies in 2011-12! Why? I don't know. If I were really interested in the question, I would look into it. However, given that I just proved they are significantly different, you really shouldn't lump the two groups together.

However, just for the sake of argument, let's suppose that somehow, against all logic, lumping them together is the right thing to do and some sort of magical blind squirrel finding an acorn of truth thing has happened. Rested goalies really do perform at 0.912 and tired goalies really do perform at 0.901.

Binomial Approximation – One Game

Let's look at one game. Teams give up just under 30 shots a game. Plugging that into the Binomial Distribution we get

Goaltender	Shots	Save Pct	Expected Goals	Std. Dev.	95% CI
Rested	30	0.912	2.640	1.552	+/- 3.04
Tired	30	0.901	2.970

The expected difference here in goals allowed is about one third of one goal. The 95% Confidence Interval for either save percentage is about +/- 3 goals. Randomness >> Difference. Converting these numbers into a p-value gives a result of 0.83162 (all tests are two-sided).

Since the difference is not statistically significant, you cannot say that rested goaltenders are different from tired goaltenders. If you do, you are making a "Type 1 Error".

To better show how similar these two groups are, I ran a small simulation. 10,000 games of 30 shots against. One goalie has a save percentage of 0.901 the other 0.912. Of the 10,000 games the "tired goalie" won 3556 games, 1775 games were tied, and the "rested goalie" won 4669 games. It is an admittedly biased way of looking at it, but the "tired goalie" finished as good or better than the "rested goalie" 53.3% of the time.

Binomial Approximation – Full Season

Sporting Charts says "The average NHL team has 14.6 back-to-back games during the 2013-14 season. The New Jersey Devils have the greatest number of back-to-back games with a total of 22 with the Carolina Hurricanes coming in with the second most at 20. While a total of three teams are tied for the fewest at 10 - Avalanche, Sharks, Jets. " Let's use 10 games and 20 games to get an estimate for the impact of rested versus tired over a season. First, 10 games:

Goaltender	Shots	Save Pct	Expected Goals	Std. Dev.	95% CI
Rested	300	0.912	26.400	4.907	+/- 9.62
Tired	300	0.901	29.700

Here the difference in expected goals is 3.3. The magnitude of randomness is 9.6. Randomness > Difference. Here the p-value is 0.50126.

I reran the simulation as 10,000 seasons of 300 shots against. Of the 10,000 games the "tired goalie" gave up fewer goals 2963 times, 530 were tired, and the "rested goalie" gave up fewer 6507 times.  Now, 20 games:

Goaltender	Shots	Save Pct	Expected Goals	Std. Dev.	95% CI
Rested	600	0.912	52.800	6.939	+/- 13.60
Tired	600	0.901	59.400

Here the difference in expected goals is 6.6. The magnitude of randomness is 13.6. Randomness is still > Difference. Here the p-value is 0.34154.

Once again, the difference is not statistically significant and you cannot say that rested goaltenders are different from tired goaltenders.

Finally, I reran the simulation as 10,000 seasons of 600 shots against. Of the 10,000 games the "tired goalie" gave up fewer goals 2460 times, 319 were tired, and the "rested goalie" gave up fewer 7221 times.

Break-even point

How many shots would the goalie have to face for the magnitude of the difference in performance would be equal to the magnitude of the randomness? N is the number of shots, Pr is the probability that a rested goalie makes a save, and Pt is the probability that a tired goalie makes a save

|N*(1-Pt) – N(1-Pr)| > 1.96*sqrt(Pr*(1-Pr)*N)

0.011*N > 0.5854*sqrt(N)

sqrt(N) > 53.22

N > 2832 shots-against.

Real Life

In real life, shot quality varies. This widens the confidence intervals. Goalies have good games and bad games. This also widens the confidence intervals. There is also going to be some differences in shot mix (ES/PK/PP) from game to game. In "Are Goalies Binomial", I looked at 5v5 shooting. Over 15 games, I found that the real world standard deviations averaged about 4% larger than the theory predicted (real SD = 1.04 binomial SD). Including the shot mix issue is only going make the SD bigger. Let's estimate that it's 5% larger here.

In the 10 game season, the p-value becomes 0.52186. In the 20 game season, the p-value becomes 0.36502.

If the real-world standard deviations are about 5% wider than the binomial equation suggests, we see that the break-even point rises to 3123 shots-against.

Conclusions

There are problems with the data that suggest it is unreasonable to pool tired goalies from 2013 with tired goalies from 2011-12.

Ignoring that and pooling the data anyway shows little practical difference between the play of tired goaltenders and the play of rested goaltenders.

If a team plays hundreds of games, and the goalies faces significantly more than 2800 shots, there is probably a benefit to playing the rested goaltender.

In a single NHL game, there is no benefit whatsoever to playing the rested goaltender versus playing the tired goaltender.

In a season of 10 or 20 NHL games, there is not a significant difference between tired goaltenders and rested goaltenders. Saying rested goaltenders are "better" is a Type 1 Error. From a practical standpoint, there is a substantial probability that a tired goaltender will play better than a rested goaltender, even summed over an entire season.