Filed under:

# Are Goalies Binomial?

Save percentage varies some from shot to shot. Does it differ enough to matter?

We know that Even Strength Save Percentage varies from shot to shot.
It varies with shot distance.
It depends on whether there is a screen or not.
It depends on whether the goalie has to move side-to-side or not.
Rebounds are harder to stop than first shots.

I've been taking it as a given that these things even out over the long run and I have been using the binomial distribution equations to approximate goalie performance.  I realized that this might not be true so I decided to test it out.

We have play-by-play data for nearly every NHL game from 2007-08 to present.  I went through and pulled the game-by-game 5v5 performance of each goaltender for 2007-08 through 2012-13.  (Note that this is 5v5 performance not ES.)  I pulled the 20 goaltenders with the most shots faced.  I also pulled Jaroslav Halak and Brian Elliott.  I ran a Monte Carlo for each goaltender.  I simulated 10,000 seasons.  For each season, I chose 55 games at random from their population of games played.  I combined the chosen games and calculated the save percentage.  Across those 10,000 seasons I calculated the average number of shots faced and the standard deviation of the save percentage.

To compare it to the binomial distribution, I took the average number of shots faced and the average save percentage and calculated the standard deviation as sqrt (sp*(1-sp)/N).  The results:

 Name Saves SavePct MonteCarlo Binomial Ratio LUNDQVIST 1218.5 0.924 0.008311 0.007580 1.096 KIPRUSOFF 1169.5 0.918 0.008268 0.008035 1.029 MILLER 1198.3 0.916 0.008825 0.008016 1.101 BRYZGALOV 1200.2 0.925 0.007887 0.007605 1.037 WARD 1209.8 0.921 0.008297 0.007765 1.068 FLEURY 1206.3 0.912 0.008179 0.008147 1.004 LUONGO 1148.9 0.924 0.008601 0.007804 1.102 VOKOUN 1162.8 0.922 0.008034 0.007860 1.022 NABOKOV 1225.4 0.923 0.007576 0.007623 0.994 BRODEUR 1067.7 0.918 0.008717 0.008378 1.040 PRICE 1152.0 0.922 0.008144 0.007888 1.032 RINNE 1187.9 0.922 0.007503 0.007770 0.966 QUICK 1185.4 0.930 0.007418 0.007433 0.998 BACKSTROM 1039.8 0.922 0.008496 0.008313 1.022 HOWARD 1321.4 0.921 0.008200 0.007405 1.107 ANDERSON 1192.4 0.920 0.007855 0.007876 0.997 HILLER 1120.8 0.920 0.008097 0.008098 1.000 THEODORE 1170.3 0.922 0.008330 0.007845 1.062 NIEMI 1342.7 0.926 0.006723 0.007134 0.942 LEHTONEN 1077.0 0.916 0.008580 0.008462 1.014 HALAK 1179.3 0.920 0.007549 0.007878 0.958 ELLIOTT 1176.9 0.930 0.007503 0.007415 1.012 Average 1179.9 0.921 0.008102 0.007852 1.032

Shots is the average number of shots faced in these simulated seasons.
SavePct is the calculated average Save Percentage in these simulated seasons.
Monte Carlo is the observed SD of these Save Percentages.
Binomial is the predicted SD under the Binomial Distribution for that number of shots faced and Save Percentage.
Ratio is the ratio of Monte Carlo/Binomial.

So, in general, these goalies are about 3% more variable in the Monte Carlo than would be predicted by the Binomial Distribution.  Next, I repeated the Monte Carlo using 15 games.

 Name Saves SavePct MonteCarlo Binomial Ratio LUNDQVIST 332.6 0.924 0.016243 0.014519 1.119 KIPRUSOFF 318.8 0.918 0.015978 0.015405 1.037 MILLER 326.9 0.916 0.016981 0.015352 1.106 BRYZGALOV 327.1 0.925 0.015081 0.014563 1.036 WARD 330.1 0.921 0.016048 0.014860 1.080 FLEURY 328.9 0.912 0.015333 0.015583 0.984 LUONGO 312.9 0.924 0.016478 0.014986 1.100 VOKOUN 317.0 0.922 0.015519 0.015055 1.031 NABOKOV 334.3 0.923 0.014371 0.014612 0.984 BRODEUR 291.2 0.918 0.016904 0.016039 1.054 PRICE 314.3 0.922 0.015887 0.015129 1.050 RINNE 324.0 0.922 0.015799 0.014875 1.062 QUICK 323.2 0.929 0.014356 0.014259 1.007 BACKSTROM 283.6 0.922 0.016368 0.015933 1.027 HOWARD 360.2 0.921 0.015799 0.014179 1.114 ANDERSON 325.8 0.920 0.015036 0.015070 0.998 HILLER 305.8 0.921 0.015467 0.015454 1.001 THEODORE 319.6 0.922 0.015981 0.015033 1.063 NIEMI 365.5 0.926 0.013026 0.013676 0.952 LEHTONEN 293.9 0.916 0.016625 0.016196 1.027 HALAK 321.7 0.920 0.014303 0.015084 0.948 ELLIOTT 321.1 0.930 0.014379 0.014213 1.012 Average 321.8 0.921 0.015664 0.015039 1.042

Here these goalies are about 4% more variable in the Monte Carlo than predicted. Finally, I repeated the Monte Carlo using 5 games.

 Name Saves SavePct MonteCarlo Binomial Ratio LUNDQVIST 110.8 0.924 0.027935 0.025253 1.106 KIPRUSOFF 106.5 0.917 0.028436 0.026735 1.064 MILLER 109.2 0.916 0.029669 0.026603 1.115 BRYZGALOV 109.2 0.925 0.026190 0.025262 1.037 WARD 110.1 0.921 0.027996 0.025707 1.089 FLEURY 109.9 0.913 0.027067 0.026932 1.005 LUONGO 104.5 0.923 0.028331 0.026012 1.089 VOKOUN 105.3 0.922 0.027060 0.026166 1.034 NABOKOV 111.7 0.922 0.025287 0.025358 0.997 BRODEUR 97.4 0.918 0.029311 0.027732 1.057 PRICE 104.6 0.922 0.027174 0.026297 1.033 RINNE 108.0 0.922 0.027549 0.025803 1.068 QUICK 107.7 0.929 0.025328 0.024713 1.025 BACKSTROM 94.6 0.921 0.028727 0.027648 1.039 HOWARD 120.1 0.920 0.027549 0.024703 1.115 ANDERSON 108.6 0.919 0.026303 0.026216 1.003 HILLER 102.0 0.920 0.026758 0.026825 0.997 THEODORE 106.4 0.922 0.027941 0.026069 1.072 NIEMI 122.0 0.926 0.022564 0.023676 0.953 LEHTONEN 98.0 0.916 0.029373 0.028037 1.048 HALAK 107.3 0.920 0.024672 0.026183 0.942 ELLIOTT 106.9 0.929 0.025464 0.024824 1.026 Average 107.3 0.921 0.027327 0.026087 1.047

Once again, these goalies are only about 5% more variable in the Monte Carlo than would be predicted by the Binomial Distribution.

This suggests it is, in fact, reasonable to use the Binomial Distribution when predicting save percentage and creating confidence intervals.