clock menu more-arrow no yes mobile

Filed under:

Are Goalies Binomial?

Save percentage varies some from shot to shot. Does it differ enough to matter?

By CanesAndBluesFan
Dilip Vishwanat

We know that Even Strength Save Percentage varies from shot to shot.
It varies with shot distance.
It depends on whether there is a screen or not.
It depends on whether the goalie has to move side-to-side or not.
Rebounds are harder to stop than first shots. 

I've been taking it as a given that these things even out over the long run and I have been using the binomial distribution equations to approximate goalie performance.  I realized that this might not be true so I decided to test it out.

We have play-by-play data for nearly every NHL game from 2007-08 to present.  I went through and pulled the game-by-game 5v5 performance of each goaltender for 2007-08 through 2012-13.  (Note that this is 5v5 performance not ES.)  I pulled the 20 goaltenders with the most shots faced.  I also pulled Jaroslav Halak and Brian Elliott.  I ran a Monte Carlo for each goaltender.  I simulated 10,000 seasons.  For each season, I chose 55 games at random from their population of games played.  I combined the chosen games and calculated the save percentage.  Across those 10,000 seasons I calculated the average number of shots faced and the standard deviation of the save percentage.

To compare it to the binomial distribution, I took the average number of shots faced and the average save percentage and calculated the standard deviation as sqrt (sp*(1-sp)/N).  The results:

Name

Saves

SavePct

MonteCarlo

Binomial

Ratio

LUNDQVIST

1218.5

0.924

0.008311

0.007580

1.096

KIPRUSOFF

1169.5

0.918

0.008268

0.008035

1.029

MILLER

1198.3

0.916

0.008825

0.008016

1.101

BRYZGALOV

1200.2

0.925

0.007887

0.007605

1.037

WARD

1209.8

0.921

0.008297

0.007765

1.068

FLEURY

1206.3

0.912

0.008179

0.008147

1.004

LUONGO

1148.9

0.924

0.008601

0.007804

1.102

VOKOUN

1162.8

0.922

0.008034

0.007860

1.022

NABOKOV

1225.4

0.923

0.007576

0.007623

0.994

BRODEUR

1067.7

0.918

0.008717

0.008378

1.040

PRICE

1152.0

0.922

0.008144

0.007888

1.032

RINNE

1187.9

0.922

0.007503

0.007770

0.966

QUICK

1185.4

0.930

0.007418

0.007433

0.998

BACKSTROM

1039.8

0.922

0.008496

0.008313

1.022

HOWARD

1321.4

0.921

0.008200

0.007405

1.107

ANDERSON

1192.4

0.920

0.007855

0.007876

0.997

HILLER

1120.8

0.920

0.008097

0.008098

1.000

THEODORE

1170.3

0.922

0.008330

0.007845

1.062

NIEMI

1342.7

0.926

0.006723

0.007134

0.942

LEHTONEN

1077.0

0.916

0.008580

0.008462

1.014

HALAK

1179.3

0.920

0.007549

0.007878

0.958

ELLIOTT

1176.9

0.930

0.007503

0.007415

1.012

Average

1179.9

0.921

0.008102

0.007852

1.032

Shots is the average number of shots faced in these simulated seasons.
SavePct is the calculated average Save Percentage in these simulated seasons.
Monte Carlo is the observed SD of these Save Percentages.
Binomial is the predicted SD under the Binomial Distribution for that number of shots faced and Save Percentage.
Ratio is the ratio of Monte Carlo/Binomial.

So, in general, these goalies are about 3% more variable in the Monte Carlo than would be predicted by the Binomial Distribution.  Next, I repeated the Monte Carlo using 15 games.

Name

Saves

SavePct

MonteCarlo

Binomial

Ratio

LUNDQVIST

332.6

0.924

0.016243

0.014519

1.119

KIPRUSOFF

318.8

0.918

0.015978

0.015405

1.037

MILLER

326.9

0.916

0.016981

0.015352

1.106

BRYZGALOV

327.1

0.925

0.015081

0.014563

1.036

WARD

330.1

0.921

0.016048

0.014860

1.080

FLEURY

328.9

0.912

0.015333

0.015583

0.984

LUONGO

312.9

0.924

0.016478

0.014986

1.100

VOKOUN

317.0

0.922

0.015519

0.015055

1.031

NABOKOV

334.3

0.923

0.014371

0.014612

0.984

BRODEUR

291.2

0.918

0.016904

0.016039

1.054

PRICE

314.3

0.922

0.015887

0.015129

1.050

RINNE

324.0

0.922

0.015799

0.014875

1.062

QUICK

323.2

0.929

0.014356

0.014259

1.007

BACKSTROM

283.6

0.922

0.016368

0.015933

1.027

HOWARD

360.2

0.921

0.015799

0.014179

1.114

ANDERSON

325.8

0.920

0.015036

0.015070

0.998

HILLER

305.8

0.921

0.015467

0.015454

1.001

THEODORE

319.6

0.922

0.015981

0.015033

1.063

NIEMI

365.5

0.926

0.013026

0.013676

0.952

LEHTONEN

293.9

0.916

0.016625

0.016196

1.027

HALAK

321.7

0.920

0.014303

0.015084

0.948

ELLIOTT

321.1

0.930

0.014379

0.014213

1.012

Average

321.8

0.921

0.015664

0.015039

1.042

Here these goalies are about 4% more variable in the Monte Carlo than predicted. Finally, I repeated the Monte Carlo using 5 games.

Name

Saves

SavePct

MonteCarlo

Binomial

Ratio

LUNDQVIST

110.8

0.924

0.027935

0.025253

1.106

KIPRUSOFF

106.5

0.917

0.028436

0.026735

1.064

MILLER

109.2

0.916

0.029669

0.026603

1.115

BRYZGALOV

109.2

0.925

0.026190

0.025262

1.037

WARD

110.1

0.921

0.027996

0.025707

1.089

FLEURY

109.9

0.913

0.027067

0.026932

1.005

LUONGO

104.5

0.923

0.028331

0.026012

1.089

VOKOUN

105.3

0.922

0.027060

0.026166

1.034

NABOKOV

111.7

0.922

0.025287

0.025358

0.997

BRODEUR

97.4

0.918

0.029311

0.027732

1.057

PRICE

104.6

0.922

0.027174

0.026297

1.033

RINNE

108.0

0.922

0.027549

0.025803

1.068

QUICK

107.7

0.929

0.025328

0.024713

1.025

BACKSTROM

94.6

0.921

0.028727

0.027648

1.039

HOWARD

120.1

0.920

0.027549

0.024703

1.115

ANDERSON

108.6

0.919

0.026303

0.026216

1.003

HILLER

102.0

0.920

0.026758

0.026825

0.997

THEODORE

106.4

0.922

0.027941

0.026069

1.072

NIEMI

122.0

0.926

0.022564

0.023676

0.953

LEHTONEN

98.0

0.916

0.029373

0.028037

1.048

HALAK

107.3

0.920

0.024672

0.026183

0.942

ELLIOTT

106.9

0.929

0.025464

0.024824

1.026

Average

107.3

0.921

0.027327

0.026087

1.047

Once again, these goalies are only about 5% more variable in the Monte Carlo than would be predicted by the Binomial Distribution. 

This suggests it is, in fact, reasonable to use the Binomial Distribution when predicting save percentage and creating confidence intervals.