How good is Jake Allen?

My apologies to Dan (Buffa82). This probably looks like a retort to his post but it is really not meant as such. I had been working on it for a while but, like most of my articles, never quite got it to where I was completely happy with it. When I went to post this I saw Dan's article on the same topic. So I held off on it for a little longer.

Figuring out the exact talent level for a goaltender statistically is take a lot of data. The problem is the data itself. Saves and goals are "discrete outcome data". A shot is either a save or a goal, nothing in between. Most of the data we work with on a daily basis is continuous. How tall? How heavy? How expensive? Compared to working with continuous outcome data, discrete data is a pain.

Here's an example. Suppose Brian Elliott weighs 92 kilos and Jake Allen weighs 93 kilos. You don't know that yet and you want to find out who is heavier. Even if they are in different cities, you could just weigh Elliott on a couple scales, take the average, weigh Allen on a couple scales, take the average. You're done! And the answer is almost certainly correct.

Suppose Elliott has an ESSP talent of 0.920 and Allen 0.930. Again, you don't know that but you want to find out who is better. You would have to have each one of them face 4000 or more shots to have a reasonable chance of telling them apart, and even then there's a decent chance you might actually come up with the wrong answer.

The approaches

There are two basic approaches to answering the question "How good is Jake Allen?". One approach is called "frequentist" and the other one is "Bayesian". A frequentist would simply look at the data. A goalie who faced 1000 shots and made 937 saves would have an observed save percentage of 0.937. A frequentist would say that observation is our best estimate of his underlying talent. A Bayesian would take that number as a starting point then add additional information to get a revised estimate. In this case, a Bayesian knows that no goalie ever has had a talent of 0.937, few goalies have a talent of 0.930, and typical NHL starters have a talent of 0.920 – 0.925. Thus the Bayesian estimate would be something less than 0.937, perhaps 0.927.

Both approaches have their strengths and weaknesses. I tend to be a frequentist for a variety of reasons. One of the reasons is that Bayesian estimates are often difficult to do. A qualitative Bayesian approach is relatively easy. We intuitively understand that, in the above example, the estimate of 0.937 is probably high. The precise quantitative answer is potentially tricky to derive.

Even Strength

Coming into the season, Allen had seen 1030 ES shots and made 943 saves for a 0.916 ESSP. This season he has 605 shots and 563 saves for 0.931, moving his career ESSP to 0.921.

Season	ES Shots	ES Saves	ES Save Pct
2012-2013	291	265	0.911
2014-2015	739	678	0.917
2015-2016	605	563	0.931
Total	1635	1506	0.921

Quality Adjusted Even Strength

AC Thomas and the team at War On Ice are breaking shots down by location. A goalie’s save percentages for high risk, medium risk, and low risk shots are plugged into a standardized average workload to get his adjusted save percentage.

season	Sv%	AdSv%
20122013	90.61	90.99
20142015	92.01	92.15
20152016	92.96	92.69
Total	92.10	92.13

The average NHL goaltender saves 97.4% of the low risk shots, 92.8% of medium risk shots and 82.5% of high risk shots. Allen looks about average here too

season	G.L	S.L	L SP	G.M	S.M	M SP	G.H	S.H	H SP
20122013	6	116	0.95082	1	72	0.986301	19	63	0.768293
20142015	12	279	0.958763	19	189	0.908654	25	177	0.876238
20152016	3	160	0.981595	4	86	0.955556	17	75	0.815217
Total	21	555	0.963542	24	347	0.93531	61	315	0.837766

Could Allen be an elite goalie?

Could Allen be a 0.930 ES goalie who has just had some bad bounces?

> binom.test(1506, 1635, p = 0.93)

Exact binomial test

data: 1506 and 1635

number of successes = 1506, number of trials = 1635, p-value = 0.1597

alternative hypothesis: true probability of success is not equal to 0.93

95 percent confidence interval:

0.9069580 0.9337086

sample estimates:

probability of success

0.9211009

Indeed, he could be. Although our best frequentist estimate of his talent is 0.921, he has faced few enough shots that his true talent could be anywhere from 0.907 to 0.934.

Qualitative Bayesian Estimates

A couple of additional bits of information would factor into a Bayesian estimate. 0.920 goaltenders are a lot more common than 0.930 goaltenders. In turn, 0.910 goalies are a lot more common than 0.920 goalies. So if he isn't a 0.920-ish goalie, he is vastly more likely to be a 0.910 goalie getting good bounces than a 0.930 goalie getting bad bounces.

Secondly, if you look at the goaltenders who either retired with elite-level numbers (Hasek, Thomas, Chechmanek) and the active guys who are there (Rask, Lundqvist, Price, Luongo), none of them were near 0.916 after 1030 shots. That's not to say it can't be done, just that the elite goalies we have seen so far have started their career much better than that.

Summary

Jake Allen appears to be an NHL average goaltender although his limited number of shots faced gives us a wide confidence interval for his underlying talent.

For comparison, so far in his Blues career Brian Elliott has seen 3003 ES shots. He has an ESSP of 0.9284 and a QASP of 0.9254.

I'm all for a 1A/1B. I'm all for riding the hot hand. And, quite frankly, I hope Allen continues to perform at an elite level. I just don't have the data to justify saying that Allen is better than Elliott.