The typical debate is set up as a choice between having 4 Cups and 3 Conn Smythes vs. having the career wins and shutouts record. I think all of those things are strongly team dependent and are correspondingly overhyped, so I'm not even going to bring them up. Having done a lot of research lately into shots against by goaltender, I have come up with some estimates of shot prevention effects, and would like to use those numbers to review the save percentage vs. league average results I posted some time ago.
I prefer to rank goalies by taking their save percentages and adjusting for all the important factors. Therefore, I have included a shot prevention adjustment (by giving goalies credit for shots they likely prevented and subtracting from the goalie's shot total for shots they likely had a hand in creating; if you haven't read any of my work on the topic check out this post estimating the effect for Brodeur) and a team defence adjustment based on the quality of scoring chances against, which gives an adjusted save percentage that can be compared against league average to account for differences in league scoring levels. I also ran the numbers for a few others (Belfour, Joseph, and Luongo) who should also be in the conversation about the best goalies in recent years.
These adjustments were mainly derived by comparing performance by backup goalies while on the same team to their performance on different teams, and using that that to assess the overall team context. There are many variables still in play in that type of comparison, such as quality of backups, quality of teams, small seasonal sample sizes, and the usual year-to-year random fluctuations, so I mostly took what the numbers said but did make a few subjective adjustments. They included adjusting downward the shot quality numbers for Roy and Brodeur, who both had results that were far overstated because their backups played a lot of games on bad teams in other cities, and increasing Brodeur's shot prevention effect and lowering Luongo's based on a more detailed comparative analysis of their performance relative to their teammates.
Here are my estimates:
Patrick Roy: 2.5% easier shot quality, 1 shot created/gm
Martin Brodeur: 5% easier shot quality, 1 shot prevented/gm
Dominik Hasek: 2.5% harder shot quality, 0.5 shots created/gm
Roberto Luongo: 0% harder shot quality, 1 shot created/gm
Ed Belfour: 0% harder shot quality, 1 shot prevented/gm
Curtis Joseph: 2.5% harder shot quality, 0.75 shots created/gm
That leads us to the following results:
|Goalie||Save %||SA/60||GAA||SA Prev||Adj SA||SA AdjS%||SQ AdjS%|
These numbers allow us to compare the adjusted save pecentages to league average and calculate a number of goals above average for each goalie that combines quantity and quality:
After letting some of the air out of Roy's numbers, Dominik Hasek reigns supreme at the top of the rankings list. The surprising result is how close to the rest of the pack Patrick Roy ends up, although these numbers are from the regular season only.
Ed Belfour's ranking is also a point of interest. I think nearly everyone would rank him 4th out of this group but well behind the top 3. That's probably underrating Belfour. I have long thought that there was much more of a gap between Hasek and Brodeur than between Brodeur and Belfour, and these numbers certainly support that.
I ran the numbers for the playoffs as well, but without the same ability to compare backup results those numbers are entirely based on guesswork. I'm sure the shot quality factors were different, since the opposition is tougher and the number of games played on each team in the regular season is different than the number of games for each team in the playoffs. Belfour likely faced easier shots in the playoffs because of all the deep playoff runs he had in Dallas, and Roy's difficulty probably rose because of strength of competition and more games played in Colorado. This is an imprecise measure to be sure, but I'd guesstimate the playoff goals above average numbers as something like the following:
1. Patrick Roy, 74.6
2. Ed Belfour, 51.8
3. Dominik Hasek, 31.3
4. Martin Brodeur, 24.3
5. Curtis Joseph, 22.3
6. Roberto Luongo, 8.1
Obviously the inputs determine the results in a study like this, so there can and should be considerable debate about the adjustments I've made. I am not claiming to have ironclad proof on any of them and I am open to the possibility of any or all of them being wrong. The most discussion-worthy adjustments, and possibly the most difficult ones to estimate, are the strength of team defensive adjustments for Patrick Roy and Martin Brodeur. I think it is pretty clear both goalies benefitted from their teams, but the question is how much.
I think Roy's context is probably a little tougher to evaluate. Playing for Montreal in the 1980s was probably relatively similar to playing in New Jersey in the 1990s, with great team defence and fewer than average power plays by the other team. Then Roy was traded to Colorado, who had a different style of play. The shot quality data we do have available from '03 and '04 indicates that New Jersey was much better than Colorado at preventing opposing scoring chances, which is primarily why I gave Roy a different overall shot quality rating than Brodeur.
Overall, I think it is pretty clear that Dominik Hasek is the best goalie of the last 20 years (and I would argue the best goalie ever). As far as Brodeur vs. Roy goes I'd still take Roy. I think Roy's peak is well ahead of Brodeur's (illustrated graphically here), and Roy has also been better in the playoffs, but it does seem like the more work I do on St. Patrick the more question marks are raised about how much of his success was from team factors.
If Brodeur keeps up his current level of play for several more seasons (the previous three seasons account for over one-third of Brodeur's career goals over average total), then he will be at least moving into the conversation with Roy. If we are able to more precisely drill down to the relative team effects and adjust for them maybe we'll find that the team factors between the two aren't that different after all, and in that case the debate could get very interesting by the time Brodeur's career is done.