There has been much debate if Carlyle's defensive system that claims to pushes shots has been beneficial and effective to the leafs. That is, the typical comment is that even though the leafs are being outshot, they are giving up more "...
There has been much debate if Carlyle's defensive system that claims to pushes shots has been beneficial and effective to the leafs. That is, the typical comment is that even though the leafs are being outshot, they are giving up more "lower quality" chances from a further distance and minimizing quality scoring chances close up.
How To Measure Effectiveness of Carlyle's Strategy?
Below I will try to quantify the "effectiveness" of Carlyle's strategy. There are a number of imperfect ways to measure effectiveness of the defensive system. Two of them are
1. How well the system minimizes shot against (SA) as a proxy for minimizing scoring chances against
2. How well the system maximizes the ratio of shots for to shots against (SF/(SF+SA) as a proxy for how well the team "outchancing" the competition.
Using Ninjastat's supershot tracker the shots for and against the leafs in road games for 2012 and 2013 were outputted. Note that, fenwick or corsi events (blocked shots, shots on goal and missed shots) from various distances are not available and so in this study we will use shots on goal as a "proxy" for Fenwick Close (actually scoring chances) in this analysis.
Shots Against
Skinnyfish has done a thorough analysis of shots against here. My focus was primarily looking at the shot differential change in 2012 vs 2013 leaf team. This analysis didn't compare earlier leaf teams as the team roster was different (Kubina, Kaberle, Poni, Antropov, Moore, Stajan etc) who were actually quite effective at outshooting competition. Looking at the 2012 vs 2013 leaf results (see spreadsheet) we see this current team has given up fewer shots as a percentage below 30 feet at a cost of more shots at a distance greater then 30 feet. On a per game basis, they have also given up fewer shots between 0 to 10 feet and 20 to 30 feet at a cost of more shots between 10 to 20 feet and dramatically more shots beyond 40 feet.
The leafs have been effective at decreasing the proportion and quantity of shots from quality area (close up) at the cost of a large increase of shots from low quality areas (far away). This fact is probably known to most or at least evident to those that looked into it with some detail. In fact, even HNIC and Mirtle and a number of bloggers have discussed this topic. What is not answered is does it matter and if so how much? More on that later.
Shot Differential
The alternate method to review is shot differential defined as SF/(SF+SA) and develop a proxy for scoring chances (or Fenwick Close) at each of these distances. This is shown in the table for 2012 vs 2013 teams. Again, the leafs have improved their SF% (shot differential) by 3.7% within 0 to 10 feet. That is impressive. However, any gains in fewer shots and outscoring that Carlyle and the team gained at 0 to 10 feet is completely overwhelmed by the team being massively out shot beyond 10 feet. The problem again is does this tradeoff of allowing fewer high quality scoring chances close in at the cost or more low risk scoring chances from further out matter? And if so how much?
Weights and SH%
At this point, we should mention that all typical shot differentials are unweighted in that they treat shots, corsi or fenwick events as the same regardless if they are from 0 to 10 feet or beyond 30 feet. This treatment would be questioned by most coaches and many fans as a shot from 0 to 10 feet is much more dangerous then a shot beyond 30 feet. However shot differential don't care and they could possibly penalize a team who allows more shots from outside. To get by the problem with the using raw shot data, most NHL coaches track scoring chances. The scoring chance team data is something which is not available though some bloggers track and publish this data so that is a possible avenue to investigate.
The method I use to determine the "effectiveness" of Carlyle's system and adjust shots for their quality is from this gem of entry b