Category: Hockey Strategy

Protecting the Blue Line and Driving Break-outs: Repeatability and Impact of Exits & Entries Against

Posted in Defensive Value, Hockey Strategy, Player ValuationTagged , , , , , 3 Comments

This article is being co-posted on NHLNumbers as well as on my own site, OriginalSixAnalytics.com. Find me @michael_zsolt on twitter. If you are able, please provide some support to public data providers Corey Sznajder (@ShutDownLine), Corsica and Puckalytics

Every team in the NHL is constantly striving to find market inefficiencies to capitalize on towards the goal of creating value and ultimately winning games. Whether done by finding undervalued assets or developing a set of sustained competitive advantages – every team should always have these concepts in mind.

The analytics community has widely acknowledged we are not as good at evaluating defensemen as we are at evaluating forwards – in part due to having fewer effective metrics available. As a result, a number of people have spent time trying to expand our knowledge in this area – as any new development would be sure to present a strong market opportunity to capitalize on.

A particular focus has been on applying Eric Tulsky’s historical zone entry and exit analysis to defensive evaluation. Given controlled entries into the offensive zone (carry-ins) have been proven to generate more shots than uncontrolled entries (dump-ins) – intuitively, if a defender can prevent controlled zone entries against his team, that should reduce shot attempts against.

There has been a lot of great work done on this subject, including articles by the likes of Sean Tierney, Dom Luszczyszyn, JenLC and more. Dim Filipovic at Sportnet has written a number of times on the value of tracking the percentage of entries against that are controlled, as well as the number of controlled zone exits that a defender creates. This type of work has broken ground recently in evaluating defensemen. However, in an article late last season Dimitri summarizes the current gaps in the analysis as well:

“It’s important to retain perspective on all of these figures. At this point they’re much more descriptive than predictive, in the sense they’re telling us how a certain player did in the games he’s already played, but not necessarily helping us forecast how he’ll do in the future. There’s also the fact that most players only have [only] ~10 games worth of data to their name, which leaves plenty of room for inexplicable swings in either direction.”

So far, the best evidence to support the repeatability of these metrics is on the zone entries against side – where Eric Tulsky used a full season of tracked data for the Philadelphia Flyers’ defensemen to test this. His results are here. While this analysis definitely hints at a relationship, I suspect Eric himself would say a sample of only six defensemen should be seen only as a ‘first step’ that can built upon in the future.

As such, my goal in this article is to test, replicate and expand Tulsky’s work, as well as to hopefully broaden it to some uncharted territory. I also hope to encourage others to recognize the amazing data being made available by Corey Sznajder’s tracking efforts – which you can access here and here. Hopefully others will decide to support Corey by purchasing his data, and do further work to expand our knowledge in these areas as well.

Definitions

Let’s start with some important definitions:

  • Zone Entry Attempt Against (ZEA) – Any time an opponent is attempting to bring the puck into your defensive zone; it is registered against the player (e.g. defenseman) who is most closely defending the play across the blue-line
  • Controlled ZEA – When the opponent gains access to your defensive zone by successfully carrying-in or passing-in the puck, retaining possession
  • ‘Break up’, or ‘Failed Entry Against’ – When the defender is able to prevent the entry from gaining access to the zone, turning over possession
  • Zone Exit Attempt – Any time a skater is attempting to move the puck across his own blue line, whether carried, passed or dumped/cleared out
  • Zone Exit with Control – Any time a skater successfully carries the puck out or passes it out of his own zone, retaining possession

A simple way to think about preventing controlled ZEA’s is that it is the act of ‘Protecting the Blue-line’. Likewise, individual controlled zone exits represent players who are strong puck-movers, or who excel at ‘Driving Break-outs’.

Key Questions

With respect to preventing controlled ZEA, being able to break-up plays, and driving controlled zone exits (whether as a % of total, or per 60 minutes of play), my key questions are as follows:

  • Predictive – Are these repeatable skills? Is past performance a reliable indicator of future performance?
  • Relationship with Goals – Do these metrics drive improvement in Goals For Differential (GF%), by either increasing goals for or decreasing goals against?
    • If not, do these metrics at least help to drive the other inputs that translate into GF%? (I.e. xGF%, Shots, Corsi-For%, etc.)

“Protecting the Blue Line” – Zone Entry Defense

Let’s start with protecting the blue line: preventing controlled entries against, or breaking up entry attempts. To be clear, a lower percentage is better – the lower the proportion of controlled entries against a defenseman, the fewer goals and shots against you would expect. (In the interest of time, I have focused more so on ZEA, rather than break up %.)

Repeatability

To test repeatability I have done a ‘split-half’ test to see how well the 1st half of a season of data predicts the second half. All data is from Corey’s 2013-2014 tracking – the only full season of publicly-available tracked entry data. I have only included defensemen with over 150 entries-against in each half, for a total sample of N=148. (Note – the mid-season mark wasn’t technically the 41st game as Corey only captured zone entry defense for the final 60-65 games).

1-protecting-blue-line-repeatabilityAs you can see above, mitigating controlled entries against % is a highly repeatable skill with a relationship (R^2) of 0.37 between first half and second half results. I think this is a very exciting finding, as it shows the recent work of Dimitri, Sean and others is spot on – and that past performance should be predictive of future performance.

Relationship with goals

The next question is ‘does this improve GF%’? Intuitively I think we all expect it would, but it is still important to check. To evaluate this I tested the relationship (R^2) between controlled entry against (% and per-60) and a wide range of metrics: goal differential, goal and shot against rates, and ‘rel’ stats – i.e. stats that adjust goal and shot rates to be more focused on individual, rather than team-level, play.

2-zea-v-goals-table-incld-rel

As you can see above, there are some interesting results. First, many on-ice stats are significantly impacted by these metrics. In particular, ZEA with control (% and per 60) show a meaningful relationship with xGF%, CF% and CA/60, which are all at an R^2 around or above 20%.

However, it is interesting to note that when you focus on individual impact (i.e. rel stats), the relationship is considerably weaker than with unadjusted on-ice stats, which are more influenced by teammates. As such, while preventing controlled ZEA is repeatable, and does meaningfully impact goals and shot rates – there may also be team/system impacts that are difficult to separate out from the individual contribution.

Last, let’s ‘zoom in’ one on of the cells above: % Controlled Entry Against vs xGF%. the chart below illustrates the direct relationship between reducing controlled entries against and improving expected goals.

3-zea-v-goals

 “Driving Break-outs” – Zone Exits with Control

Now, let’s look at the zone exit side, or ‘Driving Break-outs’. Comparing the Exit data to ZEA, there are a couple nuances to mention: first, the data is slightly more limited in this area. Corey provides raw tracked data in 2013-2014 for all offensive zone entries and for all ZEA – which allowed me to separate it into split-half components. Unfortunately, I checked with Corey, and for the 2013-2014 zone exits the tracking was done offline and aggregated at the end. As such, there is no full season of raw data that we can use to do a repeatability test.

On the positive side, Corey has been providing the raw data for his new 2016-2017 tracking work which is well underway (maybe 10-15+ games tracked per team thus far). As such, by the end of the season, we should have another full year of data – and I will at that point confirm whether or not Controlled Zone Exits are a repeatable skill. My guess is that this will ultimately be shown to be the case, as simply from watching, certain defensemen seem to be natural puck-movers (Karlsson, Josi), and others seem to be less so (Polak).

Last, the 2013-2014 zone exit data lacked some key elements (which Corey has done well to include in the 2016-2017 work). The old data did not have a broad ‘failed’ zone exit category, nor did it show what Corey now calls ‘Transitional Plays’ – i.e. getting the puck out of the zone while surrendering possession via a clearing attempt. As such, we can’t say the proportion of the time an individual’s exit attempts were successful. However, we are still left with some very valuable data – individual Zone Exits/60 and Controlled Exits /60 – as defined above.

Relationship with goals

So – if we assume for now that Controlled Exits are a repeatable skill (and not simply randomness) – do they have a strong relationship with goal events?

4-zexits-v-goals-incl-rel

The chart above shows some very interesting findings. First, zone exits with control have a fairly meek impact on GF% and xGF% directly. This isn’t entirely surprising, as exiting your own zone with control can only get you so far towards scoring. However, there is a more relevant relationship between Controlled Exits/60 and CF% – with an R^2 of 15.5%. This shows that having a defender who is strong puck mover can definitely begin to ‘move the needle’ on winning the shot-differential battle.

Lastly, what I found most intriguing here were the Rel Stats – where the metric is adjusted for how a player’s team does with him on the ice versus off the ice (i.e. isolating individual contribution). While teammates can play a bigger role in helping a defensemen prevent controlled ZEA, Zone Exits are actually the opposite – both CF/60rel and CF%rel have a ~19% relationship (R^2) with Controlled Exits/60, stronger than CF% alone. This demonstrates that strong individual play from puck-moving defensemen can have a major CF% impact strictly from their ability to exit the zone with control.

Before wrapping up, let’s ‘zoom in’ again on the relationship between Controlled Exits/60 and CF%rel.

5-exits-60-v-cfrel

The chart above shows how a Defensemen’s individual controlled exits/60 have a meaningful impact on his ability to be a net-contributor to his team’s Corsi-For %.

Tying it all together – what these two sets of analysis tell us is that a great puck-moving defender can have a substantial impact on his team’s shot-attempts, as well as overall shot attempt differential, through his ability to Drive Break Outs. Further, a defenseman who excels at Protecting the Blue-line will have the greatest impact when surrounded by teammates and playing within a system that does so as well.  Naturally, the greatest value will be found in acquiring players who show both these skills , even if they don’t necessarily contribute significantly in terms of goals and points.

Conclusion & Practical Applications for Teams

As should now be clear, these types of new metrics for defensive evaluation are very interesting and important. They appear to be repeatable skills (at least for ZEA), and they do have a material impact on a team’s xGF% or at the very least, its CF%. While they are visible to anyone watching a defenseman closely – just like batting average or save % – only over dozens of games of tracking can we truly quantify and understand how well a player performs on each.

So – practically speaking – what should NHL or teams at other levels do about this? Gather the data! At a minimum, every team in the league should be tracking their own player’s performance on these metrics in order to better game plan, develop, and observe improvements or deficiencies over time. Further if any team ever took a systematic effort to track these on a league-wide basis (like – say – the Florida Panthers…), it would almost certainly present a clear competitive informational advantage that can help them build value over the long term. 

Such a team could have a greater understanding of the value of their own players, and also directly identify defenders around the league that may have a big impact on their team’s results, despite low goal or point totals. At this point, teams should be compelled to begin acting on this knowledge to build an ‘edge’. If not, eventually they may be doing so just to prevent themselves from falling too far behind.

 

Does Every Team Need a Troy Terry? Quantifying the Value of a ‘Shootout Specialist’

Posted in Hockey Strategy, Player Valuation, Salary Cap Management, ShootoutTagged , , , , , 2 Comments

This article is being co-posted on NHLNumbers as well as on my own site, OriginalSixAnalytics.com. Find me @michael_zsolt on twitter.

 Troy Terry. Jonathan Toews. T.J. Oshie. These names are all recognizable in part because of a heroic shootout performance they each have put on for their country. Troy Terry was the most recent, scoring 3 goals in the USA World Junior semi-final win over Russia, and ultimately being the sole scorer in the championship game, winning Team USA a gold medal over Team Canada.

Due to shootouts being the subject of many conversations over the last couple weeks, I found myself wondering: for an NHL team, what exactly is it worth to have a shootout specialist in your line up? Should every team have one? Or is it a ‘nice to have’, merely a secondary consideration when evaluating players?

So – let’s see if we can figure it out.

Quantifying the Value of a ‘Shootout Specialist’

In order to answer this, I have looked at three areas:

  1. Simulated outcomes of two league average teams in the shootout
  2. Simulated outcomes of adding a ‘Shootout Specialist’ to a league average team
  3. Estimating the resulting win probability improvement, and quantifying it in terms of standings points/contract dollar value

In order to answer these questions, I have drawn data about the league average shootout (SO) Sh% and Sv% from the last three full seasons courtesy of Hockey-Reference (2013-14, 2014-15 and 2015-16), summarized below:

  • 31% SO Sh% (% of shot that are goals)
  • 69% SO Sv% (% of shots that are saved, or (1 – 31%))

In the next two sections, I will look at a hypothetical team and their Shootout Win Probability %. Let’s call the team in question ‘Team A’, and for now we will treat both Team A and their opponents as league average.

League Average versus League Average

Given shootouts, like wins, are zero-sum (i.e. same number of wins and losses), the average winning percentage on a league-wide basis will always be 50/50. As such, as a starting point, I have created the table below summarizing the round by round probabilities to reach that 50%.

lg-avg

As you would expect, the probability pretty cleanly comes out to 50% winning probability for Team A. If we define a ‘Round Win’ as when one team scores more goals in that round than their opponent, then we get the following distribution in each round:

  • 21% chance that Team A ‘wins’
  • 57% chance of a ‘draw’
  • 21% chance that the Opponent ‘wins’ the round

Now, naturally this is pretty straightforward, as the overall outcome will inevitably be 50/50. However, if we want to know what happens when we start changing players – and having these probabilities update dynamically – it gets a bit more complicated. To address part (b) – adding a Shootout Specialist to Team A, in the next section – I created a tool that simulates the outcome of a shootout, given a particular Sh% and Sv% for each team/goalie.

After running 1000+ simulations, the probabilities here represent the average outcomes, and I am happy to provide more detail on the actual calculations if anyone is interested.

Adding a ‘Shootout Specialist’ to Team A

Now, looking at ‘regular’ SO shooters – i.e. those who have 10 or more attempts in the sample – I will define a ‘Shootout Specialist’ as the players who are in the 90th percentile of these ‘regulars’. Here are all of the players who meet this level over the past three seasons:

list

I will come back to the specific individuals, but as you can see, this cohort of players will score on roughly 52% of their shootout attempts – an impressive number.

So let’s imagine that Team A decides to go out of its way to sign or trade for a Shootout Specialist – what happens if  we add him to our prior probability table?

specialist

With the simplifying assumption that a team will put their specialist in frequently (i.e. every shootout), you can see that a ‘Shootout Specialist’, or a top 10% shooter in the league will increase his team’s likelihood to win a shootout from 50% to 58%. This increase is a far cry from a guaranteed win, but also not an insignificant jump.

Now what exactly is that worth?

In order to convert this 8% improvement to standings points, and ultimately a contract dollar value, there are a couple more steps. Given 2016-2017 is one of the first seasons where the 3-on-3 is well-understood and teams have clearly formed strategies, let’s use the first half of the 2016-2017 NHL season as our proxy for a ‘typical’ year going forward. This half has shown roughly 4 games going to shootout per team so far – giving us ~8 or so for the year.

value

So assuming teams will typically be in ~8 shootouts over the year, and that a shootout specialist will increase their probability of winning by ~8% (50% to 58%), this illustrative scenario implies a shootout specialist would contribute an additional 0.68 shootout wins per season.

Given a shootout win is worth a single standings point, adding a SO specialist to a league average team is worth approximately 0.68 standings points; or about 1/3 of a win ‘win’ per year. Using @Behindthenet’s win value estimate of $2.80M per win (against the salary cap), this skill set is worth about $950K in contract value.

Going back to our first question, of ‘if every team needs a shootout specialist?’ – I think the answer here is likely ‘no’. On a big picture basis, NHL teams should have higher priorities than finding their own Troy Terry or T.J. Oshie – especially since teams shootouts are only relevant during the regular season. However, I would also say that SO capabilities are an important secondary consideration that teams and agents should still absolutely have in mind when negotiating player contracts.

For anyone who noticed the lesser known names on our list of specialists – like Brandon Pirri and Jacob Josefson – these guys might just be examples of ‘Moneyball’ contracts that New York (previously Florida) and New Jersey keep around in no small part for their shootout capabilities. Although both are RFAs, each team has them signed at low risk, 1-year contracts worth $1.1M for each player (Source: CapFriendly). Putting aside Josefson’s injury issues – assuming these guys contribute almost anything in the typical run of play – at #3 and #4 in the league in Shootout Sh% at 56%, they each are likely paying for their cap space in Shootout goals alone.

Other Considerations

Before closing, I should add a couple last qualifiers to my analysis, above:

  • For teams that are currently well below league average in shootouts, the ‘marginal’ value of a specialist is significantly greater than I have shown above, as their ‘baseline’ starting point is much lower
  • Further, any team who has focused on acquiring one or more Specialists should naturally be playing to their own strengths – potentially employing a more conservative strategy in 3-on-3 OT in order to drag extra games into shootouts (again – creating additional value beyond the average 8 games shown above)

Some areas for analysis that others may want to explore further include the value of other ways a team could go about increasing their shootout win %. What if a team had three specialists? Or what if they found a strong shootout goaltender? While each of these merit much further research, my short answers are that:

  • Adding a goalie will naturally have a dis-proportionately large effect, because he gets to face every shot against. However, given that teams have only one starting goalie (versus 12 forward roster slots), the shootout is less likely to be a goaltending priority
  • If a league average team were to have multiple Shootout Specialists, their win percentage in my analysis above would jump even further. My current estimate is that this would reach 66% with two specialists – meaning 1.3 shootout wins over a season, 1.3 standings point, or an aggregate $1.8M in salary cap value

Conclusion

In conclusion – while being a Shootout Specialist certainly increases the value a player adds to his team, I wouldn’t say it is a characteristic that every team ‘needs’ to have in their roster. That said, the New York Rangers and New Jersey Devils may have made some stealth, intelligent signings between Pirri and Josefson, who are some of the highest-value ‘depth’ Shootout Specialists a team could add. Further, when very high caliber players like T.J. Oshie (or Troy Terry down the road) head into their contract negotiations, I certainly would recommend both teams and agents do some work to understand exactly what they should be paying for this skill.