Month: April 2016

Wading into a Minefield: Why On-ice Sv% Stats May Not Be Completely Meaningless

Posted in GoaltendingTagged , , , , , 2 Comments

This article is being co-posted on the new Puckalytics Blog as well as on my own site, OriginalSixAnalytics.com. Find me @michael_zsolt on twitter.

 For those who don’t know – there is a hotly contested debate in the hockey analytics community about whether players are able to influence their teams’ save percentages. Many hockey analysts have looked into the subject, and some of our best and brightest have come out on opposite sides of the debate (e.g. Eric Tulsky, Garret Hohl, David Johnson and Kyle Dubas). As you can imagine, it is not a straightforward concept to pin down.

The core of the debate centers on the idea that for a statistic to be meaningful, it has to be repeatable – there needs to be evidence that players can consistently outperform/underperform on the metric over time. The current consensus is that there has been no evidence found to date to show that impacting on-ice Sv% is a repeatable skill. Instead, when you look across the league as a whole, what you find are outcomes that essentially resemble randomness.

Having been a goaltender my whole life, I never felt right about this conclusion. All of the analysis I have seen was sound, but my ‘gut’ (twitter explosion alert) told me that there must be some variance in the type of chances a goalie faces with different skaters in front of him. As a result, I have taken an attempt at reviewing past work on the subject, and trying to dig a little deeper – in particular, by trying to control for some additional variables that could be creating ‘noise’ in our findings.

Although I think I have reached some interesting conclusions, before getting into those results I’d like to openly invite the online ‘peer-review’ process. Only once others have pressure-tested these findings and concluded the work is (or is not) statistically meaningful will I be fully confident in the legitimacy of this outcome. However, hopefully this work is able to begin to move the conversation forward on the topic of whether or not relSv% and Sv%RelTM are relevant metrics in player analysis.

(For those who don’t know – Sv%RelTM represents a team’s save percentage when a particular player is on the ice, compared to how his teammates performed without him. It is expressed as a percentage above or below zero (much like CF%RelTM) – representing the amount he is above or below his teammates’ time-weighted average. Due to data availability, I have focused much of my analysis on Sv%RelTM, and to be clear, relSv% and Sv%RelTM are slightly different metrics. However, the findings shown here are almost exactly the same when using relSv%).

Prior Analysis of On-ice Sv% / relSv%

To start, I’d like to re-visit some of the historical work done on the subject. Eric Tulsky, a very well-regarded member of the hockey analytics community, wrote the initial ‘book’ on the subject. Tulsky demonstrated that on a league-wide basis, neither forwards nor defensemen are able to consistently display above/below average performance on the metric. He showed this in his 2013 article by using on-ice Sv% itself (rather than a relative stat), and by comparing a player’s ‘first three seasons’ of on ice Sv% to that player’s next three seasons. If this type of approach revealed a strong correlation, it would suggest players can consistently impact their team’s Sv% – for better or worse.

Tulsky’s analysis of forwards from 2007-2010 and 2010-2013 is shown below (his analysis of defensemen reaches largely the same conclusion):

Tulsky

(Source: Eric Tulsky)

As you can see, the plot appears to be scattered almost evenly and has an R2 value of less than 1%. This result suggests there is almost no correlation between past and future impact on on-ice Sv%, and that it is not a ‘repeatable’ skill.

However, following Tulsky’s work, there continued to be articles suggesting that certain NHL players do in fact have consistent impacts to on-ice Sv% relative to their teammates over time – arguably based on ‘anecdotal’ evidence. As a result, another well-known member of the community, Hockey Graph’s Garret Hohl, conducted an updated piece of analysis to show that – even when using a relative metric (here he used relSv%) – there remains limited evidence of it being a metric that is fundamentally able to predict future performance. I have copied his output below:

Holh

(Source: Garret Hohl)

As you can see, Garret used a similar methodology and demonstrated that the historical relSv% of a defenseman is not a reliable predictor of how that defender will do in the future – coming out at an R2 of only 2.6%.

Areas to Dig Further

However – as I suggested at the beginning – while these are absolutely valid findings based on the samples used, personal on-ice experience has kept this topic on my mind. Many writers often debate the wide range of factors that could be causing ‘noise’ in these results, such as:

  • Player usage
  • Quality of teammate
  • Quality of competition
  • Team-level systems
  • Quality of goaltender
  • Etc.

However, just because we have factors that could be causing noise in the findings does not mean we have a statistically valid statistic. Thinking through the lens of a goaltender, I hypothesized two specific factors I saw as major potential drivers of the noise in the relationship. They were:

  1. Changes to the starting goaltender a skater typically plays with
  2. Changes to the team a skater plays on

Intuitively, I think these factors make sense as things that could possibly impact on ice / relative Sv% numbers over time. For the first point, if a skater is playing in front of an elite goalie like Braden Holtby or Henrik Lundqvist – only specific, highly dangerous defensive lapses will actually turn into goals. However, if a skater is playing in front of Ray Emery – those same types of lapses may turn into incremental goals against. As a result, I wanted to investigate how Sv%RelTM’s predictive capabilities changed when a skater is consistently playing with the same starting goalie.

Secondly, when a player changes teams one can safely assume it would create a mess for this metric. For example, when Phil Kessel went from Toronto to Pittsburgh he changed not only the systems he played within, but also the individual teammates he is being compared against (on an innately relative metric). He also implicitly changed #1 – the starting goalie that he is most frequently playing with.

Controlling for Goaltender and Team Changes

As such, I set out to conduct a very similar analysis to what Garret did, while also attempting to control for the two factors above. I did so by limiting my sample to the following criteria:

  • Players included must have played for a single team for all four seasons studied (2010-2014)
  • Teams included must have had the same starting goalie for all four seasons
    • Starting goalie was defined as 50+ starts in full seasons and 20+ starts in the 2012-2013 lockout season
  • Players included must have played at least 500 minutes per season, and have been on the ice for at least 800 shots against

As you would expect, using all of the filters listed significantly reduced the size of my relevant sample. For example, only nine teams met the starting goalie criterion in this time period: SJS, MTL, CHI, NYR, DET, ANA, LAK, DAL and PIT. Further, across these nine teams, only 21 defensemen and 25 forwards (total n=46) played four consecutive seasons for one of those teams over this period. Of course, I need to point out that these sample sizes are definitely on the lower boundary of statistical significance. However, given the need to control for various factors, I think this was a necessary trade-off, and trying to increase sample size will be an important area to further investigate in the future.

As I move into the results in the next section, I have to say: they surprised me.

Save % Relative to Team – Defensemen only

Naturally, I think everyone considers supporting the goaltender and/or playing a major defensive role to be the primary job of just that – defensemen. This was my hypothesis going into the analysis, so, like Garret, I initially focused on just those players. However, as you can see from my chart below – the results were far from groundbreaking:

1 - Sv%RelTM Defense

 (Note – all data shown is even strength in order to remove the impact of PP and PK Sv%).

As you can see – there appears to be almost no relationship (R2 < 1%) between time periods in terms of a defenseman’s ability to impact his team’s on-ice Sv%. This finding is consistent when using relSv%.

One thing worth pointing out: this data has two relatively unique results in the top-left quadrant – Nick Leddy (CHI) and Cam Fowler (ANA). Both of these players had very poor Sv%RelTM results in 2010-2012 (roughly -2%), but improved significantly in 2012-2014 (1%). Given they both had their rookie seasons in 2010, one could argue their own development could have impacted their differences between periods. For interest’s sake, removing the two of them increases the R2 to approximately ~15%. Although this is interesting to note, this adjustment would need to be applied more broadly before we could draw conclusions from it.

Although this was ultimately less meaningful than I had hoped, my next thought was to add forwards to the sample – if nothing else, this would increase the sample size. This brings me to the next category:

Save % Relative to Team – Defensemen & Forwards

2 - Sv%RelTM Forwards &amp; D

Now, as you can see in the chart above, what I found here was both exciting and surprising. The exciting aspect is that there actually was some relationship – something otherwise unheard of in this type of analysis.  The surprising aspect of this came up when I asked myself: why are forwards the category of skaters who are able to show a sustained skill in Sv%RelTM, and not defenders? Before I had a good answer, I proceeded to do one final chart to test this unexpected result on its own:

Save % Relative to Team – Forwards only

3 - Sv%RelTM Forwards only

Looking at this chart is where I was blown away. By removing defensemen altogether (getting to a total n=25), you can see that the predictive value of Sv%RelTM reaches the highest found in any analysis I have reviewed – an R2 of 41% (or 38% for the same players when using relSv%). Although this isn’t a massive correlation, it is within the same range that xG and CF% fall into in terms of predicting a team’s future GF% over a season – metrics that are both widely regarded as valid for forecasting future outcomes.

Explaining These Findings

Now – as mentioned – the most difficult question here is: why are we getting this result? Why would a defenseman who fits the same team/goaltender-controlled data as the forwards sampled not be able to repeatedly impact his team’s on-ice Sv%? I certainly can’t say I know the answer, but one hypothesis that came up when discussing this with David Johnson: defensemen often play generalist roles, while forwards more often play specialized ones. For example – a 1st forward line is typically a scoring line, and a 3rd is typically a shutdown line. On the other hand, defensemen often get paired with their ‘opposite’ – having an offensively-minded defensemen paired with a stay-at-home one. I think these hypotheses intuitively make sense, however, the major drivers of these results remain open for interpretation.

Cross-team Applicability

One last question that will likely come up: given players can only consistently demonstrate Sv%RelTM impact when on a single team with a single starting goalie, does that mean it is not a skill transferable to other teams? My own (short) answer to this would be ‘no’. The relationship shown here for forwards indicates to me that it is in fact a skill, whether or not it would be displayed to the same degree on a different team or with a different goalie.

Much like CF% has significant team-level and player-level components – and won’t necessarily be directly transferrable between teams and systems – we still think of players as being strong/weak at driving shot-attempt differentials. I think the same logic can be applied here. But again – I am very curious about others’ thoughts.

Conclusion

In the end – this has been a hotly debated topic for quite a long period of time, and I doubt my work today will stop people from having different views on it. Although the analysis I have shown reaches a particular conclusion, I personally will not rely too heavily on this finding until it is broadly tested by the analytics community. However, given the desperate need in hockey to find additional ways to evaluate a player’s defensive contribution, hopefully this analysis helps us all continue to learn more about the merits of relSv% / Sv%RelTM as metrics for player evaluation. I look forward to anyone’s feedback or thoughts, @michael_zsolt  on twitter or at OriginalSixAnalytics [at] gmail [dot] com.

Valuing Stamkos: Estimating What Steven’s Next 7 Seasons are Worth

Posted in Player ValuationTagged , , , , , , 3 Comments

 This article is being co-posted on Maple Leafs Hotstove as well as on my own site, http://www.originalsixanalytics.com. Find me @michael_zsolt on twitter.

 All season long, Toronto fans have been talking about Steven Stamkos. It is well known that Stamkos will be an unrestricted free agent (UFA) this July. This means that – you guessed it – Steven Stamkos could become a Leaf. Naturally, if it happens, many fans have presumed Stammer would be named captain, immediately become the face of the franchise, and lead the Leafs through its centennial season and beyond.

As a result, many have also asked – what is Steven Stamkos worth? Assuming he reaches UFA status, what will it cost the Leafs to sign him? Does he deserve the same type of money as Kopitar, Kane, and Toews? Back in January, Ryan Kennedy over at thehockeynews.com made some reasonable comparisons and suggested that he sees Stamkos as being worth roughly ~$9M in AAV. Full credit to Ryan – as that was earlier in the season and somewhat different situation – however, in this article I will argue that it is extremely unlikely that any team, Toronto included, will be able to sign Stamkos for an AAV of less than $10M, or for much less than the maximum term of 7/8 years.

Before getting into the approaches to valuation, let’s look at some of Stamkos’ key statistics to understand exactly how he adds value.

Stamkos’ Goals Above Replacement (GAR)

Stamkos GAR BARs

 (If this data doesn’t mean much to you – I recommend you check out my summary of the practical applications of Goals Above Replacement).

For individual player analysis, GAR (Goals Above Replacement) can be a useful starting point to illustrate the big picture. Looking at Stamkos’ GAR shows us:

  • He performs at an elite level, hitting 15-20+ GAR all but his rookie season, something done in fewer than ~6% of all player-seasons
  • Stamkos contributes value to his team almost entirely through his offensive play:
    • 95% of Stamkos’ career GAR comes from his offensive ability, approximately 2/3rd of which coming from his shooting percentage (15.2% at 5v5) and 1/3rd from his impact on shot-rates (Corsi For)
    • At times over his career he has been a slight defensive drag to his team – but ultimately he is a relatively neutral contributor in non-offensive areas
  • As noted in the chart, Stamkos’ 2012-2014 results may not be indicative of his capability as the seasons were impacted by the lockout and his broken leg, respectively

We likely didn’t need GAR to tell us any of the above, as many would know most of this from simply watching Steven. However, GAR will be important to keep in mind for when we estimate Stamkos’ value later on.

WARRIOR Chart

Next, I have copied Domenic Galamini’s WARRIOR chart of Stamkos, showing Steven’s relative performance within the league on a number of metrics. For those not familiar with WARRIOR charts, Dominic provides a great summary here.Stammer solo

Looking at this chart confirms the following:

  • Stammer is an elite scorer and a solid playmaker, scoring very high on Goals/60, Primary Points/60, RelCF60 and RelxGF60
  • Stamkos should not be considered a two-way forward: his defensive contribution is sub-par, as shown by his shot/goal suppression (CA60Rel/xGA60 Rel)
  • As a result, when valuing Stamkos, we should be comparing him to similar elite shooters (e.g. Ovechkin, Kane, Perry), as opposed to the leagues’ finest two-way forwards (e.g. Toews, Kopitar, Bergeron)

Given that elite shooters don’t always excel at driving 5v5 CF%, it will be important to keep in mind shooting percentage and power play results when comparing Stamkos to other players.

Before we get into valuing Stamkos, let’s quickly look at the qualitative side of the equation.

A Note on Intangible Factors

Although the hockey analytics community tends to discount ‘intangible’ factors, I personally consider them very meaningful. Consider: if you have two players with similar stats – would you pay more for the driven, hard-working, high-character guy, or the player who demonstrates none of these traits?

We all know Steven Stamkos is a strong leader. Last season he captained his team to game six of the Stanley Cup final. He also has an incredible work ethic; he is well known for his rigorous offseason training with Gary Roberts. While these strong intangibles are difficult to quantify, they still should play a meaningful role in justifying a premium valuation for Stamkos. But first – what aspects can we quantify?

Approaches to Financial Valuation

In the world of corporate finance – buying and selling companies – great effort is spent attempting to value the businesses being acquired. Generally speaking, there are two approaches:

  1. Relative (market based) valuation methods
  2. Intrinsic valuation methods

Let’s start with relative valuation methods.

Market Based Valuation: Comparable Company Player Analysis

For investors, ‘comparable company’ analysis is a key aspect any deal. The valuation multiples of companies across an industry (e.g. ‘Enterprise Value / Earnings’) set the baseline for determining what the price should be for a similar one. As many readers will know, the exact same approach is often applied to players: we can look at a set of comparable elite shooters to help derive a benchmark for Stamkos’ market value. Keep in mind – relative valuation methods like comparables tell us what others are willing to pay for an asset (the market), not necessarily the inherent or ‘true’ value of it.

Using Corsica’s Similarity score feature, focused specifically on Stamkos’ most important stats (e.g. G/60, P/60, Sh%), I have arrived at the following set of comparables. All players listed below had a similarity score of at least 85% with Steven. I have also tried to present this similarly to how it is done in finance:

Comp Table

(Note – All data shown is all situations, except where specifically noted).

As you can see, few players in the league score goals and contribute primary points at the same rate as Stamkos. Compared to this set as a whole, Stamkos outperforms the group average for all metrics listed, except for the Corsi stats and FirstA/60.

However, for this analysis to truly be relevant, we can only really compare Steven to players in the same free agency situation when signing, i.e. UFAs. As such, I have highlighted the four UFA players that I consider as having the most comparable performance statistics to Stamkos: Patrick Kane, Alexander Ovechkin, Evgeni Malkin, and Corey Perry. I have also shown their four-player average at the bottom of the table. As you can see, Stamkos is neck and neck with these players on every metric.

In terms of dollars, these four players averaged an AAV of $9.7M, and had an average term of the maximum 8 years. Considering that these players are also not historically known for their exceptional leadership qualities – which Stamkos has in spades – you can begin to see the case for Steven to be earning $10.0M-$10.5M+ per season.

Intrinsic Valuation Methods

In financial markets, the other approach to valuation is using ‘intrinsic’ methodologies – e.g., trying to derive what is a company is worth in an absolute sense. When valuing a company, this is based on the discounted ‘present value’ of a company’s future cash flows – how much money a company will generate cash for its new owner in the future.

My view is that the best ‘intrinsic’ player valuation method available based on public stats is using GAR, which I introduced at the beginning. The first approach I will use is valuing a player based on his historical GAR score, which I have shown previously. The second is a new approach, where I will show a high level attempt to forecast a player’s future GAR – and use that to derive what he should be paid. When we have confidence in a forecast for a player – (which I won’t necessarily say about my own Stamkos GAR forecast today) – forward-looking analysis will be the most ‘pure’ measure we can use in valuation.

Historical GAR

First I will plot Stamkos on the Fair Market Value curve, in order to see what he is worth based on what he has done historically. Typically I would encourage using a 3-year average GAR – however, due to the issues of the lock-out season and Stamkos’ injury, I will instead use his career average (ex-rookie season) GAR of 22.

Historical GARLooking at this chart shows:

  • Stamkos’ career Avg GAR/season of 22 values him at $10.8M in AAV
  • However, historical GAR will tend to over-estimate a player’s value, as it is backward-looking, and does not adjust for how players decline with age
  • As such, though this is a useful data point, we should take it with a grain of salt

Forward-Looking GAR: A High Level Estimate

To truly estimate a company’s, or players’, intrinsic value we need a forecast of how it/he is going to do in the future. Of course, like any prediction, this type of estimate is inherently flawed and almost never is how reality actually plays out. However, creating a forecast can help us to see some reasonable scenarios of how Steven might perform in the future. To do this, I will combine his past performance with the aging curve of a typical NHL forward.

Fortunately for us, part 2 of Moneypuck’s Building a Contender Series has an excellent chart that summarizes the league-average GAR aging curve for forwards (is anyone tired of me referring to this series, yet?):

Moneypuck GAR Aging curve

(Source: Moneypuck)

In order to connect this to Stamkos’ expected performance, I will use a rather blunt methodology of simply applying the same absolute value of decline in GAR score showed here to ‘extend’ forward Stevens’ historical performance. The most accurate way to do this would likely be to base Stamkos’ long term decline on how his comparables declined at the same age. However, for the purpose of brevity, I will use this simplified approach to help illustrate the methodology.

Stamkos GAR Forecast

In the chart above I did the following:

  • Extended the ‘historical’ performance by one year in 2015-2016, assuming his current season is comparable to 2014-2015
  • Starting at Stamkos’ current age of 26, I applied the absolute decline from Moneypuck’s chart to Stamkos in five and three year chunks
    • (As you can see, the projection begins to decline more quickly after 2020-2021F)
  • Last, I adjusted this ‘base’ case by +20% and -20% to create illustrative upside and downside cases

Although this is a very high level estimate, I think this gives an interesting, basic idea of what we could expect from Stamkos over the next eight seasons. What does this equate to in terms of dollars? The base case results in an average GAR per season of 18.3, equating to $9.1M per year in AAV. The downside and upside cases each average GARs of 14.7 and 22.0, respectively, coming out at AAV values of $7.4M and $10.8M.

Sizing it all Up

So – lets stack these all up next to each other:

Aggregate

Naturally, looking at a wide range of methods gives a wide range of potential values. However, all three of these approaches can be used to justify a valuation for Stamkos at $10.0M+ of AAV. There is also every precedent for him to get the full 8-year maximum term, or 7 years with the Leafs (in the case of a UFA signing with a new team). We can all be sure Stamkos’ agent is well aware of all of these approaches and will be pushing for the high end, trying to secure as much as possible for his client.

Practicalities of TML’s Negotiation Position

Much like in valuing and buying/selling companies – you can do as many fancy calculations as you want – but something is ultimately worth what someone is willing to pay for it. Given that Stamkos is a UFA, he is essentially able to auction his contract to the highest bidder – putting all of the power in his hands. This power dynamic, combined with the prior valuation approaches, gives me strong reason to believe Stamkos deserves $10M+ – and teams should be willing to pay it, if they have the cap space available.

I’m sure some readers will point out that Stamkos’ basic counting stats (goals/assists/points per game) have declined in the past two seasons, which may be a red flag for the future. Travis Yost and Stephen Burtch both addressed this somewhat, arguing that his quality of teammate and usage are both factors that may be impacting his performance. However, in terms of the value he commands, the strength of his negotiation position will ultimately rule the day. If teams want him, it will take big dollars to get him – either they will be willing to pay for an elite player, or they will not – and I suspect there will be at least one team left standing at a double-digit AAV.

Conclusion

In the end, I know Leafs fans will continue to argue that Stamkos wants to come home, and that maybe he will take a hometown discount to join the Leafs. Although this is possible, in reality the Leafs should not be expecting to walk into a bargain contract with Stamkos. He is an elite scorer, a strong leader, he deserves a top dollar contract, and I strongly believe there will be a team willing to step up and give him the $10M+ per season he commands. Stamkos is entirely worth that amount, and he is an excellent candidate to lead this young Leafs’ team through their centennial season, and six more after that. All us fans can do now is sit and wait.

Your move, Lou.

 

This article is presented by OAK Coasters, where you can buy hand made, One of A Kind (OAK) coasters that make the perfect housewarming gift. Check them out at OAKCoasters.com.

 

 

 

The Importance of Financial Analysis: A League-wide Review of Salary Cap Efficiency

Posted in Wins Above ReplacementTagged , , , , , , 6 Comments

Dimitri Filipovic recently wrote a great article summarizing that – despite the NHL’s rhetoric of ‘parity’ – it is a still a league where the teams’ with the greatest financial resources tend to be the most successful. After reading his piece, I wanted to write a short article building on this concept, where I will show that although having more resources is an advantage, how efficiently a team manages its salary cap space is just as important as how much they spend.

In this piece I will apply the same methodology I have used previously, where I showed how the Goals Above Replacement (GAR) metric can be used to quantify both a player’s and their team’s salary cap efficiency. I will apply the same methodology in order to do a league-wide review of salary cap efficiency in 2014-2015. Doing so, I will argue that (i) salary cap efficiency is an integral lens for all teams to look through when building their roster, and (ii) although an efficient use of salary cap alone will not win a team the Stanley cup – ignoring this type of financial analysis can quickly push them out of Cup or playoff contention entirely.

Determining Team-level Salary Cap Efficiency

Salary Cap Efficiency is defined as how effectively a team uses its salary cap space, based on a bottom-up analysis of each individual player.  To calculate it, I first derive the fair market value of all players as a function of their GAR (FMV = $575K + ($467K * GAR)). Then, I compare that to the Average Annual Value (AAV) of each player’s contract – his cap hit – in order to see if he is being over or under-paid. By aggregating this analysis for an entire team, you can see (a) which player contracts on each team are the most cost-efficient, and (b) approximately how well that team manages its salary cap as a whole.

As a quick refresher, here are two previous charts that summarize the cap efficiency of the 2014-2015 Chicago Blackhawks. The first chart shows all members of the team in relation to the Fair Market Value equation that I just described. The second shows the amount of value created or lost on each individual’s contract.

Hawks - Arbitrage line

Hawks - by player calc

You can see from this that – as the Stanley Cup winning team – most of Chicago’s players are quite close to the FMV line, with only a few exceptions (Seabrook, Toews, Saad). Also, to be clear, we should never expect a team to price/negotiate any given player’s contract ‘perfectly’ – there will always be far too many changing variables in that player’s performance in the years after signing. However, a team can be most competitive by largely signing players with positive GAR, and by generally paying them within a reasonable range of their FMV – or much less, as enabled by their ELC/RFA deals.

In the second chart, I also want to draw your attention to the ‘Net Value Created/(Overpaid)’ figure shown under the legend, of -$6.3M. This figure is the sum of all individual player amounts displayed on the chart. Keep it in mind, as it provides the core of my analysis in the next section.

 (Side note: this Net Value Created figure for the Blackhawks specifically will not tie perfectly to my analysis in the next section. The next section is based on team 2014-2015 GAR scores, where the chart above is based on 3-year average GAR).

Application Across the League

Now, I will apply this type of analysis across the league in order to show which teams had the most cap-efficient roster in 2014-2015, and how that may have impacted their results in the standings. Given this is a GAR-based analysis I will start by showing the total Goals Above Replacement of all teams in the league.

Chart 1 - GAR by team

Looking at this, a few things are worth pointing out:

  • The vast majority of playoff teams are concentrated on the left hand side of this graph, and likewise the non-playoff teams are on the right
  • There are obviously exceptions where teams likely should have made the playoffs, but didn’t (e.g. BOS, LAK), or teams that did make the playoffs and maybe shouldn’t have (CGY, VAN)
  • Although this is a sample of only one season, I think it supports the validity of GAR as having a strong connection with a team’s success – despite only being available retroactively

In order to keep this brief I will fast forward a few steps (e.g. I will not show the calculations for getting the FMV of each team’s roster, and subtracting their total salary cap hit from that). Instead, here is the result, which is the Net Value Created/Overpaid for each team over the season, in absolute dollars.

Chart 2 - Net Value Created by team

Although this is a useful way to look at how well a team spent, it is also impacted by their total salary cap dollars. Given Dimitri’s point about the varying spending power across teams, it is important to look at this on a basis that is directly comparable across teams. As such, I have calculated each team’s ‘Salary Cap Efficiency %’ by simply dividing their Net Value Created by their total cap hit. The larger the percentage, the more efficiently a team has managed its cap.

The chart below shows the relationship between where a team finished in the playoff race along with its Cap Efficiency %. To be clear – a higher percentage (top of the chart) is a more efficient use of cap space, and a lower percentage is a worse use of cap space.

Chart 3 - Team Level Cap Eff - A

Here are some observations from this chart:

  • Most playoff teams are above 0% on cap efficiency – e.g. successful playoff teams are not wasting money by overpaying for players
  • Notably, Chicago was barely above 0%, having won the cup, and both Nashville and Winnipeg were in the ~30% range, while both being eliminated in the first round – showing that maximizing salary cap efficiency clearly won’t win you the Stanley cup on it is own

As Filipovic’s article made clear, greater financial resources will always help a team get the best talent, despite their efficiency or inefficiency. This chart does not highlight which teams spent the most overall, and naturally if two teams are getting the same ‘bang for their buck’ (spending at equal efficiency), the team that spends more will of course win out.

Despite this, a team can very easily ruin its own chance of being a contender by failing to look at its roster through a properly quantified salary cap efficiency lens. Below is another version of the same chart, where I have highlighted the large and unfortunate set of teams whose inefficient use of cap space has very likely impacted their ability to be a contender.

Chart 4 - Team Level Cap Eff - B

Fortunately for the teams in the bottom left quadrant, I think they are all well aware of their situations, and doing everything in their power to undo the current mess they find themselves in (with maybe one exception).

Conclusion

To wrap up, I think it should be clear that any team not conducting salary cap efficiency (or similar) analysis when contemplating its long-term roster construction is taking a significant risk. Being the most cap-efficient team may not be able to win you a Stanley Cup on its own (which Boston and San Jose learned the hard way in 2014-2015) – but having a very poorly managed cap is uniquely capable of taking your team out of contention. All that being said, we of course should keep in mind that cap efficiency is only one piece of the puzzle – to use dollars efficiently a team first has to acquire players worth spending money on.