Understanding ‘WAR’ and its Practical Applications to Player Evaluation (Part 1)

(OSA’s WAR ‘Explainer’ – Part 1)

This article is being co-posted on Maple Leafs Hot Stove as well as on my own site, http://www.originalsixanalytics.com. Find me @michael_zsolt on twitter.

@DTMAboutHeart of Hockey-Graphs.com and I recently exchanged tweets about the ‘WAR’ metric.

Twitter Exchange

Twitter Exchange pt2

This exchange brought to my attention that, despite the great work that has been done by the creators of WAR, it is a very complex metric. As a result, many don’t know exactly what ‘Wins Above Replacement’ (WAR) is, or how it works in hockey. It also struck me that, currently, much of the hockey community falls into one of two groups:

  1. Those who don’t know about, or fully understand WAR (or similar metrics), and thus ignore them, and
  2. Those who understand WAR, but think it is not yet developed enough to fulfill the original single number dream

(As well as perhaps a third group: teams that are secretly using WAR but not telling anyone… possibly evidenced by the creators of WAR now both being employed by professional sports teams).

As such, I am writing this series as an attempt to amplify the work done in this area. My main objectives are to increase the number of people in the conversation, as well as to demonstrate some of the practical applications, and limitations, of using the WAR metric to evaluate players. I will be focusing solely on the WAR metric developed in 2014/2015 by the great team at WAR-On-Ice.com (WOI), who are sadly leaving the public hockey world on March 31st, 2016. For those who don’t know, they are providing access to their entire database’s raw data up until that date. For reference beyond March, I have also uploaded their raw WAR/GAR by season output to my own website.

Further credit is owed to Tom Awad’s work on ‘Goals Versus Threshold’, a very similar stat that the WOI ‘WAR’ metric built upon. Last, I also want to give credit to Moneypuck’s excellent 2015 ‘Building a Contender’ series, where he applied GAR to building a winning franchise, which I have cited multiple times in article below.

So – before I get into WAR itself – why should you care?

Why is WAR Important?

I don’t think anyone will disagree with the following:

  1. The purpose of hockey is to win the game
  2. A team wins the game by scoring more goals than its opponent
  3. A player’s ‘ultimate’ contribution to his team is defined by his ability to improve his team’s goal differential (e.g. to increase goals scored and decrease goals against)

Although these three things are very basic, they are the foundation of why WAR is an important metric. It is easy to evaluate players strictly on the stats everyone has readily available: Goals, Points, Assists, etc. However, the classic example of where these fall short is the high-flying scorer who gives up two shots/goals against for every one that he gets. Corsi has become very popular for addressing this through an adjusted, game state-specific plus/minus, based on shot attempts. Although Corsi and WAR are built off similar concepts, ‘WAR’ tries to take shot rate metrics like Corsi, combine them with other factors, and then tie the result directly to the column on the score-sheet that matters the most: Wins.

What is WAR?

As mentioned, ‘WAR’ is a metric that attempts to combine a player’s contributions in offensive, defensive, and other aspects of the game, into the number of ‘Wins’ he contributes to his team. A fundamental concept of WAR is that it constantly compares NHL players to a set of ‘baseline’ expectations. This baseline is similar to looking performance versus league average, though it ends up closer to the league ‘minimum’.

Baseline expectations are important because of player value, and cost: an NHL team should only pay more than a bottom-tier salary if a player is contributing more than bottom-tier results. Thus, ‘baseline’ (or ‘replacement’) level players represent the quality of player that could be acquired for relatively little salary/cost on the free agent or trade markets. WOI calls their method the ‘Poor Man’s Replacement’, as they derive it based on players who have limited NHL experience. Conceptually, these are the players called-up to fill a 3rd/4th line role when injuries require it.

Finally, in order for WAR to convert performance into wins, we must first derive ‘goals’ contributed. Eric T has shown that roughly ~6 goals = 1 win – directly connecting goals above baseline into wins. Ironically, many people have now realized that ‘Goals Above Replacement’, or ‘GAR’, is actually easier to interpret than WAR, especially for individual players. As a result, from here on I will largely focus on GAR, though you should keep in mind that the two metrics are interchangeable at the rate of 6:1.

One last comment before the data…

A Caveat on ‘Catch-All’ Stats in Hockey:

As Michael Lewis’ Moneyball has shown the world, Baseball is the perfect sport for a WAR-type metric. Hitting, fielding and pitching all arguably equate to individual skills disguised within a team game, easily allowing statisticians to separate out individual contributions versus context, and noise.

Hockey, on the other hand, is a sport where it is very difficult to create a single statistic that will summarize ‘all’ of a player’s contribution in one number. As illustrated in this helpful post by Eric T from 2013, it can (conceptually) be almost impossible to perfectly adjust for all aspects of the game at once. However, those who have trudged through the Road to WAR series will see the extreme amount of adjusting for context that WOI has done, where they simultaneously controlled for teammates, opponents, game-state, and many other things – getting all the way down to elements as seemingly minor as travel fatigue (e.g. home/away team performance, and impact of playing on back to back nights). This level of detail and rigor suggests to me that WAR is among the most advanced publicly available stats to date.

Regardless of if you choose to place much value on WAR/GAR, I want to emphasize that no metric will ever justify ignoring other methods of player evaluation. Given WAR/GAR says nothing of a player’s role, Rob Vollman’s player usage charts are a very complementary tool to use alongside it. I also encourage the uninitiated to check out Eric T’s straightforward primer on different metrics that can be used for player evaluation.

Now – back to GAR, and finally, some actual numbers.

What is a ‘good’ GAR/WAR score versus a bad one?

At a high level:

  • If a player has a ‘GAR’ of zero – they are equivalent to a baseline/replacement level player
  • If a player has a positive GAR, they are ‘better’ (at contributing to their team’s goal differential) than a baseline player
  • If a player has a negative GAR, they are worse than a baseline player

In order to be a bit more specific, let’s look at some data from Moneypuck’s series, which was very insightful on this front. His third article looks at:

  1. The GAR scores of every NHL player from each season in his sample
  2. The GAR scores of the players from the four conference finalist teams each year

The two charts below summarize his data.

( Note: I haven’t made any changes to his data – these are the same numbers shown in a different format)

GAR Distribution - All Players

Top Team - Player Count by GAR

From this data we can make a few observations:

  • It is very difficult for a player to pass a GAR score of even 10 in a given season
    • The first chart shows that fewer than 14% of players achieve this each year, and the second shows that even conference final-reaching teams usually only have ~4.5 players with a GAR of 10+
  • Even fewer have a GAR of 15+, at approximately 5.9%, or only 1 in 17 players in the league
  • Last – as highlighted on the first graph – over 70% of the seasons played in the NHL score a GAR of 5 or lower
    • Put differently, 70% of NHL players fluctuate in and around the league minimum level of contribution to their team’s goal-differential

Now, in order to illustrate how various players are scoring in terms of GAR – I have summarized the following table for you to compare against your own, personal eye-test.

Example Players by GAR Range

Hopefully this table helps to set some benchmarks in your own mind about how various players score on GAR. This table also makes it clear that Goalies and Defensemen are under-represented in the top GAR ranges, when looked at on a three-year average basis. This highlights an important qualifier of the GAR metric: like most NHL player evaluation, it currently best evaluates a player’s offensive contributions.

Looking at the components of GAR (the next section) will explain why: defensemen will largely contribute to just one or two of the six components (e.g. impact on shot rates), while forwards will contribute to shot rates while also providing material contribution through their shooting percentage, face-offs, and penalty drawing. As a result, when using the current GAR metric to evaluate players, it will be most accurate to compare players within positions, rather than across them. For what it’s worth, WOI previously hoped to add other defensive components to WAR, as well as a measure of play-making ability, helping to offset this gap. However, the closure of their site means these areas will not be publicly incorporated until another brave statistician picks up where they left off.

Now that we know why GAR is important, what a good/bad score is, and who typically scores where – what factors is this number actually considering?

What Are the Components of GAR/WAR?

 As WAR-On-Ice has already given a very detailed summary of the math behind the metric, I’ll instead focus on the big picture of its component parts. WAR is currently made up of the following six components for skaters, which I have grouped into the three broad categories below:

Offensive Contributions

  • Shot rate for
  • Shooting percentage

Defensive Contributions

  • Shot rate against

‘Gameplay’ Contributions

  • Faceoff win percentage
  • Ability to draw penalties
  • Ability to avoid taking penalties

Whether or not you are familiar with statistics, I think most of us can agree that increasing your team’s shot rate differential, shooting percentage, faceoff percentage, and power play opportunities, while decreasing your team’s minutes on the penalty kill, are all going to help contribute to goals and wins. As a reminder, each of these have their own definition of ‘replacement level’ that GAR is calculated against. One last side note: Goalies are calculated as their own category, based on Sv%, which I have omitted here.

Now, the fun part: 

How to Analyze a Player’s WAR/GAR – A Case Study

In order to demonstrate the various components of GAR, I have chosen a player that we all know, and who also happens to contribute at both ends of the ice: star two-way center, Jonathan Toews.

Looking at Toews’ GAR metrics in the 2013-2014 season gives us the following:

Toews 6 bars

Keep in mind: across all components, a positive GAR is a better result. E.g., Toews’ positive ‘Shot Rate Against’ GAR score (3.6) means he has been successful in reducing shot attempts against.

The above data shows us that:

  • Although Toews is a major offensive threat, a significant amount of his impact comes from his defensive and ‘gameplay’ contributions, e.g. shot suppression, face-offs, and ability not to take penalties
  • In 2013-14 Toews won more than half a game (e.g. ~3 GAR) in his face-off percentage alone, putting him among the leagues’ best
  • Toews contributes most offense through his possession and shot-attempt driving capabilities (shot-rate), rather than by being a sniper
    • In contrast, while not shown here, Patrick Kane contributes more through his strong shooting percentage, often scoring 6-7 GAR per season in Sh% alone

Introducing a style of chart I will label ‘GAR Bars’, we can summarize a player’s total GAR contribution back into those three major categories:

Toews 2013-2014 GAR BAR 

This chart shows the exact same data as the prior one; the only difference is that now I have aggregated each of the six components into their general buckets. For two final illustrations, we can expand on this bar to analyze Toews based on his GAR over time. The charts below show (i) Toews’ absolute GAR contribution over a number of seasons, as well as (ii) Toews’ relative GAR contribution, showing each category as a portion of his total.

Historical Absolute GAR

Toews Historical Absolute GAR

This chart shows us that:

  • Toews has been in the 20-25 GAR range for most of his career – an extremely high score, especially when considering only 2.4% of all NHL seasons exceed 20 GAR
  • Despite winning the cup in 2014-2015, Toews’ individual performance last season dropped to his lowest level since his rookie year
    • While ~16 is still a very good score, this decline may be indicating signs of Toews’ age, suggesting we should expect a slightly reduced level of performance from him going forwards

Historical GAR Distribution

Toews Historical Relative GAR

Finally, looking at Toews’ GAR contribution by category shows us the biggest step down in 2014-2015 was in his defensive play. Although I haven’t watched enough Blackhawks games to observe this myself, one reason this could be happening is a slight decline in skating ability/speed with age, preventing him from being as involved around the rink as he once was.

Conclusion

Now, I haven’t used this data to hammer home a unique point of view about Toews – no one needed me to quantify it to know he is a future Hall of Fame-caliber player. Rather, the point of this article has been to provide some colour behind the basics of the WAR/GAR metric, and to illustrate in a simple, straightforward manner how anyone could apply this metric to their own thinking on player evaluation.

Regardless, it is clear that Jonathan Toews is a hugely valuable player to Chicago at both ends of the ice, sitting in the elite, 20+ GAR echelon, and having peers among the likes of Crosby, Kopitar, and Ovechkin. This analysis also says nothing about the leadership skills he has demonstrated on and off the ice, taking his team to an unprecedented three cup wins in the last six years. As a result, I think we all believe that Captain Serious earns every dollar of his $10.5M cap hit. But how can we know for sure? Unfortunately – for that, you’ll have to wait for the next installment of the series, where I will demonstrate how to use WAR/GAR to quantify what a player is worth.

 

Advertisements

2015 Draft Day: How Hunter and Dubas May Have Out-Played the League (Part 2)

This article is being posted here as well as in parallel as a guest post at Maple Leaf Hotstove.

In my last post I shared my analysis on long term player performance and development based on draft round. As a follow-on to that article, I’d like to do two things: first, I’ll convert my last analysis into a relatively straightforward and ‘usable’ metric for draft pick value. Then, I’ll apply this metric to two short case studies in order to illustrate who won each of the Leafs draft day trades this past summer (hint: it wasn’t Ron Hextall or Jarmo Kekalainen…).

Draft Pick Value

The third and final objective from my original report posed the following question:

  • How much more valuable is a pick in the first round versus the other rounds? All things being equal, what should a pick from each round be worth in a trade?

Building on the analysis done by others mentioned in my last post, the chart below summarizes how I have approached ‘converting’ long term performance data into a relative draft pick value metric. To be clear: I am not proposing the values shown in this chart, rather, I hope to use this chart to illustrate the methodology I have applied across a number of metrics.

Games played data is one metric that can inform relative pick value by draft round

Pick Value Demo Chart 

  • The chart above shows how likely a player from each round is to play ~2+ seasons in his career, and by when he should be expected to do so
  • The chart then calculates how much more likely a player from each round is to pass 150 GP than the bottom cohort of rounds (e.g. average of Rounds 4-9) – shown as a multiple of those rounds
  • Thus, if we define a pick in rounds 4-9 as the ‘base unit’ (e.g. ‘1.0 units’), using the >150 GP threshold shows a third round pick to be worth 1.8 units, a second round pick being worth 2.4, an 11th-30th overall pick being worth 6.0, and a top 10 overall pick being worth 7.6

Applying this approach to multiple metrics will give us a more robust view of relative pick valueRelative Pick Value Chart

  • The table above shows the ‘Draft Value Units’ (working name) of a pick in each round across three metrics: >30 Pts, >100 Pts, and Avg Career Pts. In essence, ‘Draft Value Units’ are comparable to a currency with which teams can value and exchange draft picks
  • As mentioned – each round is shown as its multiple of the lowest group (Rounds 4-9) – and most of my attention going forward will be on the far right, highlighted column; also, all of the values of this chart are derived from the data shown in my last article
  • As Michael Schuckers and Stephen Burtch previously showed, this data suggests teams should use caution when trading their first and second round picks, as they can be worth many times more valuable than the other rounds
  • It is worth noting that Lifetime Production data can also shed light on ‘absolute’ pick value; e.g. in a trade for active players, a pick in the top 10 overall should be treated as if it has a career lifetime value of ~350+ points as a forward, or ~170+ points as a defensemen – something directly comparable to ‘remaining’ production in an active NHLer

Part of the goal of this exercise was to create a pick valuation methodology that is highly simple, and usable by many, regardless of their level of analytical sophistication. As someone very familiar with the world of corporate finance and valuations, I can tell you first hand that – despite financial firms using the ‘fanciest’, most complex valuation models you could imagine – the most effective of these models will often reduce complexity, rather than create it. All investors and bankers also know that valuation analysis is a ‘blunt’ tool, and it will never give you an exact, ‘true’, intrinsic value for a corporation or a stock (i.e. picture using an axe to carve a statue). I think the same thought process applies to this draft pick value methodology – it is directional, rather than exact – but hopefully it also intuitive to understand and apply. My general philosophy is that decision-makers do best when considering metrics that are reflective of the big picture, while simultaneously weighing those against the typical qualitative information they bring to the table, such as team needs, player skill, size, character, etc.

Now – let’s get into the deals.

Draft Day 2015 Deal #1 – Dubas and Hunter v. Ron Hextall of the Philadelphia Flyers

After seemingly endless conversations and hustling around the draft floor, Dubas and Hunter’s first trade of the day was with Philadelphia:

 TOR PHL Trade Chart

Now, based on the far right column in my ‘Draft Value Unit’ table above, we would think to assign the following values to these picks:

TOR PHL - 'Wrong' Pick Chart

Huge win for the Leafs, right?

Not necessarily. As we all know, and as Schuckers and Burtch’s analyses clearly demonstrate – all picks in each round are not created equal. The 11th overall pick is not equivalent to the 30th, even though the table above would be treating them as having the same value. Because of this, the Draft Value Units shown above would be most appropriate to use when a trade is done well in advance of a draft, and it is not known exactly which overall draft number a given pick will relate to. In order to make this metric meaningful to trading ‘known’ pick numbers, we will have to do some adjusting.

Applying ‘Draft Value Units’ directly to draft pick numbers will show some counter-intuitive results

Draft Value Units - Step Function

Instead, for individual pick numbers, we need to ‘fill in the gaps’ with a new curve, equivalent to the equation shown below

 DVU - Curve

  • Once we do know what pick numbers each team will have, we need to adjust the value of each pick appropriately
  • To do this, I have derived the solid, light blue line in the chart above, which best fits the original Draft Pick Values shown
  • This line shows what the appropriate Draft ‘Value’ is for a pick once we know the exact pick number that it relates to
  • The line was derived by looking at the (x,y) coordinates of each pick number and its respective Draft Value Units, after assigning the values in the first table shown to the mid-point of each round (e.g. 5th overall pick being worth ~11.1 DVUs, 20th overall pick being worth ~7.0, etc.)

In order to test validity, we can compare this curve/equation to those derived by Shuckers and Burtch. The similarities between the three draft value methodologies help to support the accuracy of the findings of each. Note – the one downfall of this curve is that, in order for it to appropriately reflect the value of the first 100 picks, the DVU’s hit zero around pick 100. As a result, my advice for those trying to use this to value picks from the 4th Round and onward is to treat each pick as having a Draft Value of 1.0 units, rather than zero (e.g. revert back to the dotted line).

Now – back to Leafs v. Philadelphia.

Plotting the three Leafs/Flyers picks traded on our curve shows the value of each individually

 DVU Curve - TOR PHL

  • The chart above shows that the theoretical value of the 24th overall pick is 6.0 DVUs, with the 29th and 61st overall being worth 5.2, and 2.3, respectively
  • As I did here, in order to use this on any other trade, all you have to do is find each pick number on the x-axis, trace it to the curve, and then trace that point on the curve back to the y-axis – giving you the Draft Value Units of that pick
    • The end of this article also has a table showing Draft Value Units for each pick number, e.g. the coordinates that make up this line

On an expected value basis, the Leafs won the trade with Philly by 25%

TOR PHL - Stacked Bar

  • Based on the values assigned above, the Leafs were the clear winners of this trade
  • As a reminder – these Draft Value Units originate based on an average of the probabilities for each player drafted to do three things: 1) Exceed 30 career points, 2) Exceed 100 career points, and 3) Maximize their lifetime (point) production

One last important thing to point out – as I’m sure many Flyers fans and general non-stats folks will want to discuss – the Flyers executed this trade because they badly wanted to pick Travis Konecny, of recent Canadian World Junior ‘fame’. They saw him as materially better than their next choice (if they had picked 29th) – Nick Merkley. This piece isn’t about scouting or individual player evaluation, which are of course important factors to consider – generally speaking, every team should be drafting with a broad strategy in mind, that drives towards filling its specific needs (which I’m sure justified this trade from the Flyers’ perspective). The point of this analysis is to say that – on a long term, expected value/probability basis – a team will do better to be on the Leafs’ side of this trade. Even if 10 years from now Konecny is the next John Tavares, and everyone thinks Ron Hextall is a genius – I think the Leafs were on the right side of this trade based on what was known on draft day.

(As a side note: this article by Travis Hughes gives a bit of background about Philadelphia’s rationale for being so eager to trade up for Travis Konecny).

So far, Leafs 1, League 0

Draft Day 2015 Deal #2 – Dubas and Hunter v. Jarmo Kekalainen of the Columbus Blue Jackets

 It didn’t take long for Dubas and Hunter to turn around and offload their newly acquired 29th overall pick either – employing a highly similar strategy in their trade with Columbus:

TOR CLB Trade Chart

Now if we apply the same analysis to this deal:

Looking at the value of each pick individually…

DVU Curve - TOR CLB

… It is clear the Leafs ‘won’ this deal too – by 23%

TOR COL - Stacked Bar

You don’t need me to tell you too much more, as the same analytical framework shows the Leafs fared similarly well in their trade with Columbus as they did with Philadelphia.

Before I wrap up, just for fun let’s look at the two deals as whole:

TOR Aggregate Trade Chart

The Leafs’ two trades during the 2015 draft created an incremental two players, 2.7 draft value units, or otherwise a 43% increase in relative value

TOR Aggregate Stacked Bar

By the end of the draft, the Leafs had used these three picks to select Travis Dermott, Jeremy Bracco, and Martins Dzierkals, based on the scouting expertise employed by Hunter and his team. It also helped fill some of the team’s draft objectives early on, enabling them to wait it out for sleeper picks like Dmytro Timashov in the fifth round – a player who turned many heads at the recent World Juniors tournament. I wouldn’t (yet) go as far as to say that Hunter has a legitimate ‘competitive advantage’ over other teams in player scouting and evaluation until more time has passed. However, Dermott’s OHL performance this year and recent World Junior selection also suggest that getting him at 34th was a also bit of a steal in its own right. Regardless, it should be clear that on this fateful day last June, Kyle Dubas and Mark Hunter made some excellent decisions, which created a ton of value and long term potential for the Toronto Maple Leafs club.

Also – I won’t go into it in depth here – but how did the Leafs acquire that original 24th overall pick? The Leafs got it by trading two ‘rental’ players (Franson/Santorelli), who were both about to become UFA’s, in return for that 1st Round pick, Brandon Leipsic (another solid prospect), and Olli Jokinen’s cap space. Keep an eye out for the Leaf’s front office to hopefully make a couple similar deals approaching the trade deadline this year, and repeat their excellent 2015 performance next June.

Leafs 2, League 0

Conclusion

When considering teams that have been successful in the NHL draft, the teams that come to mind intuitively support the findings of this analysis. The Chicago Blackhawks are a great example where their distinct strategy has been ‘quantity over quality’: rather than trying to pick ‘better’, as has been shown to be very difficult to do, Chicago has simply focused on using transactions like these to draft as many players as possible. Chicago’s massive, league leading number of picks from 2000-2004 show that the Hawks certainly planted their seeds – and in case anyone has been paying attention, they have been doing ‘OK’ in the last 5-10 years. Another great example of this strategy is Bill Belichick and the New England Patriots – who seem to have done alright in the last 10 or so years as well…

In the end – as one of the small but growing number of patient, excited Leafs fans out there, I will make our collective opinion clear: the current front office knows what they are doing – and we are behind them.

Appendix Table

What Draft Round Can Tell Us About a Player’s Expected Long Term Performance and Development (Part 1)

This article is being posted here as well as in parallel as a guest post at Maple Leaf Hotstove. FYI, for those who read my last post – as promised – this article is a succinct summary of my draft analysis ‘report’ shared earlier. 

The hockey analytics community has looked at many aspects when projecting a player’s performance over their career: prior league, prior scoring rate, performance of players with similar characteristics, size, and date of birth – amongst others. One example of such work was an article from earlier this year by ‘moneypuck’ at NHLnumbers.com.

Moneypuck’s analysis derives its foundation from an excellent study by Michael Shuckers in 2011, where Shuckers was one of the first to create a standardized view of ‘draft pick value’. The quality of Schuckers’ analysis drove many other authors to do work that followed suit, building on his approach. However, in his paper, Michael chose to define ‘draft pick value’ entirely based on likelihood to play >200 games in the league. Although that is a reasonable metric – and few would argue that reaching 200 NHL games means a draft pick was NOT successful – there are limits to using only a single metric to define ‘success’.

How ‘successful’ a pick was, and the ensuing value of a draft pick, is highly sensitive to how we choose to define success. Is a pick successful after 40 games, 80, or 200? Are they successful after 30 career points, or 100? How about their points per game? Or their total career points? How would our definition of ‘success’ change on each of these metrics if they are a forward or a defenseman?

As you can imagine – this isn’t a simple thing to answer. Earlier in 2015, Stephen Burtch did some interesting work down this path, where he combined expected GP with expected Pts/Gm to create a new draft pick value figure – which was a big step in the right direction. However, even Pts/Gm has its gaps, given that it only considers players still in the league. As time goes on, the least successful players will leave the league sooner, increasing the average Pts/Gm of those remaining. In a perfect world, we would want a metric that has already been adjusted for a player’s likelihood to succeed in the league, rather than one based on his success if he can stay in the league. (Though – to be fair – Burtch does seek to address this point through multiplying probability of reaching 200 games by expected Pts/Gm).

In order to address these points, I have taken a very detailed look at long term player performance and development based on draft round, incorporating a wide range of metrics into my analysis. Specifically, I have reviewed the five years of players drafted from 2000-2004, as well as the ensuing 9-13 years of NHL season data.

Arguably the biggest factor in whether or not analysis is put into practice is if a team’s front office and coaching staff truly understand it, and believe the results of the analysis enough to buy-in to it – which will often come down to the method by which that analysis is communicated. As such, I have tried to simplify the statistical methodology involved in this work, and display the output visually in a way that is easily understood and hopefully very accessible to stats and non-stats folks alike.

(As a note – this article focuses strictly on metrics related to player performance and development. However, a natural follow on to this is then connecting that information to draft pick value, as mentioned, and after that, how successful teams have been in drafting – both of which are covered in my full report, originally posted here). 

What I Hope To Answer

The objective of this analysis is to investigate ‘typical’ player performance and development trajectory after being drafted in a given round, in order to answer the following three questions:

  • If a player is drafted in round X, and is ultimately able to make the NHL, by when should they be expected to be a contributing NHL player?
  • How well does the typical player perform over the course of his career (on various metrics) after being selected in a given round?
  • Within the first round, how do the top 10 overall picks perform versus those taken 11th-30th?

So – let’s get into it.

Analysis of Long Term Player Performance and Development by Draft Round

I have split out the upcoming sections of analysis by each type of metric used. I will then revisit the three questions above directly in the final section on conclusions.

Games Played Thresholds

As Michael Shuckers showed very clearly – players drafted in the first 2-3 rounds are much more likely to appear in the NHL; however, the likelihood of a playing one or more full seasons diminishes substantially after the first round

Games Played Pic 1

Games Played Pic 2

In terms of player development, this data suggests that:

  • If a 1st round pick hasn’t played a game by their fourth potential NHL season, they likely will never appear in the NHL
  • 20-30% of successful 2nd and 3rd round players only begin to meaningfully play for their franchise between 5-7 years after being drafted (e.g. the pink shaded area on the ’80 games played’ chart)
  • The gap between the top 10 overall and the rest of the first round is actually relatively small when looking at the likelihood to pass the 150 game threshold (especially in comparison to metrics later in the article)
  • And, as we know – all other rounds after the first three appear to have close to equal likelihoods of producing long term NHL players

Points / GM Data

Forwards taken in the top 10 overall show an unbelievable ability to outperform in P/GM over their careers (which Stephen Burtch has shown is even more distributed within the top 10¸ where the top 1-3 picks overall are meaningfully better than picks 4-10)

Pts per Game - F

  • Interestingly, 2nd and 3rd round forwards tend to increase their per-game output over time, largely converging with players drafted 11th-30th overall
  • However, given this metric is an average of those still playing, there will be a survivorship bias that partially drives this effect
    • E.g. Low producers will leave the league more quickly, increasing the average for those remaining – as shown by the fact that 30% of the players shown are from Rd 1 in season ‘6’, this increases to 40% by season ‘10’
    • This data can more reasonably be said to tell us that, in order to stay in the NHL over the long term, a forward must achieve a minimum of roughly 0.20 points per game

Defensemen naturally display a much more narrow distribution of results, accounting for the fact that a ‘strong’ defenseman will not always play a significant point-scoring role

 Pts per Game - D

  • P/GM data for defensemen is not terribly insightful, but I have included it in order to provide the data for those interested
  • One note – If you look closely, you can see surprisingly strong (and erratic) performance of Round 5 defensemen – starting very weak (no points registered in season ‘2’), but then ultimately being among the highest points per game in seasons ‘5’ through ‘10’
    • This particular point is driven by a small sample size issue: 49 D were drafted in the 5th round, but only a handful played many games – three of whom happen to be John-Michael Liles, Kevin Bieksa, and James Wisniewski

Points Scored Thresholds

A player’s likelihood to surpass the 30-point threshold tends to resemble their likelihood to pass ~150 career games played…

Pts Threshold Pic 1

… However, players drafted in the third round fall behind in terms of likelihood to pass the 100-point career threshold

Pts Threshold Pic 2

  • Where earlier charts show strong similarities between the long term potential of 2nd and 3rd round players, the ability of those taken in the 2nd round to break the 100-point career threshold is a clear differentiator between the two
  • Based on this, teams may do well to target top scorers in rounds 1 and 2, before moving to defensemen, shut down forwards and goalies in the third round and onwards
  • Again, top 10 overall picks differentiate themselves here as well, with over 70% passing 100 career points

Cumulative Career Points

The ideal metric to compare performance by round must be adjusted for players with limited NHL careers – which brings me to Lifetime Production, or Cumulative Career Points Scored

Lifetime Production - F

Lifetime Production - D

(Note – Forwards and Defensemen are shown on different scales)

 

 

  • Here, 1st round picks wildly outperform all others, showing that the combined skill and typical longevity of even a mid-to-late 1st round player (11th-30th) will equate to an average 159 points over 10 seasons for forwards, and 105 points over the same timeframe for defensemen
  • This compares to the significantly lower 68 average career points for second round forwards, and 44 average career points for second round defense
  • Notably, third round forwards also re-assert their value here, showing that – although they will only typically produce a total of 36 points over 10 seasons – they still will consistently outperform rounds 4-9 in career points

Drawing Some Conclusions

Having now walked through each chart and its meaning, I want to summarize my findings from above. To do so, let’s revisit the original list of three questions:

If a player is drafted in round X, and is ultimately able to make the NHL, by when should they be expected to be a contributing NHL player?

  • First round players typically make their initial NHL appearance within 1-2 years, and will almost always have played their first full season (~80 games) by their fourth year after being drafted
  • 2nd and 3rd round players take much longer to develop, and many only play a full season by their 5th-7th years after being drafted
  • Players who haven’t played by these general timelines become highly unlikely to ever make serious NHL contributions (>1 season played)

How well does the typical player perform over the course of his career (on various metrics) after being selected in a given round?

  • Most players drafted outside the first round never make the league at all (2nd round players have a 60% likelihood of playing one game, and a 35% likelihood of playing a full season; for 3rd round players, closer to 40% play one game, and only 28% play a full season)
  • Based on their combined likelihood to play 2+ NHL seasons, score 30+ NHL points, and reach 0.4-0.5 or more pts/gm, 1st, 2nd and 3rd round players are the only players with a meaningfully higher likelihood in succeeding in the league
  • However, based on the likelihood to score >100 NHL points, 1st and 2nd round players are able to separate themselves from the 3rd round as well

Within the first round, how do the top 10 overall picks perform versus those taken 11th-30th?

  • The top 10 overall picks are significantly more capable than all others, even versus their first round peers
  • Over 70% of top 10 overall picks pass 100 career points, typically after ~6 seasons, versus 50% of those picked 11th-30th, who often take 9-10 years or more
  • Only forwards taken in the top 10 overall can truly be expected to score 0.6-0.7 pts/gm or more over their careers (although there are many examples of players who perform at this level of production that were taken outside the top 10, such as Ryan Getzlaf and Corey Perry)
  • In a hypothetical trade for active players, a ‘typical’ top-10 overall pick should be treated as likely reaching >350 career points as a F, or >170 as a D – thus, one-for-one, a team should be expecting to get a true star player in return if they are giving up a potential top 10 overall pick

In the end…

The long term performance expected of a player based on their draft round is something that is highly relevant to teams throughout their decisions in trades, on draft day, and in supporting a player’s development over his career. Hopefully you have found this analysis to be interesting, and found that the work was also able to build upon what is already out there by expanding the range of metrics that we look at. As mentioned, the PDF I linked to above also begins to apply this to both a revised (and straightforward) metric of draft pick value, as well as to answer the question of ‘Which teams were the most successful?’ in the draft years studied. Keep an eye out for ‘Part 2’ of this article – where I apply the data above to the trades done by the Leafs on Draft Day last summer, in order to see if Hunter, Dubas and friends were winners or losers in their exciting deals…

This article is presented by OAK Coasters, a website where you can by beautifully crafted, hand made One of A Kind (OAK) coasters that make the perfect gift. Check them out at OAKCoasters.com.

 

 

 

What is an NHL Draft Pick Really Worth?

A Detailed Analysis of Player Performance and Development by Draft Round

It is hard to describe drafting as anything less than essential to the success of a professional sports franchise. The teams that are able to plant the seeds for long term success on draft day each year will have a clear advantage over the five to ten years that follow – if not sooner. We all know that every NHL franchise has an expert scouting team, and many likely use metrics like league equivalencies to see what they are getting from a pick – but how many teams know what to expect from a player’s long term performance based on the round they drafted him? Further, how would that knowledge impact how a team assigns ‘value’ to its future picks in trades?

In order to answer these, and other questions, I have dug into some of the data available, and summarized into a report that you can find hereWarning – it is long. In this ‘report’, I try to answer the two questions above, as well as a more detailed list of questions that you can find at the bottom of the page.  Quickly, thanks to Hockey-Reference.com for the draft year data, and HockeyAbstract.com for the historical NHL season data.

As the first piece of work I am releasing to the NHL analytics world, the most basic reason that I have focused on draft analysis is because I personally am very curious about it. Like many fans, I often find myself looking at the exciting picks selected each year, but previously I had not quantitatively understood what to expect from those players in the NHL – specifically on a long term, year by year basis, generalized to the round they were drafted in. Before starting to analyze some of the data available, I also didn’t have a full appreciation of how a team should be treating the long term potential value behind those picks versus what else is out there (though some great work has been done on that in the past, by Stephen Burtch and Michael Shuckers, amongst others).

Lastly, I also think draft analysis has a number of strong parallels to the analysis done in value investing (aka the purpose of this blog). When an investor is evaluating a potential company to buy, the major focus is on its growth and earnings potential over a very long term time horizon, e.g. 4-7 years, or more. Much like ‘investing’ in the potential, growth and long term development of a player, the same type of thought process needs to be followed when buying a company. Investors operate with scarce resources and other constraints (e.g. contract limits, salary caps), they must have a clear understanding of their own strategy and needs (e.g. immediate cup contender vs. rebuild scenario), they have to undergo prioritization by evaluating the alternatives against a set of criteria (e.g. prioritizing positions, ranking players in each), and ultimately they have to make a decision on how to move forward – a decision that they will often have to live with for half a decade or more (in the case of private companies).

In investing, as in drafting to help build an NHL organization – the more detailed, understandable, and accurate quantitative information you have to support your decisions, the more likely you are to be successful. Thus, as the NHL enters the months before the trade deadline, I hope the attached analysis can give the online community some food for thought as to who the winners and losers are of trades to come. Please let me know any comments, questions, feedback or areas for further analysis at @OrgSixAnalytics or OriginalSixAnalytics@gmail.com.

 

This article is presented by OAK Coasters, a website where you can by beautifully crafted, hand made One of A Kind (OAK) coasters that make the perfect gift.  Check them out at OAKCoasters.com.

 

For reference, the analysis seeks to answer the following questions:

  • If a player is drafted in round X, and is ultimately able to make the NHL, by when should they be expected to be a contributing NHL player?
  • How well does the typical player perform over the course of his career (on various metrics) after being selected in a given round? Within the first round, how do the top 10 overall picks perform versus those taken 11th-30th?
  • How much more valuable is a pick in the first round versus the other rounds? All things being equal, what should a pick from each round be worth in a trade?
  • Which teams were the most effective at drafting in the period sampled?

 

 

About the author, and this site

Original Six Analytics is (yet another) blog focused on exploring the world of advanced statistics within the NHL and other hockey leagues. My own background is in the field of private equity investing, and before that as a management consultant – both fields heavily focused on summarizing quantitative analysis into easily understood output in order to support decision making. As such, I hope to specifically use this blog to apply the concepts of value investing (and to an extent, corporate strategy) to the world of hockey analysis.

Lastly – as the image of Morgan Rielly and Leo Komarov may suggest – I am a lifelong Leaf fan, with a great deal of patience and support behind the approach of the current front office. The name of ‘Original Six’ Analytics refers to what seems to be the vision of the current Leafs organization – ‘returning an original six franchise to its rightful place in the league’. Which I think we can all get behind.

My inaugural post (with actual analytics content) will be coming soon, on the topic of the value of draft picks, based on typical player performance and development from a given draft round. Please feel free to reach out at any time, with questions, comments, feedback or any requests for a future type of analysis:

  • @OrgSixAnalytics on Twitter
  • OriginalSixAnalytics@gmail.com