Puck Luck: Understanding PDO



Ice hockey analytics have improved by leaps and bounds in recent years. Many of the best new statistics (like Corsi and Fenwick) measure different kinds of shot attempts. Another newer statistic, PDO, is supposedly a measure of luck.

PDO was originally introduced by Brian King on a now-defunct blog. It doesn’t stand for anything: it was an online screen name of King’s. A team’s PDO is defined to be their shooting percentage plus their save percentage. Usually PDO counts shots when neither team is shorthanded, i.e., in 5v5 situations. A player’s individual PDO is the sum of his team’s shooting and save percentages while that player is on the ice.

In theory, PDO measures luck. The idea is that teams can’t actually do much to change their shooting percentage and save percentage. If a team scores 6 goals on 30 shots, they got lucky and aren’t expected to maintain such a high shooting percentage.

Extreme values of PDO over short periods of time can certainly be explained by random chance, but some teams maintain moderately high or low PDO over the course of an 82-game season. The 2018-19 New York Islanders, for example, finished the season with a PDO of 1.022 and a league-leading .937 save percentage in 5v5 situations. Can this be reasonably explained by chance error, or did the Islanders benefit from elite goaltending and defensive play?

The Data

I downloaded 2018-19 statistics from Natural Stat Trick. I only considered 5v5 situations.

In [1]:
import pandas as pd
In [2]:
df = pd.read_csv('Team Season Totals - Natural Stat Trick.csv', index_col='Team')

df.head()
Out[2]:
Unnamed: 0 GP TOI W L OTL ROW Points Point % CF LDSA LDSF% LDGF LDGA LDGF% LDSH% LDSV% SH% SV% PDO
Team
Montreal Canadiens 1 82 3985.9500 44 30 8 41 96 0.585 4358 904 52.94 29 16 64.44 2.85 98.23 8.16 91.91 1.001
Toronto Maple Leafs 2 82 4084.4833 46 28 8 46 100 0.610 4396 985 46.17 37 21 63.79 4.38 97.87 9.47 92.43 1.019
Boston Bruins 3 82 3994.0333 49 24 9 47 107 0.652 3867 766 54.83 21 25 45.65 2.26 96.74 7.34 93.11 1.005
Washington Capitals 4 82 3934.5167 48 26 8 44 104 0.634 3713 840 49.67 37 17 68.52 4.46 97.98 10.09 92.08 1.022
Calgary Flames 5 82 3903.1167 50 25 7 50 107 0.652 3946 728 53.78 27 29 48.21 3.19 96.02 9.09 91.81 1.009

5 rows × 71 columns

Some of the numbers (including PDO) are rounded to two or three decimal places. We’ll compute more precise values directly from the SF, SA, GF, and GA columns (that’s shots for, shots against, goals for, and goals against). We’ll drop each of the other columns and recalculate SH% (shooting percentage), SV% (save percentage), and PDO.

In [3]:
df.columns
Out[3]:
Index(['Unnamed: 0', 'GP', 'TOI', 'W', 'L', 'OTL', 'ROW', 'Points', 'Point %',
       'CF', 'CA', 'CF%', 'FF', 'FA', 'FF%', 'SF', 'SA', 'SF%', 'GF', 'GA',
       'GF%', 'xGF', 'xGA', 'xGF%', 'SCF', 'SCA', 'SCF%', 'SCSF', 'SCSA',
       'SCSF%', 'SCGF', 'SCGA', 'SCGF%', 'SCSH%', 'SCSV%', 'HDCF', 'HDCA',
       'HDCF%', 'HDSF', 'HDSA', 'HDSF%', 'HDGF', 'HDGA', 'HDGF%', 'HDSH%',
       'HDSV%', 'MDCF', 'MDCA', 'MDCF%', 'MDSF', 'MDSA', 'MDSF%', 'MDGF',
       'MDGA', 'MDGF%', 'MDSH%', 'MDSV%', 'LDCF', 'LDCA', 'LDCF%', 'LDSF',
       'LDSA', 'LDSF%', 'LDGF', 'LDGA', 'LDGF%', 'LDSH%', 'LDSV%', 'SH%',
       'SV%', 'PDO'],
      dtype='object')
In [4]:
df.drop(['Unnamed: 0', 'GP', 'TOI', 'W', 'L', 'OTL', 'ROW', 'Points', 'Point %',
       'CF', 'CA', 'CF%', 'FF', 'FA', 'FF%', 'SF%',
       'GF%', 'xGF', 'xGA', 'xGF%', 'SCF', 'SCA', 'SCF%', 'SCSF', 'SCSA',
       'SCSF%', 'SCGF', 'SCGA', 'SCGF%', 'SCSH%', 'SCSV%', 'HDCF', 'HDCA',
       'HDCF%', 'HDSF', 'HDSA', 'HDSF%', 'HDGF', 'HDGA', 'HDGF%', 'HDSH%',
       'HDSV%', 'MDCF', 'MDCA', 'MDCF%', 'MDSF', 'MDSA', 'MDSF%', 'MDGF',
       'MDGA', 'MDGF%', 'MDSH%', 'MDSV%', 'LDCF', 'LDCA', 'LDCF%', 'LDSF',
       'LDSA', 'LDSF%', 'LDGF', 'LDGA', 'LDGF%', 'LDSH%', 'LDSV%', 'SH%',
       'SV%', 'PDO'], axis=1, inplace=True)
In [5]:
df['SH%'] = df['GF']/df['SF']
df['SV%'] = (df['SA']-df['GA'])/df['SA']
df['PDO'] = df['SH%']+df['SV%']

Significance Testing

In order to do a statistical test on each team’s PDO, we need to know how PDO is distributed. In order to know how PDO is distributed, we need to know how SH% and SV% are distributed.

If p is the probability of scoring a goal on a single shot, then shooting percentage after n shots is binomially distributed (and hence approximately normally distributed when n is large) with mean p and standard deviation \sqrt{\frac{p(1-p)}{n}}. Similarly, save percentage for m shots against will be normally distributed with mean 1-p and standard deviation \sqrt{\frac{(1-p)p}{m}} (as long as m is large enough). I’ll estimate p by taking the league-wide average shooting percentage, and then compute SH% and SV% standard deviations for each team.

In [6]:
p = sum(df['GF'])/sum(df['SF'])

p
Out[6]:
0.08059031018174341
In [7]:
df['SH% std dev'] = (p*(1-p)/df['SF'])**0.5
df['SV% std dev'] = ((1-p)*p/df['SA'])**0.5

PDO is the sum of SH% and SV%, and the sum of two normally distributed random values is also normal. If X is normal with mean \mu_X and standard deviation \sigma_X and Y is normal with mean \mu_Y and standard deviation \sigma_Y, then X+Y is normal with mean \mu_X+\mu_Y and standard deviation \sqrt{\sigma_X^2+\sigma_Y^2+2\rho\sigma_X\sigma_Y} where \rho is the correlation between X and Y.

Intuitively, shooting percentage and save percentage should be uncorrelated. In other words, \rho should be 0. SciPy’s pearsonr function can test this.

In [8]:
from scipy import stats
In [9]:
stats.pearsonr(df['SH%'],df['SV%'])
Out[9]:
(-0.1482567896939233, 0.4260607970853276)

The sample correlation between SH% and SV% is about -0.148 and the p-value is about 0.426. This means that uncorrelated quantities have a 42.6% chance of producing a sample correlation that far from 0, so it seems reasonable to assume that SH% and SV% are uncorrelated. Now we’re ready to compute PDO standard deviations and z-scores for each team.

In [10]:
df['PDO std dev'] = (df['SH% std dev']**2+df['SV% std dev']**2)**0.5
df['PDO z-score'] = (df['PDO']-1)/df['PDO std dev']

df.head()
Out[10]:
SF SA GF GA SH% SV% PDO SH% std dev SV% std dev PDO std dev PDO z-score
Team
Montreal Canadiens 2303 2016 188 163 0.081633 0.919147 1.000779 0.005672 0.006062 0.008302 0.093888
Toronto Maple Leafs 2176 2247 206 170 0.094669 0.924344 1.019013 0.005835 0.005742 0.008187 2.322309
Boston Bruins 2125 1844 156 127 0.073412 0.931128 1.004540 0.005905 0.006339 0.008663 0.524029
Washington Capitals 1952 2033 197 161 0.100922 0.920807 1.021729 0.006161 0.006037 0.008626 2.519038
Calgary Flames 2058 1819 187 149 0.090865 0.918087 1.008952 0.006000 0.006382 0.008760 1.021891

For any given team, we could do a z-test to determine if there is a statistically significant difference in PDO. Each null hypothesis is that a team’s PDO will converge to 1000. Of course, the more tests we run the more likely it is that we’ll get a false-positive results. Running 31 tests (one for each team) makes false-positives very likely.

We’ll use the Holm-Bonferri method to control the probability of getting a false-positive result. We’ll sort our DataFrame by PDO p-value and create a Holm-Bonferri column with \alpha=0.05. If a team’s PDO p-value is less than the entry in the Holm-Bonferri column, we will reject the null hypothesis for that team (and every other team above it in the DataFrame).

In [11]:
df['PDO p-value'] = 2*stats.norm.cdf(-abs(df['PDO z-score']))

df.sort_values('PDO p-value', inplace=True)
In [12]:
alpha = 0.05

df['k'] = range(1,len(df)+1)
df['Holm-Bonferri'] = alpha/(len(df)+1-df['k'])

df
Out[12]:
SF SA GF GA SH% SV% PDO SH% std dev SV% std dev PDO std dev PDO z-score PDO p-value k Holm-Bonferri
Team
New York Islanders 1891 2002 162 127 0.085669 0.936563 1.022232 0.006260 0.006084 0.008729 2.546982 0.010866 1 0.001613
Washington Capitals 1952 2033 197 161 0.100922 0.920807 1.021729 0.006161 0.006037 0.008626 2.519038 0.011768 2 0.001667
Toronto Maple Leafs 2176 2247 206 170 0.094669 0.924344 1.019013 0.005835 0.005742 0.008187 2.322309 0.020216 3 0.001724
Tampa Bay Lightning 2102 2003 206 157 0.098002 0.921618 1.019619 0.005937 0.006082 0.008500 2.308297 0.020983 4 0.001786
Minnesota Wild 2072 1858 140 155 0.067568 0.916577 0.984145 0.005980 0.006315 0.008697 -1.823075 0.068292 5 0.001852
Florida Panthers 2062 2013 162 189 0.078565 0.906110 0.984675 0.005994 0.006067 0.008529 -1.796855 0.072359 6 0.001923
Pittsburgh Penguins 2198 2170 182 150 0.082803 0.930876 1.013678 0.005806 0.005843 0.008237 1.660477 0.096818 7 0.002000
New Jersey Devils 1933 2072 146 184 0.075530 0.911197 0.986727 0.006191 0.005980 0.008608 -1.541974 0.123080 8 0.002083
Buffalo Sabres 2133 2184 154 185 0.072199 0.915293 0.987492 0.005894 0.005825 0.008286 -1.509485 0.131175 9 0.002174
San Jose Sharks 2127 1793 192 185 0.090268 0.896821 0.987089 0.005902 0.006428 0.008727 -1.479437 0.139024 10 0.002273
Arizona Coyotes 1982 2002 131 157 0.066095 0.921578 0.987673 0.006114 0.006084 0.008625 -1.429144 0.152963 11 0.002381
Calgary Flames 2058 1819 187 149 0.090865 0.918087 1.008952 0.006000 0.006382 0.008760 1.021891 0.306833 12 0.002500
Edmonton Oilers 1906 2099 146 178 0.076600 0.915198 0.991798 0.006235 0.005941 0.008613 -0.952345 0.340922 13 0.002632
Los Angeles Kings 1874 2058 140 170 0.074707 0.917396 0.992102 0.006288 0.006000 0.008692 -0.908698 0.363509 14 0.002778
Carolina Hurricanes 2232 1866 160 148 0.071685 0.920686 0.992371 0.005762 0.006301 0.008538 -0.893541 0.371568 15 0.002941
Vegas Golden Knights 2233 1916 173 162 0.077474 0.915449 0.992923 0.005760 0.006219 0.008477 -0.834867 0.403792 16 0.003125
Winnipeg Jets 1995 2123 167 163 0.083709 0.923222 1.006931 0.006094 0.005908 0.008488 0.816604 0.414155 17 0.003333
Philadelphia Flyers 1977 2118 165 190 0.083460 0.910293 0.993753 0.006122 0.005915 0.008512 -0.733920 0.462998 18 0.003571
Vancouver Canucks 1872 2089 145 175 0.077457 0.916228 0.993685 0.006291 0.005956 0.008663 -0.728935 0.466042 19 0.003846
Ottawa Senators 1956 2361 170 219 0.086912 0.907243 0.994155 0.006155 0.005602 0.008323 -0.702341 0.482467 20 0.004167
Chicago Blackhawks 2132 2267 183 184 0.085835 0.918835 1.004670 0.005895 0.005717 0.008212 0.568718 0.569548 21 0.004545
Nashville Predators 2133 1951 164 141 0.076887 0.927729 1.004616 0.005894 0.006163 0.008527 0.541361 0.588259 22 0.005000
Boston Bruins 2125 1844 156 127 0.073412 0.931128 1.004540 0.005905 0.006339 0.008663 0.524029 0.600259 23 0.005556
Columbus Blue Jackets 2122 2022 187 170 0.088124 0.915925 1.004049 0.005909 0.006053 0.008459 0.478664 0.632178 24 0.006250
Dallas Stars 1980 2056 136 133 0.068687 0.935311 1.003998 0.006117 0.006003 0.008571 0.466479 0.640873 25 0.007143
New York Rangers 1869 2132 145 172 0.077582 0.919325 0.996906 0.006296 0.005895 0.008625 -0.358686 0.719830 26 0.008333
Detroit Red Wings 1886 2202 147 177 0.077943 0.919619 0.997561 0.006268 0.005801 0.008540 -0.285557 0.775217 27 0.010000
St Louis Blues 2049 1859 166 147 0.081015 0.920925 1.001940 0.006013 0.006313 0.008719 0.222546 0.823889 28 0.012500
Colorado Avalanche 2046 1978 156 153 0.076246 0.922649 0.998895 0.006018 0.006120 0.008583 -0.128682 0.897610 29 0.016667
Montreal Canadiens 2303 2016 188 163 0.081633 0.919147 1.000779 0.005672 0.006062 0.008302 0.093888 0.925198 30 0.025000
Anaheim Ducks 1845 2068 136 154 0.073713 0.925532 0.999245 0.006337 0.005986 0.008717 -0.086650 0.930950 31 0.050000

In each case, we do not reject the null hypothesis. This means that we can’t conclude that any individual team had a statistically significant effect on their PDO. This doesn’t mean that there is no effect, it just means this test can’t detect any effect.

The upshot of all this is that PDO really is a robust measure of luck. The Islanders’ PDO of 1.022 is only about 2.5 standard deviations away from 1.000. 2.5 is rather large in terms of standard deviation, but it’s not unusual to see in a sample of 31 teams. I still suspect that some teams can affect their PDO without getting lucky, but not by much. Sooner or later, luck will even things out.

Leave a Reply

Your email address will not be published.