This all got me thinking about statistics when involved with sports and how subjectivity should be avoided at all costs. The other major leagues rely primarily upon objective stats (the MLB error is the only subjective stat that comes to mind) as does hockey for the most part (goals, assists, +/-, shutouts, wins, losses etc…). However, lately there has been an increased interest in RTSS (Real-Time Scoring System) stats and I have seen these metrics used more and more when comparing players and teams. For those unfamiliar with the concept of RTSS, it is the use of a dedicated scorer to record the total number of hits, blocked shots, missed shots, giveaways, takeaways and faceoff outcomes for each team. It has been in place since the beginning of the ’97-’98 season but has only become accessible to everyday fans in the past year or so. Every NHL boxscore on ESPN features these statistics prominently, alongside shot and PIM totals, above the scoring summary. The placement of these subjective figures atop the page next to more traditional, objective statistics gives them an importance and legitimacy that is not truly deserved. Thankfully, the other main hockey/sports sites (SI, TSN, NHL) do not list this ‘information’ at all, or make it difficult to find as is the case with NHL.
Now don’t get me wrong... I am a big proponent of using statistics to compare players and teams. I love the fact that more and more statistical analyses are becoming available and that hockey and the NHL are moving in that direction. The game of baseball has changed dramatically over the past decade or two thanks to the advent of SABRmetrics and similar statistics which delve further into how the game is played. These advanced analyses have allowed for the objective comparison of players from any era by measuring those players against their peers (the only possible way to legitimately compare players of different eras). I have seen a similar approach taken with RTSS stats by adjusting the numbers to remove any inherent bias from certain arena scorers. However, too many questions arise with their methodology and the concept in general...
- What equations are to be used?
- How is a bias judged?
- How is it to be done?
- Will it be a rolling adjustment done after every game?
- Will it be based off last season's biases?
- Will it have to wait until season's end when all the numbers have been accumulated?
- What happens if the scorer, and thus the bias, changes mid-season?
Hits – How do you define a hit? How hard does it have to be? Does a player have to be knocked to the ice or does a simple two-handed shove count? Sure, there are plenty of bruising hits with solid contact but there are just as many instances where some level of contact is made… how much constitutes an official hit? Also, while there are plenty of instances where there is a clear-cut aggressor, what happens when both players are trying to be physical? Does the hit get credited to both players or only to one, and if the latter who ‘wins’ the hit?Admittedly, the extent of the subjectivity in these statistics varies with hits, giveaways and takeaways having the potential to be ruled by scorer bias while the other three instances are more clear cut. However, the situations where each stat is opened to subjectivity (bias) are enough to skew the data over the course of the season and make them statistically useless. To prove the deficiencies of these stats, I will provide examples in my next entry...
Blocked Shots – Is intent considered or does dumb luck play a role in this stat? Does the shot have to actually be on net (or appear to be doing so) to count as a block or is it merely recognition of the bravery to step in front of that vulcanized rubber disc, regardless of its destination? If the shot is blocked and caroms into the net, does it count as a blocked shot, a positive stat, despite the negative impact of the play?
Missed Shots – Same issue as the blocked shots but in reverse… does a shot not on net count as a miss even if blocked? And what about shots that are clearly going over the crossbar (and thus not on net) yet are gloved by the goalie… is that a shot or a missed shot?
Faceoff wins – How is a faceoff win determined? Is possession the only important thing or does it have to do with which side of the dot the puck ended up on? In other words, if the Boston center knocks the puck back towards his side but an opposing player is the first to gain possession who gets the win? The opposition clearly has the puck and thus won the faceoff and yet the Boston center did nothing to merit a loss since he controlled the puck better than his opponent.
Giveaway/Takeaway – Once again… what is the definition here? Apparently the two are mutually exclusive (a play can only be one or the other, never both) and therefore the two numbers do not have to balance out. Still the question becomes what counts as what… does a failed dump-in count as a giveaway or takeaway? Does an intentional icing or slow-roller (to prevent icing) count as a giveaway? What about errant passes… do they count and if so, who gets (dis)credited with it, the passer or intended receiver? Not always clear who is at fault for the missed opportunity. Or do they simply count as a takeaway regardless of whether the pass would have been ‘completed’?
3 comments:
I just wrote a wicked long response but there was an error on Blogger. Miserable.
Solid piece by the way. The stats you mention used to be referred to as “intangibles.” Nowadays, it seems now there’s a stat for everything. If you think about it, it’s the American way; we’re always looking for a way to make things bigger, faster, and better—to make the best product possible. Sadly, it takes a lot of the fun and humanity out of the game. What it really comes down to is money. Owners are businessmen and not athletes. They want to know exactly how productive or expendable a certain player is, so they analyze absolutely everything about the game. They need statistics to back everything up—to ensure that they are putting the best “product” on the ice. It’s brutal because it is a business.
On the other hand, the only stat that you mentioned that I think is worthy of being a stat is face-off winning percentage. Even high school teams keep that stat. It’s important because if the game is on the line, then you need to know who has the best percentage. In the last minute of a game, the game can be won or lost in the face-off circle and you want to be able to put someone out there that you trust will win the draw.
By the way, guess who led the 00’ Arlington Spy Ponders with a FO win percentage of 75%? Stevie Yzerman, eat your heart out.
Interesting read, and I agree with you on a lot of points. Hockey is subject to a lot of these biased stats, maybe more so than any other major sport. Have you heard of Puckprospectus.com? It's from the same people who brought us Baseball Prospectus. Should be interesting to see what the come up with.
A few years back I used to keep RTSS for NESN during games and my numbers always differed slightly from the 'official' count given by the game scorer. Always pissed me off ...
Post a Comment