This all got me thinking about statistics when involved with sports and how subjectivity should be avoided at all costs. The other major leagues rely primarily upon objective stats (the MLB error is the only subjective stat that comes to mind) as does hockey for the most part (goals, assists, +/-, shutouts, wins, losses etc…). However, lately there has been an increased interest in RTSS (Real-Time Scoring System) stats and I have seen these metrics used more and more when comparing players and teams. For those unfamiliar with the concept of RTSS, it is the use of a dedicated scorer to record the total number of hits, blocked shots, missed shots, giveaways, takeaways and faceoff outcomes for each team. It has been in place since the beginning of the ’97-’98 season but has only become accessible to everyday fans in the past year or so. Every NHL boxscore on ESPN features these statistics prominently, alongside shot and PIM totals, above the scoring summary. The placement of these subjective figures atop the page next to more traditional, objective statistics gives them an importance and legitimacy that is not truly deserved. Thankfully, the other main hockey/sports sites (SI, TSN, NHL) do not list this ‘information’ at all, or make it difficult to find as is the case with NHL.
Now don’t get me wrong... I am a big proponent of using statistics to compare players and teams. I love the fact that more and more statistical analyses are becoming available and that hockey and the NHL are moving in that direction. The game of baseball has changed dramatically over the past decade or two thanks to the advent of SABRmetrics and similar statistics which delve further into how the game is played. These advanced analyses have allowed for the objective comparison of players from any era by measuring those players against their peers (the only possible way to legitimately compare players of different eras). I have seen a similar approach taken with RTSS stats by adjusting the numbers to remove any inherent bias from certain arena scorers. However, too many questions arise with their methodology and the concept in general...
- What equations are to be used?
- How is a bias judged?
- How is it to be done?
- Will it be a rolling adjustment done after every game?
- Will it be based off last season's biases?
- Will it have to wait until season's end when all the numbers have been accumulated?
- What happens if the scorer, and thus the bias, changes mid-season?
Hits – How do you define a hit? How hard does it have to be? Does a player have to be knocked to the ice or does a simple two-handed shove count? Sure, there are plenty of bruising hits with solid contact but there are just as many instances where some level of contact is made… how much constitutes an official hit? Also, while there are plenty of instances where there is a clear-cut aggressor, what happens when both players are trying to be physical? Does the hit get credited to both players or only to one, and if the latter who ‘wins’ the hit?Admittedly, the extent of the subjectivity in these statistics varies with hits, giveaways and takeaways having the potential to be ruled by scorer bias while the other three instances are more clear cut. However, the situations where each stat is opened to subjectivity (bias) are enough to skew the data over the course of the season and make them statistically useless. To prove the deficiencies of these stats, I will provide examples in my next entry...
Blocked Shots – Is intent considered or does dumb luck play a role in this stat? Does the shot have to actually be on net (or appear to be doing so) to count as a block or is it merely recognition of the bravery to step in front of that vulcanized rubber disc, regardless of its destination? If the shot is blocked and caroms into the net, does it count as a blocked shot, a positive stat, despite the negative impact of the play?
Missed Shots – Same issue as the blocked shots but in reverse… does a shot not on net count as a miss even if blocked? And what about shots that are clearly going over the crossbar (and thus not on net) yet are gloved by the goalie… is that a shot or a missed shot?
Faceoff wins – How is a faceoff win determined? Is possession the only important thing or does it have to do with which side of the dot the puck ended up on? In other words, if the Boston center knocks the puck back towards his side but an opposing player is the first to gain possession who gets the win? The opposition clearly has the puck and thus won the faceoff and yet the Boston center did nothing to merit a loss since he controlled the puck better than his opponent.
Giveaway/Takeaway – Once again… what is the definition here? Apparently the two are mutually exclusive (a play can only be one or the other, never both) and therefore the two numbers do not have to balance out. Still the question becomes what counts as what… does a failed dump-in count as a giveaway or takeaway? Does an intentional icing or slow-roller (to prevent icing) count as a giveaway? What about errant passes… do they count and if so, who gets (dis)credited with it, the passer or intended receiver? Not always clear who is at fault for the missed opportunity. Or do they simply count as a takeaway regardless of whether the pass would have been ‘completed’?
