At the MIT Sloan Sports Analytics Conference, discussion generally surrounds the quant-friendly nuances of how the statistics of the game can be collected, analyzed and adjusted to provide the closest numerical representation of the truth. What are the limitations of Player Efficiency Rating? How valid is adjusted plus-minus? Can any of these advanced numbers ever really show us an objective reality or are they all too biased by the contextual roles that players have within their unique roles on their teams?
These are all fine debates and ones that will continue to rage on throughout every corner of the Dorkapalooza community.
What is most relevant to NBA fans today, however, is how front offices across the league are using these numbers to make decisions in a practical sense. Some of these answers became clearer on Saturday as Mavs owner Mark Cuban, Blazers GM Kevin Pritchard and Celtics Assistant GM Mike Zarren gathered to speak on a panel alongside two of the NBA’s statistical pioneers, Dean Oliver, who works with the Nuggets, and John Hollinger, who we all know from ESPN.
How do team execs really use the numbers?
“Depends on the time of year,” said Pritchard, noting that different stats mean different things depending on whether he is thinking about the trade deadline, the draft or optimizing lineups to match up with an opponent in a seven-game series. “Overall, it’s on the personnel side.”
Trade and free agency decisions are increasingly being made with more statistical information, and Oliver broke down how widespread this is all becoming and stated that he knows of eight teams that have actually integrated advanced analytics into their decision-making (Boston, Cleveland, Dallas, Denver, Houston, Oklahoma City, Orlando and Portland). In all, he says that he saw 14 different teams with personnel on the attendee list for Sloan Conference this year — and knew of two other assistant GMs who were not listed. Kevin Pelton also broke down which statisticians are now working in the league, in the process, revealing that he is now consulting with Indiana.
Cuban has been bullish on gathering data since he first bought the team, and early on began consulting with noted statistician Wayne Winston, who he had as a professor for a statistics course while attending Indiana University and had lost touch with until he got to the NBA. “I hadn’t seen him until three weeks before I bought the team. I saw him on Jeopardy, and thought, ‘Hey, I should give that guy a call.’”
In his early days, he admittedly made some mistakes. At least twice during the day, Cuban mentioned Evan Eschmeyer as a guy who he overvalued — and badly overpaid — based largely on some plus-minus data that, in retrospect, he realizes was based on too small of a sample size to translate to the future.
That was something Cuban learned from, but it didn’t deter his reliance on plus-minus, which he not only used recently to make a perhaps sea-changing trade for the Western Conference, but to help decide to hire Rick Carlisle before the 2008-09 season. He ran the numbers, and found that Carlisle was the NBA coach who had the greatest positive affect on the plus-minus rankings of those players who joined new teams. “It was Rick by a long shot,” said Cuban earlier in the day.
A big challenge to all this, however, is just gathering the data. “[Only] 20% or maybe a quarter of defense shows up in a box score,” said Hollinger. Steals, blocks and personal fouls are there, but what happens on all the other plays is not. Who forced a shooter to miss? Who blew a rotation? The box score will never tell you that.
“The box score is an incomplete story,” said Pritchard. “And more than that, it can be misleading.”
Still, unless you have the time to watch, chart and analyze every play qualitatively, the numbers — many of which can now be instantaneously collected automatically from play-by-play data — provide an invaluable base level of evidence on which to make better decisions. Oliver summed it up perfectly. “Individuals see a game better than the numbers,” he said. “But the numbers see all the games.”
Cuban and Rockets GM Daryl Morey, who run the Sloan Conference in partnership with MIT, have devoted a ton of money and organizational resources to mining all the games to find the data that goes beyond the box score. But they have both expressed their desire to see more of this being done by the league.
“Evaluating players, you have to do a lot more work and that’s what’s frustrating to us,” said Cuban. “You have to have someone charting every play. And there is no more inefficient use of someone’s time.”
Zarren understands the frustration and knows that Boston and the other major teams embracing data mining are probably wasting resources just to come up with the same data that other teams are unveiling. “There has to be a lot of duplication of work going on.”
Some have suggested that if the league — or some third party provider — does the work and makes it available to all the teams, it would take away some of the competitive edge for those on the cutting-edge. Cuban doesn’t seem worried about that and feels like it is what you do with the general data that really matters. “We all have our own special sauce,” he said. “We’re only talking about the data gathering … All of the teams are going to catch on, we may as well nip it in the bud.”
Hollinger, whose PER metric is all-too-often proselytized as a Holy Grail player ranking despite his insistence that that is not its utility, similarly seems to believe that most advanced numbers are more important within the context of individual front offices than they are when used as some monolithic, numerical judge of every player in the league. He says that he “wouldn’t want to pick the All-Star teams” based on league-wide advanced statistical measures alone, but, within the operating philosophy of each front office, almost all of these numbers — when put into proper context — can be used to help make better decisions.
Getting the numbers from the stat heads to the people coaching the team is the next hurdle. “As important as the work you do is how you communicate it,” said Zarren. In Boston, it took him and Doc Rivers a while to understand each other and the vocabulary barrier is something that will always be difficult to overcome.
Dean Oliver has had similar experiences working with the Nuggets, but has learned a lot in his time there and the whole organization has continued to improve its ability to talk the same language. “These communication skills are not trivial,” he said. “And improving these skill may be more important than improving how you calculate adjusted plus-minus.”
Once everyone is one the same page, teams can start seeing some real results. Coaches can tailor their systems to the overall organizational philosophy and get the players to do those things they were brought in by the GMs to do. And while a guy like Shane Battier has famously embraced this from the player side, a lot of this stuff can stop with the coaches. It may not be necessary to have the players in on the math.
“It’s really important for the coaches to design schemes around the data,” said Zarren. “But it’s not important for the players to know everything that went into designing it.”
The goal should be to integrate the analysis into the overall coaching philosophy, but for many players — some of which can’t even remember the plays they are supposed to run — advanced stats are not something that can be used to change the way they play.
“We had Gerald Green,” said Cuban, with a glance over towards Zarren. “You had Green. He does stuff [athletically] that makes you say ‘Oh my God!’ … He just doesn’t understand the game of basketball.”
Offering further evidence of the difficulty in putting any of this into the players hands, Cuban talked about the logistical problems presented by the arduous schedule of the NBA season. “We haven’t had a practice since the trade,” he said in reference to the deal that brought Caron Butler and Brendan Haywood to the Mavs.
“We’ve had two since then,” said Zarren.
But while the numbers are not something Cuban will be using immediately to help Haywood play better individual defense as the Mavs make a run at an NBA title, they are a big reason he is now in Dallas. “Defensive numbers absolutely had a large part in the trade for Brendan Haywood,” he said. “And we wouldn’t have done the trade without him.”
Still, when it comes to the actual game, there is a lot that this can do.
Much to the chagrin of Cuban, Pritchard recounted a late-game play between his Blazers and Cuban’s Mavs in an earlier match up this year that showed how these things can affect the games on a day-to-day basis. With Portland needing a big hoop with seconds left, Juwan Howard hit a 15-footer that sealed the win. Knowing Howard’s shooting percentages and tendencies from different locations on the floor, Cuban couldn’t believe that Juwan hit that shot. That was shot he never makes, and it was a shot Cuban would love to see Howard take all game long.
Pritchard told Cuban that the look on his face after it went in was priceless. “That’s the only 15-footer he’s hit this year,” said Cuban.
“He’s hit two,” said Pritchard.
And whether or not that number is an exact figure that Pritchard can pull off the top of his head or just a quant-centric joke, I think it’s safe to say that Dorkapalooza isn’t just for dorks anymore.
(Giant hat tip to Kevin Arnovitz for some additional reporting.)
