web analytics
<
Tag Archive - dorkapalooza

It’s Not Whether You Can Find the Perfect Stats, It’s How You Use Them to Play the Game

March 6, 2010 in by with 5 Comments

At the MIT Sloan Sports Analytics Conference, discussion generally surrounds the quant-friendly nuances of how the statistics of the game can be collected, analyzed and adjusted to provide the closest numerical representation of the truth. What are the limitations of Player Efficiency Rating? How valid is adjusted plus-minus? Can any of these advanced numbers ever really show us an objective reality or are they all too biased by the contextual roles that players have within their unique roles on their teams?

These are all fine debates and ones that will continue to rage on throughout every corner of the Dorkapalooza community.

What is most relevant to NBA fans today, however, is how front offices across the league are using these numbers to make decisions in a practical sense. Some of these answers became clearer on Saturday as Mavs owner Mark Cuban, Blazers GM Kevin Pritchard and Celtics Assistant GM Mike Zarren gathered to speak on a panel alongside two of the NBA’s statistical pioneers, Dean Oliver, who works with the Nuggets, and John Hollinger, who we all know from ESPN.

How do team execs really use the numbers?

“Depends on the time of year,” said Pritchard, noting that different stats mean different things depending on whether he is thinking about the trade deadline, the draft or optimizing lineups to match up with an opponent in a seven-game series. “Overall, it’s on the personnel side.”

Trade and free agency decisions are increasingly being made with more statistical information, and Oliver broke down how widespread this is all becoming and stated that he knows of eight teams that have actually integrated advanced analytics into their decision-making (Boston, Cleveland, Dallas, Denver, Houston, Oklahoma City, Orlando and Portland). In all, he says that he saw 14 different teams with personnel on the attendee list for Sloan Conference this year — and knew of two other assistant GMs who were not listed. Kevin Pelton also broke down which statisticians are now working in the league, in the process, revealing that he is now consulting with Indiana.

Cuban has been bullish on gathering data since he first bought the team, and early on began consulting with noted statistician Wayne Winston, who he had as a professor for a statistics course while attending Indiana University and had lost touch with until he got to the NBA. “I hadn’t seen him until three weeks before I bought the team. I saw him on Jeopardy, and thought, ‘Hey, I should give that guy a call.’”

In his early days, he admittedly made some mistakes. At least twice during the day, Cuban mentioned Evan Eschmeyer as a guy who he overvalued — and badly overpaid — based largely on some plus-minus data that, in retrospect, he realizes was based on too small of a sample size to translate to the future.

That was something Cuban learned from, but it didn’t deter his reliance on plus-minus, which he not only used recently to make a perhaps sea-changing trade for the Western Conference, but to help decide to hire Rick Carlisle before the 2008-09 season. He ran the numbers, and found that Carlisle was the NBA coach who had the greatest positive affect on the plus-minus rankings of those players who joined new teams. “It was Rick by a long shot,” said Cuban earlier in the day.

A big challenge to all this, however, is just gathering the data. “[Only] 20% or maybe a quarter of defense shows up in a box score,” said Hollinger. Steals, blocks and personal fouls are there, but what happens on all the other plays is not. Who forced a shooter to miss? Who blew a rotation? The box score will never tell you that.

“The box score is an incomplete story,” said Pritchard. “And more than that, it can be misleading.”

Still, unless you have the time to watch, chart and analyze every play qualitatively, the numbers — many of which can now be instantaneously collected automatically from play-by-play data — provide an invaluable base level of evidence on which to make better decisions. Oliver summed it up perfectly. “Individuals see a game better than the numbers,” he said. “But the numbers see all the games.”

Cuban and Rockets GM Daryl Morey, who run the Sloan Conference in partnership with MIT, have devoted a ton of money and organizational resources to mining all the games to find the data that goes beyond the box score. But they have both expressed their desire to see more of this being done by the league.

“Evaluating players, you have to do a lot more work and that’s what’s frustrating to us,” said Cuban. “You have to have someone charting every play. And there is no more inefficient use of someone’s time.”

Zarren understands the frustration and knows that Boston and the other major teams embracing data mining are probably wasting resources just to come up with the same data that other teams are unveiling. “There has to be a lot of duplication of work going on.”

Some have suggested that if the league — or some third party provider — does the work and makes it available to all the teams, it would take away some of the competitive edge for those on the cutting-edge. Cuban doesn’t seem worried about that and feels like it is what you do with the general data that really matters.  “We all have our own special sauce,” he said. “We’re only talking about the data gathering … All of the teams are going to catch on, we may as well nip it in the bud.”

Hollinger, whose PER metric is all-too-often proselytized as a Holy Grail player ranking despite his insistence that that is not its utility, similarly seems to believe that most advanced numbers are more important within the context of individual front offices than they are when used as some monolithic, numerical judge of every player in the league. He says that he “wouldn’t want to pick the All-Star teams” based on league-wide advanced statistical measures alone, but, within the operating philosophy of each front office, almost all of these numbers — when put into proper context — can be used to help make better decisions.

Getting the numbers from the stat heads to the people coaching the team is the next hurdle.  “As important as the work you do is how you communicate it,” said Zarren. In Boston, it took him and Doc Rivers a while to understand each other and the vocabulary barrier is something that will always be difficult to overcome.

Dean Oliver has had similar experiences working with the Nuggets, but has learned a lot in his time there and the whole organization has continued to improve its ability to talk the same language. “These communication skills are not trivial,” he said. “And improving these skill may be more important than improving how you calculate adjusted plus-minus.”

Once everyone is one the same page, teams can start seeing some real results. Coaches can tailor their systems to the overall organizational philosophy and get the players to do those things they were brought in by the GMs to do. And while a guy like Shane Battier has famously embraced this from the player side, a lot of this stuff can stop with the coaches. It may not be necessary to have the players in on the math.

“It’s really important for the coaches to design schemes around the data,” said Zarren. “But it’s not important for the players to know everything that went into designing it.”

The goal should be to integrate the analysis into the overall coaching philosophy, but for many players — some of which can’t even remember the plays they are supposed to run — advanced stats are not something that can be used to change the way they play.

“We had Gerald Green,” said Cuban, with a glance over towards Zarren. “You had Green. He does stuff [athletically] that makes you say ‘Oh my God!’ … He just doesn’t understand the game of basketball.”

Offering further evidence of the difficulty in putting any of this into the players hands, Cuban talked about the logistical problems presented by the arduous schedule of the NBA season. “We haven’t had a practice since the trade,” he said in reference to the deal that brought Caron Butler and Brendan Haywood to the Mavs.

“We’ve had two since then,” said Zarren.

But while the numbers are not something Cuban will be using immediately to help Haywood play better individual defense as the Mavs make a run at an NBA title, they are a big reason he is now in Dallas. “Defensive numbers absolutely had a large part in the trade for Brendan Haywood,” he said. “And we wouldn’t have done the trade without him.”

Still, when it comes to the actual game, there is a lot that this can do.

Much to the chagrin of Cuban, Pritchard recounted a late-game play between his Blazers and Cuban’s Mavs in an earlier match up this year that showed how these things can affect the games on a day-to-day basis. With Portland needing a big hoop with seconds left, Juwan Howard hit a 15-footer that sealed the win. Knowing Howard’s shooting percentages and tendencies from different locations on the floor, Cuban couldn’t believe that Juwan hit that shot. That was shot he never makes, and it was a shot Cuban would love to see Howard take all game long.

Pritchard told Cuban that the look on his face after it went in was priceless. “That’s the only 15-footer he’s hit this year,” said Cuban.

“He’s hit two,” said Pritchard.

And whether or not that number is an exact figure that Pritchard can pull off the top of his head or just a quant-centric joke, I think it’s safe to say that Dorkapalooza isn’t just for dorks anymore.

(Giant hat tip to Kevin Arnovitz for some additional reporting.)

Be Less Than You Can Be

March 6, 2010 in by with 11 Comments

Brian Skinner’s presentation of his paper, “The price of anarchy in basketball,” at the Sloan Sports Analytics Conference hits in two distinct ways:

  • One hand hand, Skinner’s proposal is so dreadfully counter-intuitive on a conceptual level that it could make a lot of NBA fans turn up their noses.
  • But on the other, that concept is such a deeply-seated part of sport as we know it, that any self-respecting follower of sports would be deemed foolish for turning a blind eye.

His work appropriately brings the contrast between macro and micro into broad daylight, but in a completely different way than many NBA fans are used to. The struggles to find balance on a basketball court typically occur in an effort to best determine the optimal method of achieving a common goal…assuming that there is a common goal. One guy may be playing for a contract, another just for the joy of scoring more than anyone else on the floor, one just because basketball is the only thing he’s ever been good at, and finally, there may actually be some players who want to win games. More realistically, their interests are some combination of those mentioned above, in the form of a veteran athlete drifting through the regular season almost out of habit, and shooting/scoring because that’s what he’s always done.

Somehow, someone has to take all of those intentions and all of the production, potential, and talent that comes along with it, and figure out the best possible way to win. It’s not an easy task, and the natural inclination is to break the incredibly complex, holistic game on the hardwood into its most basic components: Steve Nash is effective bringing the ball up the floor and running the offense, but Amar’e Stoudemire is probably not. Kobe Bryant is an excellent shooter, and D.J. Mbenga is not. Then, we can take all of these things, take into account all of the players on the floor and their relative strengths and weaknesses, and decide which conceivable outcome will give you the best possible chance of scoring points on a given possession. Our logic and reasoning processes tell us that choosing the option with the greatest probability for success is an easy call. Skinner tells us that your logic and reasoning could be completely off-base.

One example that Skinner highlighted is Ray Allen. Allen’s shot usage over the course of his career makes him an easy candidate; in Milwaukee and Seattle he was called upon to be The Man, but in Boston, he’s simply a man. He’s a shooter, a scorer and leader, and one of the three. He offers plenty on the court but in a completely different capacity, and with a markedly lower percentage of his team’s total shots. There is power in variety, and with the offensive options that have been available in Boston (in 2008, namely, though still today on a theoretical basis, if nothing more), Allen, a talented offensive player, actually benefits his team by not shooting. Not because his teammates are better shooters than he is on a per-possession basis necessarily, but because putting so much of the offensive production on one source creates myriad problems. Fatigue. Defensive attention. Heat checks. Skinner invokes Dean Oliver in stating that as usage goes up, a player’s offensive efficiency goes down, and that makes a ton of sense.

But at the same time, that creates a bit of a boggling result: a team’s best play is sometimes to have their best shooter not shoot. That conclusion naturally led Allen’s former teammate, Brent Barry, to pipe up from the audience and announce that he texted Ray the results of the study with a note to not shoot so much. I’m pretty sure Denzel Washington told him the same thing over ten years ago, though, so I wouldn’t expect some kind of drastic change.

It’s also interesting to note that such an idea is pretty much in direct conflict with the love of excess when it comes to sports. We want a player to one day hold up a piece of paper reading “101″ in a locker room, to waltz into Madison Square Garden and drop 50, to average a triple-double for an entire season. America is a nation of decadence and sports are absolutely no exception. But the underlying mentality that drove guys like Michael Jordan or Larry Bird to greatness is, despite each player’s phenomenal success, not the best possible approach.

Think about that. It’s completely possible, supposing you buy Skinner’s basic argument, that Jordan and the Bulls underachieved. If they had achieved the perfect offensive balance — where Jordan may have stopped scoring well before the point which Skinner describes as something akin to the Nash equilibrium, in which a player’s likelihood of scoring is equivalent to that of his teammates scoring — the ’96 Bulls could very well have improved upon their 72-10 record. Or maybe that’s exactly what allowed that particular team (and all of Chicago’s championship teams, really) to excel compared to some of Jordan’s earlier campaigns.

When you think about it that way, it’s the same argument we’ve heard over and over again: Player X should shoot less for the benefit of the team. It’s disguised as “trusting their teammates” and, as Skinner noted, “keeping the defense honest,” but it’s probably as old as winning itself. Where this argument differs from others is the disregard of our micro-obsessed desire to dissect the game’s minutiae. We do possession breakdowns and pick apart the decision-making of every player on the court, but sometimes the view is so narrow that it obscures the bigger picture. Your best player getting a good shot isn’t always the best case scenario for an offense in the long-run…even if it’s LeBron James kicking it to an open Donyell Marshall in the corner.

But Skinner’s study, while deeply theoretical, lacking in obvious applications, and definitely limited and assuming in a lot of respects, represents a pretty interesting intersection that occurs at places like Sloan; new, innovative research confirms something that we already knew, but in such a way that offends even our modern sensibilities. That the Lakers could actually benefit from Kobe Bryant passing to Smush Parker? That the Nuggets could run a more efficient offense by having Chauncey Billups pass up a shot in favor of Joey Graham? It’s not always an easy fact to stomach, but the research shows, at least at a basic level, that those plays are so wrong that they’re right.

You can read Skinner’s paper in its entirety here.