They Said My Methods Were Unsound
Henry Abbot at True Hoop remarks that "The Wages of Wins blog keeps having all these team analyses, but they're of no use if you don't buy into the author's central theory of how wins are produced" and that "10,000 applications of the same formula seems to educate little, especially when that formula has been met with mediocre reviews by some respected names in the field, and quite frankly fails the common sense test." That seems very misguided to me. Performing team-by-team preseason predictions is precisely what the authors of a controversial method of quantitative analysis ought to be doing.
The criticism one would make of the WoW methodology is that even if you grant that they've succeed in decomposing the elements of team success, they're method of breaking down individual players' contributions is highly insensitive to the way actual basketball teams work. Consequently, the fact that in retrospect you can add up a team's "wins produced" score and get a number very close to its actual wins has little value in rebutting that criticism. The real test is what kind of predictive value the WoW analysis has, so offering preseason analysis that we can then test against reality is precisely the correct, stand-up thing to do and if you're interested in the general subject of quantitative analysis of the NBA it makes sense to pay attention.
Comments
I mostly agree with Henry here. The thing is, most of Berry analysis at his blog is sort of repetitive (A had x wins produced, B had z wins produced, etc, etc.) and unimaginative. Really good NBA statistical analysis isn't like that. Plus his method has a couple of serious flaws (basically, failing to recognize that some stats, like rebounding, must be treated differently at the individual and team level) and he has refused to address the criticism. I don't really feel like I'm learning anything reading his blog, frankly.
Hmmmm....
"The real test is what kind of predictive value the WoW analysis has, so offering preseason analysis that we can then test against reality is precisely the correct, stand-up thing to do"
I agree with this paragraph entirely, but Berry's analysis isn't really doing this, is it?
Berry is just chatting about Orlando's prospects while throwing in WoW numbers for illustration. He's not doing anything predictive like adding up the WoW numbers and predicting wins.
I think Abbot is mostly speaking the truth.
------
And tangentially, the problem with WoW-ism is its weird Abrahamic nature. I am the one and only stat. Thou shalt have no other stats before me.
WoW's formula produces a mildly interesting statistic, much like plus/minus numbers. The WoW priests claim a level of meaning for this stat that far exceeds the reality.
Off-topic blog tech related:
The comment submission spam protection system here stops a number of legitimate messages. For example, I wanted to post on this thread a short message with two links in it.
The posting gets flagged for moderation and thrown into some wastebasket queue. Resubmitting the post won't help. I've also noticed that when I've tried to post messages longer than 3 or 4 paragraphs, the same thing has happened.
So obviously the spam filter is overly sensitive to length and number of links. I'm not opposed to having a spam filter, but the settings should be tweaked, if possible.
Well, people from the Apbrmetrics community (Dan Rosenbaum, in particular) approached Berri and tried to get him to collaborate with the rest of the guys in the forum, but Berri refused (apparently, he felt it was besides his academic status). It's a pity, really because he seems like a very competent guy and that attitude hurts his work.
I kind of agree with Petey (gack). The posts appear to be of this formula: Team X has Players Y and Z. Players Y and Z had above/below average WP48 last year. Therefore team X should be good/bad this year.
Here's my question(s): is past performance a guarantee of future results with WP48? What is the year-to-year expected variation in these numbers? Yes, I'm very happy that he says that Jason Kidd has consistently been an above-average WP48 player for his career. But, given his age, what is the likelihood that he will continue to be one this year?
If he really wants to make predictions to test his theory, give me an expected range of WP48 for each player for 2006/7. Then we can look back at the end of the season and see how he did (and the various teams did).
Hey, a little off-topic here, but according to this retire colonel, military operations are already underway in Iran:
http://transcripts.cnn.com/TRANSCRIPTS/0604/14/ywt.01.html
Now that's some scary shit.
"I kind of agree with Petey (gack)."
The beauty of the Petey persona being mildly obnoxious yet infallibly correct is how often it gets responses like this.
The beauty of the Petey persona being mildly obnoxious yet infallibly correct is how often it gets responses like this.
Yeah, I know the feeling.
I love Reality Man saying that "military operations are already underway in Iran" and then posting a link to a CNN program from April. Those are some super secret programs to be kept quiet for 5 months, Reality Man!
Well, Reality Man is technically correct, right? I mean, isn't it true that Lincoln has already been assassinated and the Pyramids have already been built?
Maybe he's referring to Operation Eagle Claw.
"they're method of breaking down individual players' contributions is..."
I have to wonder whether Matthew threw in this grammatical error intentionally (perhaps to add some sweaty gristle to the American Sports ambience), or inadvertently (perhaps due to some subconscious, atavistic, jock-related dementia).
I'm not trying to be a grammar nazi here - Matthew is certainly more erudite and well-spoken than I'll ever be. I just find this odd.
Prediction is important, but if you gave me a method or formula(e), inputting only data from the beginning af a season, and that method successfully outputted the final results for every NBA team in every season for the last forty years, I would take such a formula quite seriously.
bob, one of the things that's important about prediction as opposed to backtested accuracy is that it eliminates formulas that only fit the past data by coincidence.
For any set of past data, there are a bunch of formulas, many of which are ridiculously complex, that will fit the data. Lots of these fit just by coincidence and don't connect to anything of real explanatory value. Each package of new data usually makes some large number of coincidentally accurate formulas fail. That's why correct prediction is so valuable -- it suggests that your theory isn't merely true by coincidence, and that it actually attaches to something of explanatory value.
(Apologies for getting all philosophy-of-science here)
WoW's formula is way too simplistic, even though Malcome Gladwell gave them a positive review. I'm glad others seem to be confirming my opinion of the site. One pet peeve I have with them is how they treat steals. They treat steals as a wholy positive thing. Steals are good when you actually steal the ball, but a big percentage of the time you have to get out of position to make the steal and if you miss, the guy you're garding goes by you. If you were really sophisticated about basketball, you would have a missed steal resulting in a basket category.
Really good defensive teams are disciplined and don't gamble. They make you take tough shots instead. Those teams (or players) may not have high steals averages and and so are rated lower defensively.
Anyway, I think basketball is hard to measure statistically. The flow is too dynamic and at the same time, individual performances can dominate whole games.
Yeah, um, I saw that on another blog and I was too drunk at the time to get through the entire link. My bad. Posting after drinking is never your friend. Let this be a lesson to all you kids out there!
"eliminates formulas that only fit the past data by coincidence"
I understand and accept that, and that was the level of argument I was dealing with. My criterion was "quite seriously" and MY's post seemed to imply that there is no amount of past-data fitting, no matter how large the data set, that would do the work of prediction.
My example was the entire and complete set of NBA games in the last 40 years. A lot of perfect matches to attain.
It was kind of philosophy of science question. But it seems to me that, for instance in economics, past-data matching is at least as important as predictive ability.
It's an interesting idea, and it has the advantage of eliminating the problem of injuries, too. The main problem would be that pre-1978 data is incomplete (no turnovers were recorded, for example), and even modern boxscore data is flawed (too little defensive information, basically). Also, rule changes like the three pointer make the job much harder.
Ahhh....one of my favorite topic:
Luck disguised as expertise.
I see basketball fans are no different than horse racing fans, economists and pundits.
When someone comes up with one of these "scientific" formulas that actually works and uses it to extract a large chunk of the GDP betting in Vegas...I'm sure we'll all hear about it.
I think anyone talking about WoW needs to consider the apbrmetrics thread on it. Carlos mentioned Dan Rosenbaum's attempt to work with the WoW crew, Dan talks about it here.
http://sonicscentral.com/apbrmetrics/viewtopic.php?t=877
"When someone comes up with one of these "scientific" formulas that actually works and uses it to extract a large chunk of the GDP betting in Vegas...I'm sure we'll all hear about it."
But, of course, you won't hear about it.
When folks are beating the markets, they don't reveal their methods, because once they do, they will stop beating the markets.
Folks write their books about how to beat the markets after their system stops working, (or if their system never worked in the first place). If the system still worked, they'd still be beating the markets, rather than writing books.
One thought I keep having about Wages of Wins: someone above said that basketball is tough to describe with statistics, which couldn't be more true. But it's increasingly clear to me that the standard stats in your newspaper are essentialy useless, and we're in the process of getting a lot better ones to replace them. Before long, NBA teams will be using reliable statistics that didn't exist a decade ago to draft better players, make better substitutions, put better combinations of players on the floor and the like. Right now, however, those stats are still marinating. They're embryonic, and the people at the APBRmetrics forum (linked above) and elsewhere are working on them. What worries me is that the well-publicized Wages of Wins could become synonymous with these new-breed of statistics, when in fact I don't think it's really all that representative of the most important basketball statistics work going on these days. (Obviously, time will tell if I'm right or wrong about that.) I would hate for anyone, especially closed-minded basketball executives (and there are plenty of them) to see Wages of Wins fail in some way or another, and conclude that the whole "Moneyball" school of basketball statistics is a crock.
"I would hate for anyone, especially closed-minded basketball executives (and there are plenty of them) to see Wages of Wins fail in some way or another, and conclude that the whole "Moneyball" school of basketball statistics is a crock."
I wouldn't worry too much about this.
The beauty of a competitive situation like the NBA is that teams are always seeking an edge. If good stats help one team win, other teams will start seeking good stats.
Petey- You'd think that would be the case. But for all its competition, there is precious little innovation in the NBA. It's a little bit of an old boy's club, and people are a little set in their way. Sure, there are exceptions. But, for instance, zone defenses were legal for years before using them effectively became a widespread practice. I can't prove it, but I believe it to be true that there are executives who aren't yet fully convinced European players can help. The new hand-checking rules have changed what you can do on offense, but as of last season only Phoenix was really set up to exploit them. And, I have it on good authority that not all NBA coaching staffs are interested in statistics at all. There are lots of potential competitive advantages that fall by the wayside in the NBA. Just because it works is no guarantee it will gain traction.
@Henry
I agree, the WoW guys act like they are the only ones doing any work in this area. Roland "Mr. 82games.com" Beech's review of the book really dissuaded me from purchasing it. Here are some excerpts
“This is the point we are making about decision making in the NBA. It is not that people in the NBA are lazy or stupid. It is just that the tools at theirdisposal do not allow them to see the value of various actions players take on the court.” (p.215)
Well, coming as it does at the end of the book this statement is revealing – first they realize they may have gone too far in claiming in previous chapters to know so much more about NBA players than the people actually in the GM, Scout, Coach jobs, but secondly they really do believe they have the system that captures everything. This frankly strikes me as incredible and full of hubris. If only the NBA people had the Berri/Schmidt/Brook player ratings at their disposal, well then all decisions going forward would be correct. It doesn’t seem to occur to the authors that many NBA teams now have far more sophisticated “tools” at their disposal than what the authors are themselves using.
(This review is available online if you're willing to jump through a couple hoops. No pun intended.) http://www.bepress.com/jqas/vol2/iss3/Kevin Pelton's (admin of the apbrmetrics site among other things) review isn't as harsh, but is hardly the gushfest that Gladwell's is.
http://www.82games.com/pelton24.htm
The best thing about WoW is it unintentionally led me to discover the apbrmetrics community and that has been an interesting experience.
Crack
Ack. All my formatting work ruined by the accidental pressing of post instead of preview. Everything from excerpts to the parenthetical before the link is quoted from Beech's review.
Crack
And, I have it on good authority that not all NBA coaching staffs are interested in statistics at all.
I have no doubt this is true - it seems to me that coaching staffs are too often made up of ex-players who would eschew using quantitative methods in favor of qualitative methods they think their playing experience gives them. Personally, I'd take as a coach my Little General L-Frank any day over some random ex-player. But that's just me.
However, isn't a quantitative approach more useful for a front office than a coaching staff? I could see that argument.
My hypothetical concern from a few days ago, that teams might mistakenly assume this book represents the whole category of Moneyball statistics? Looks like it's coming true in Toronto.
mirc
mirç
mırc
mırç
mircturk
turkmirc
turkiyemirc
mircturkiye
mirch
mırch
mirc indir
mirc yükle
mirc yukle
mırc indir
mırc yukle
mirc turkiye
turkiye mirc
turk mirc
mirc turk
türkçe mirc
türkce mirc
türkçe mırc
turkce mirc
turkçe mirc
turkçe mırc
turkce mirc
mirc chat
mirc sohbet
mirc ara
muhabbet
chat
blog
muhabbet
forum
forum
turkiyeforum
türk forum
mirc
mircturk
turkmirc
mirc indir
sohbet
chat
bedava sohbet
turksohbet
turkiye sohbet
sohbet odaları
bedava chat
chat odaları
türk chat
türkiye chat
turkada
turkmirc
mirc inndir
karar
slm
sex shop
Thank you very much for this information.
sohbet
sesli sohbet
chat
muhabbet
mirc
evden eve nakliyat
mirc
mirc
sohbet
kelebek
chat
oryantal kostüm
Thanks Best Regards
mirc
mırc
mırç
mircturk
mirctürk
turkmirc
mirc indir
mırc indir
mirç indir
mirc yükle
mırc yükle
mirc yukle
mırc yukle
mirch
mırch
mirc turk
turk mirc
mırc turk
mircada
mircturkiye
muhabbet
mirc sohbet
mırc sohbet
mirc chat
mırc chat
mırc ındır
mirc ındır
türkçe mirc
turkce mirc
turkçe mırc
turkce mırc
oper mirc
sohbet
forum
chat
online sinema
catlak
türk forum
turk forum
kafkas
kafkasya
ders
siyaset
izmir
karşiyaka
karsiyaka
forum
karsıyaka
sohbet
penis büyütücüler
harbiarkadas.com
harbiarkadas.net
harbiarkadas.org
itirafet.org
ebedava.net
elektronikmarket.net
ameribress.com
clitoriacream.net
superspenisbuyutucu.com
megabress.com
rednightperformans.com
performansartirici.com
penisplus.tv
penispluspenisbuyutucu.com
penispluspenisbuyutucu.net
cinselmerkez.com
aseks.net
erotikcamasirlar.com
vajinatr.com
bakirevajina.com
cinselkozmetik.com
kozmetikmedikel.com
eturknet.com
tecavuz.net
yutuvideo.com
ponotubesex.com
laraperuk.com
sackanagimerkezi.com
peruksa.com
perukmarket.com
aseks.com
aloveshop.com
erotikgiyim.com
geciktiricispreyler.com
geciktiricihap.com
geciktiriciler.com
azdirici.com
bayanuyarici.com
fntazialemi.com
fantaziservisi.om
cinselmazemeler.com
cinselfantaziurunleri.com
erotikdakikalar.com
erotikmarketiniz.com
seksmarketiniz.com
sekshatlari.com
erotikdergiler.com
erotikderginiz.com
penisbuyutucuviprx.com
penisbuyutucuvigrx.com
penisbuyutuculer.com
vigrxpenisbuyutucu.com
sismebebekler.com
sismebebekshop.com
yemekeviniz.com
sanalmarketiniz.com
elektronikmarket.net
ebedava.net
kontortr.com
elaydin23.com
turkcellkontorcu.com
aveakontoral.com
vodafonekontoral.com
toptankontorcu.com
cinselkozmetik.com
bayanpartnerler.com
erkekpartnerler.com
kizarkadaslar.com
yonjaarkadas.com
siberalem-siberalem.com
sexpartnerler.com
sekspartnerler.com
erotikpartnerler.com
gencyuz.com
erkekarkadaslar.com
bayanarkadaslar.com
yemekeviniz.com
sanalmarketiniz.com
baskahaber.com
medikalkozmetik.net
kozmetikmedikal.com
zayiflamavediyet.net
zayiflamahapii.com
zayiflamabandii.com
kilovertr.com
zayiflamatr.net
diyettr.com
toksinbandi.net
botoxtr.com
botokstr.com
selulittedavii.com
selulitgiderici.net
selulitkremii.com
catlaktedavisii.com
catlakgiderici.net
catlakkremii.com
sex shop
Post A Comment