my deep disgust for the state of affairs within computer graphics research community and my inability to fit well within existing systemHis grievances are many, and appear to be a direct consequence of the hegemonic nature of SIGGRAPH, by far the most prestigious conference in graphics. Specifically, he argues that all the usual problems with conference reviewing (extreme subjectiveness, poor quality of the reviews, clique-formation) are exacerbated by the overwhelming influence SIGGRAPH papers have on one's career (being hired, advancing in one's career, getting tenure, etc).
None of the objections are particularly novel; indeed, I have yet to go to a theory conference where people have NOT complained about the reviewers. However, what takes this beyond just your ordinary embittered-researcher-rant is that many prominent researchers in graphics appear to publicly both agree with him.
Michael says in his letter that senior graphics researchers recommended that he host a forum devoted to discussing this issue, and once he set the forum up, many well known and respected graphics researchers (John Hart, Marc Levoy, and Jonathan Shewchuk among others), commented publicly on the matter. In fact, Marc Levoy (who's the papers chair for SIGGRAPH 2007) went further and organized a town hall meeting at SIGGRAPH 2006 to discuss paper reviewing procedures (I don't know what transpired there).
There are many comments on the forum from anonymous commenters who claim to be published authors at SIGGRAPH. As far as I can tell, not one person disagrees with the primary claims that Michael makes, although Marc does attempt to mount a defense of the paper review process, while still acknowledging the main problems, and outlining strategies that he will employ in 2007 to fix some of them.
Many good suggestions were made. One of the primary ones was to add a new SIGGRAPH-like conference so that one venue didn't have to take all the load (STOC and FOCS were cited favorably here). Prohibiting committee members from submitting was another idea (again, the theory community was cited), although this was frowned upon by Marc Levoy, who complained that he wouldn't be able to find people for a committee (he did aver that he wouldn't be submitting anything).
This is probably the first time I've seen this kind of discussion take place (partly) on the record without dismissing the complaint as sour grapes. The question of course is whether anything will come of it in the long term. It's worth mentioning that even in our conservative (by nature, not by politics) research world, change can happen, and can often happen rapidly. Witness the collective revolt by academicians of all stripes against Elsevier, and closer to home, consider the split in ICRA (one of the main robotics conferences) to form RSS (Robotics: Science and Systems).
Of course, Marc Levoy is the mother of all cliques in the graphics community. Letting him lead the discussion on it is more or less like letting the wolf guard the chickens.
ReplyDeletePoor chickens - not only they are going to be eaten alive, but the wolf is also going to lecture them on his good intentions before that.
Posted by Anonymous
There is nothing wrong with having a conference such as siggraph that emphasizes novelty over incremental results. Incremental results can be submitted elsewhere.
ReplyDeletePosted by Anonymous
There is nothing wrong with having a conference such as siggraph that emphasizes novelty over incremental results.
ReplyDeleteBig names in the community bitterly complain about it and yet "there is nothing wrong" with it? I'd posit that the mere fact that so many people are unsatisfied with it proves your statement wrong.
Posted by anon
There is nothing wrong with having a conference such as siggraph that emphasizes novelty over incremental results.
ReplyDeleteIn the discussion it was pointed out that as a field matures most results (good or bad) are all incremental, and refereeing boils down to judging how big the increment is. Good papers have large increments, mediocre papers have small increments.
Yet, according to the discussion currently in SIGGRAPH an important increment on a fundamental problem has less chance of getting accepted that a solution to a totally new, made-up problem.
Fortunately I don't think this is the case in the main theory conferences but we should be vigilant about it, particularly when some start to equate "incremental" with "mediocre".
Posted by anon
Fortunately I don't think this is the case in the main theory conferences but we should be vigilant about it,
ReplyDeletethis is partially true: we don't have the exact SIGGRAPH-style biases. But we do have our own blind spots (Lance wrote an article about this a while back, on how to get a paper into STOC/FOCS), and vigilance about one's own blind spots is always a good thing.
Posted by Suresh