* To handle the case of recycled papers that go from conference to conference in the hope of finding reviewers that haven't seen them:I was reading the Dec issue of CACM (all right, I admit it: I do read the CACM, but ONLY because this issue was on the blogosphere!), and in David Patterson's letter was this nugget:
- Create a repository where all submitted papers are registered. When a paper is submitted to a conference, a link is set up, and then when reviews return, they are filed with the paper. If the paper is submitted again, the reviews are there to see.
Perhaps the most novel approach to the whole problem is being taken by the database community under the leadership of SIGMOD. The three large database conferences are going to coordinate their reviewing so that a paper rejected by one conference will be automatically passed along to the next one with the reviews. Should the author decide to revise and resubmit the paper, the original reviewers will read the revision in light of their suggestions. The next program committee would then decide whether or not to accept the revision. Hence, database conferences will take on many of the aspects of journals in their more efficient use of reviewers' efforts in evaluating revisions of a paper.I assume the three bigs are SIGMOD, PODS and VLDB ? It would be interesting to see how this works out. There are pros and cons in carrying prior reviews along with a paper, and a lab experiment such as this might reveal interesting side effects.
Another interesting chart in this letter was a plot of submissions and acceptance ratios for four major conferences (SIGMOD/STOC/ISCA/PODC) over the past 5 years.
What I find most striking about this graph is that on the one hand, SIGMOD submission counts have gone up (pretty much what one would expect), but the acceptance rate has held roughly steady, implying the conference has grown in size over the years. In STOC, on the other hand, submission counts rose, and then remained steady, but acceptance rates have dropped precipitously ! Seems strange that this would happen.
One of my pet peeves has been hearing a slew of theories trotted out to "explain" the increased submission rate in a variety of conferences (though not STOC); my gripe has been that none of this theories have anything more to back them up than plausible sounding words. Interestingly, Patterson's letter has something to say about this as well:
ACM's research conferences are run by its Special Interest Group (SIGs). I've been working with the SIG Governing Board to help form a task force to study this issue, looking at why submissions are increasing and documenting approaches like those discussed here, and to evaluate their effectiveness. They plan to report back in early 2005. If you have any comments or suggestions, please contact task force chair Alexander L. Wolf (alw@cs.colorado.edu).
The idea of autmoatically passing on rejected papers with reviews to the "next" conference is horrible. It almost immediately implies that the paper would be rejected - because acccepting it would mean that this conference is worst than the other conference. The current method, where the PC member might learn that a paper he/she reviewing got rejected from a different conference, and might get the previous reviews, is better since it guarentees the paper would have more fair chance of getting fairly reviewed. Anyway, since the theory community is msall, there is a good chance the two committees would ask the same people (more or less) to referee those papers.
ReplyDeletePosted by Coralloid
The three big ones are SIGMOD/PODS, VLDB, and ICDE.
ReplyDeletePosted by Anonymous