In a comment on my previous post, David Molnar asks:
Could you be more specific about the "many other conferences" that give better feedback than theory?
Well, I don't want to name names :). The reason I don't want to name names is because every time I have, the usual reaction has been "but our papers are much higher quality than theirs", which I think misses the point and clouds the issue.
Let me say though what I consider a "good" set of reviews: the areas I have in mind exemplify this to varying degrees. Note that I don't necessarily think there is any such thing as a good review per se; I am more interested in a good set of reviews. This good set would contain:
* at least one that comments on the merits of the work and how it fits into the broader picture (or not, in case the reviewer doesn't like the paper).
* Some review that tracks detailed issues (even typos, minor errors etc): not all reviews need to do this, but it is useful for the author to know that someone read it thoroughly.
* Something that comments on why the reviewer likes the paper. This is an underrated feature: it's ok to learn from criticism, but without any sense of whether the reviewer actually appreciates my work, it is hard to get a sense of perspective on the criticism.
Overall, a sense that the reviewer understands the paper and assesses it from that perspective. Personally, I can accept a scathing review when the reviewer demonstrates an understanding of my work; it is harder to accept one when I see glaring gaps in the reviewer's comprehension.
My comments about "other areas" comes from submitting papers and looking at the extensive reviews that I get back, and comparing that with reviews from theory conferences, where I can occasionally be lucky to get even one line total from all three reviews. It also comes from the reaction of some of my coauthors (who are not used to theory reviews).
As I said earlier, a lot of this can be explained by the mechanics of the review process: it is a lot easier to review 10 papers thoroughly than 60.
Thanks for your considered response! I will read it at more length. I am starting to act as a reviewer, and while it is helpful to read the various guides available, it is even more helpful to read more up to the minute perspectives. -David Molnar
ReplyDelete