Sunday, August 28, 2011

A way forward on reformatting conferences

In some circles, it's a badge of honor to attend as few talks at a conference as possible. Some of the usual comments are:
  • "I go to meet people, not attend talks"
  • "All the interesting conversations happen in the hallways"
  • "Talks are too difficult to follow: I'd rather read the paper or just ask the author"
  • "I read this paper 6 months ago when it appeared on the arxiv: it's old news now, and there are improvements"
You have to wonder why anyone gives talks at a conference any more ! And Moshe Vardi asks this question in a CACM note. Riffing off of "a conference is a journal held in a hotel" (attributed to Lance Fortnow, who attributes it to Ed Lazowska), he talks about the low quality of conference talks, and suggests ways to improve them.

But there's a certain 'band-aid on a bleeding carcass' aspect to this discussion. Indeed, between overloaded reviewers, authors who need the imprimatur of a prestigious conference, and registration fees that skyrocket as meetings get longer, it almost seems like this system is heading for a nervous breakdown.

But there are a number of experiments in play that point the way towards a gentler, kinder conference system (even if we decide not to grow up). In this G+ discussion, Fernando Pereira and Zach Ives describe two models that put together address the main problems with our conference process.

NIPS receives over 1400 submissions, and accepts a small fraction (generally under 20%, and usually much less a little over 20%). All papers are presented as posters (with a few special talks). This does two things:
  1. It removes artificial limits on number of papers accepted based on conference duration. Posters are presented in (semi)-parallel. 
  2. It eliminates the "20-minutes of droning with no questions" style of many conference talks. Posters are a much more interactive way of presenting material, and it's easier to skim papers, talk to the authors, and have a good discussion. The papers are still in the proceedings, so you can always "read the paper" if you want. As an aside, it really helps with communication skills if you have to answer questions on the fly. 
VLDB has now moved to a journal-based submission process. There's a deadline each month for submitting papers for review. The review process is fairly quick: 45 days or so, with enough time for back and forth with the authors. Accepted papers are published in the proceedings, and while I'm not sure exactly how the conference selects talks for presentation, it's possible that all accepted papers are then presented. The main advantages of this process:
  1. There isn't a huge burst of submissions, followed by a draining review process. Reviews are spread out over the year. Moreover, area chairs are used to partition papers further, so any (P)VLDB reviewer only gets  a few papers to review each month. This can only improve the quality of reviews.
  2. The journal-style back-and-forth makes papers better. Authors can make changes as recommended, rather than trying to defend their choices in an often-contentious rebuttal process. 
Between these two systems, we have a better review process for papers, and a better way of delivering the content once reviewed. Why not combine them ? 

Disqus for The Geomblog