SODA is allowing PC members (but not the PC chair) to submit papers this year. To preserve the integrity of the review process, we will handle PC member submissions as follows.
1. PC members are required to declare a conflict for papers that overlap in content with their own submissions (in addition to other CoI situations). These will be treated as hard conflicts. If necessary, in particular if we don't have enough confidence in our evaluation of a paper, PC members will be asked to comment on papers they have a hard conflict with. However, they will not have a say in the final outcome for such papers.
2. PC submissions will receive 4 reviews instead of just 3. This is so that we have more confidence on our evaluation and ultimate decision.
3. We will make early accept/reject decisions on PC members submissions, that is, before we start considering "borderline" papers and worrying about the total number of papers accepted. This is because the later phases of discussion are when subjectivity and bias tend to creep in the most.
4. In order to be accepted, PC member submissions must receive no ratings below "weak accept" and must receive at least two out of four ratings of "accept" or above.
5. PC member submissions will not be eligible for the best paper award.
My understanding is that this was done to solve the problem of not being able to get people to agree to be on the PC - this year's PC has substantially more members than prior years.
And yet....
Given all the discussion about conflicts of interest, implicit bias, and double blind review, this appears to be a bizarrely retrograde move, and in fact one that sends a very loud message that issues of implicit bias aren't really viewed as a problem. As one of my colleagues put it sarcastically when I described the new plan:
"why don't they just cut out the reviews and accept all PC submissions to start with?"and as another colleague pointed out:
"It's mostly ridiculous that they seem to be tying themselves in knots trying to figure out how to resolve COIs when there's a really easy solution that they're willfully ignoring..."
Some of the arguments I've been hearing in support of this policy frankly make no sense to me.
First of all, the idea that a more heightened scrutiny of PC papers can alleviate the bias associated with reviewing papers of your colleagues goes against basically all of what we know about implicit bias in reviewing. The most basic tenet of human judgement is that we are very bad at filtering our own biases and this only makes it worse. The one thing that theory conferences (compared to other venues) had going for them regarding issues of bias was that PC members couldn't submit papers, but now....
Another claim I've heard is that the scale of SODA makes double blind review difficult. It's hard to hear this claim without bursting out into hysterical laughter (and from the reaction of the people I mentioned this to, I'm not the only one). Conferences that manage with double blind review (and PC submissions btw) are at least an order of magnitude bigger (think of all the ML conferences). Most conference software (including easy chair) is capable of managing the conflicts of interest without too much trouble. Given that SODA (and theory conferences in general) are less familiar with this process, I’ve recommended in the past that there be a “workflow chair” whose job it is to manage the unfamiliarity associated with dealing the software. Workflow chairs are common at bigger conferences that typically deal with 1000s of reviewers and conflicts.
Further, as a colleague points out, what one should really be doing is "aligning nomenclature and systems with other fields: call current PC as SPC or Area Chairs, or your favorite nomenclature, and add other folks as reviewers. This way you (i) get a list of all conflicts entered into the system, and (ii) recognize the work that the reviewers are doing more officially as labeling the PC members. "
Changes in format (and culture) take time, and I'm still hopeful that the SODA organizing team will take a lesson from ESA 2019 (and their own resolution to look at DB review more carefully that was passed a year or so ago) and consider exploring DB review. But this year's model is certainly not going to help.
Update: Steve Blackburn outlines how PLDI handles PC submissions (in brief, double blind + external review committee)
Update: Michael Ekstrand takes on the question that Thomas Steinke asks in the comments below: "How is double blind review different from fairness-through-blindness?".
I completely agree.
ReplyDeleteAnother factor that is often ignored, is how much this would affect the external reviewers' decision on PC members' papers if they also have a submission. These issues are very hard to measure in general, but these "ad hoc" reviewing methods make it even harder. If we want the PC members to submit, then we should move towards a double blind review process. This has been tested and shown to be effective in both theory conferences and elsewhere.
"... and in fact one that sends a very loud message that issues of implicit bias aren't really viewed as a problem."
ReplyDeleteHi Suresh,
I am having trouble "hearing" this message. What is the implicit bias that is being ignored here? Positive bias towards PC members?
The dominant implicit bias in academic reviewing tends to be institutional (there was a paper about this recently). There's also of course other forms of bias, specifically gender bias. But In particular what I'm saying is that the idea that a PC can "police their own bias" flies in the face of what we know about *implicit* bias in general.
DeleteI understand the motivation to go double blind but some of the comments are unnecessarily aggressive and suggest that people are hopelessly biased which is also far from the truth.
ReplyDeleteI don't view the issue as people being hopelessly biased. I view the issue as a lack of recognition of how implicit bias appears. We are all biased, which is why it's important to have checks and balances in place.
DeleteI'm disappointed with this blog post. At best, it seems like a missed opportunity to constructively discuss the actual proposals for handling PC submissions.
ReplyDeleteI completely disagree with the assertion that double-blinding is "a really easy solution" to conflicts of interest. It's particularly ridiculous given that you are active in the FAT* and FATML community, which (to the best of my knowledge) fundamentally rejects the idea that bias can simply be removed by blindness to race/gender/etc.
I also don't think the CS theory community should automatically be "aligning nomenclature and systems with other fields". There are some things that I think theory conferences do better than other fields, such as quality of reviewing.
Thomas, many of us have been constructively discussing this for more than a few years now. And as for double blind review, you make an interesting analogy to work in the FAT* world. However, there's ample evidence that the specific kinds of bias issues associated with paper review (most commonly institutional bias and gender bias) can be mitigated by a staged double blind review process. Why this works differently compared to "fairness through bliindness" in automated decision making is something i have to ponder.
DeleteAs for quality of reviewing, I'd be skeptical of assertions that theory conference reviewing is "better" - at least that was not my experience in the past, and in my current roles supervising and reviewing for other venues I see a similar mix of quality reviews and weaker reviewing that isn't significantly different.
Since my comment has gotten a reaction, I'll elaborate:
DeleteSODA is trying to implement some form of "fairness through awareness" for handling PC submissions. It's far from perfect, but I was hoping that there would be a constructive discussion of its merits and shortcomings. Disappointingly, that has not happened.
This blog post seems to dismiss the approach out of hand on the grounds that obviously the solution is fairness through blindness. Well, it's not obvious to me. Double-blinding is also far from perfect.
Reviewer bias is a complex and difficult problem. There is no easy solution. And I don't think it is helpful to pretend that there is one. Double-blinding mitigates some problems -- I don't doubt that -- but it also sweeps others under the rug. In particular, I think it does a poor job of addressing PC submissions.
Based on my experience with reviewing for ML conferences, the double-blindness there is an empty gesture. Often it is very clear from the discussions among the reviewers that everyone is well aware of the identity of the authors. This is basically unavoidable given the widespread use of arXiv.
ReplyDeleteIf we ignore the pretense of double-blindness, ML conferences actually give a very good example of a nicely working system in which PC submissions are the norm. As long as there is no physical meeting like in STOC or FOCS, I do not see a good reason why PC members will favor submissions of other PC members over submissions of other colleagues who they know equally well and did not happen to be on the PC.
And that's really the point I should have been making in the comments. Setting aside the issue of double blind review (which I recognize is still a controversial topic in the theory community) there are much better ways of handling PC submissions that we should have considered adopting.
DeletePeople use "widespread use of arXiv" as an excuse why we can't have double blind reviewing. On a recent major theory PC, most of the submissions I got assigned were not yet on the arXiv. Someone can do a statistic of this for the next SODA, but it is a poor excuse to dismiss DB.
DeleteThanks Suresh for bringing up this issue in a public discussion.
ReplyDeleteDid you know that STOC 2019 already allowed PC submissions ?
I only found out about this during the SODA 2019 business meeting.
It is really good to know that ESA 2019 is trying DB this year.
I didn't know that. And I'm assuming it was also not blinded, so that the PC had to deal with CoI in some other way.
DeleteThere are three papers on submodular maximization achieving the same result in STOC 2019 accepted list. One of them was written by two PC members, Yaron Singer and Aviad Rubinstein. It would be enlightening to know how CoI on these submissions were handled.
DeleteDear Suresh, thank you for your comments on this topic. As others have said, reviewers and PC members already have to deal with implicit bias when it comes to authors that are well known or they are friends with. With the steps we plan to take, I believe the extent of this bias will be no worse with PC submissions than before. Please note that all discussions for SODA take place online -- there is no physical meeting. PC members will not be able to obtain any information whatsoever about their own submissions, even indirectly through discussions/decisions of other submissions. So, in my view, if PC submissions are handled properly, this change is orthogonal to the issue of double blind.
ReplyDeleteYou said in a previous comment that "Setting aside the issue of double blind review (...) there are much better ways of handling PC submissions that we should have considered adopting." Can you please elaborate on this?
I think what I was referring to are the processes currently in place at other venues. But they make use of 1) double blind review 2) pre-registration of reviewers (so no ad hoc reviews except in extraordinary circumstances 3) extensive CoI declarations at author submission time.
DeleteI'm not sure how 2) and 3) can be implemented without 1), but software allows for all of these things to happen fairly smoothly.
More specifically, as a reviewer I pre-register yourself and declare conflicts based on domain (so if you and I wrote a paper I'd add wisc.edu to my domain exclusion list). As an author I go through the entire reviewer list (hence the need for preregistration) and mark off different kinds of conflicts - this has to happen before my paper can be reviewed and usually is required before submission. Then the system can automatically manage conflicts. At this point it no longer matters whether a reviewer submits a paper or not.
DeleteBut if "serving on the same PC" is considered a conflict then as an author you need to mark off all PC members as conflicts, and then there's no one left to review the paper, no?
DeleteServing on the same PC is not considered a conflict because of the double blind component .
DeletePC submissions and double-blind submissions are two different issues.
ReplyDeleteRe PC submissions: It's a problematic issue in conferences regardless of whether they have a double-blind submission process or not. Ideally I'd rather have no PC submissions - it just doesn't look good. Imagine a conference where half of the papers came from the PC - people would not have a lot of faith in the process, regardless whether the reviewing is "double blind" or not. (If it was DB, people would justifiably be suspicious that the PC didn't really know who the authors were.)
Note that having many PC-authored papers is quite likely even if you have no bias since by definition the PC is composed of highly active members that publish a lot in the given conference. However, disallowing PC-authored papers creates its own problems, especially if you have a multi-author paper where several authors are on consecutive PC's.
Every conference grapples with this in a different way and double-blind doesn't solve it. For example in Eurocrypt (a DB-reviewed conference), the rules are "Program committee members are allowed to submit one paper, and a second one if both submissions are co-authored. In borderline cases, submissions by PC members will be held to a higher standard. Program chairs are not allowed to submit." SODA's rules are also a reasonable solution (perhaps even higher bar than EUROCRYPT, though I do like the idea of a quota to ensure that the conference doesn't end up with a program full of PC-authored papers).
I do think that when there is no physical meeting, the issue of bias towards PC-authored papers is not as strong. (One of the drawbacks of electronic decisions process is that you often don't really have a PC-wide discussion and might not even know the names of the other PC members you are co-serving with; this can be bad for quality of the decision but would reduce the bias towards a PC member).
Also, I disagree that the literature on implicit bias says that people cannot correct themselves. If it was the case then there would be no point in implicit bias training. The whole point of these SODA rules is to make the bias towards PC-authord papers explicit instead of implicit and then try to correct for it. In this case it is easier because it is OK to "over-correct" and be unfair towards PC-authored papers. This does not solve the issue of bias for and against other populations.
Double-blind reviewing is a different issue and deserves more discussion. (I wrote my thoughts on this once on https://windowsontheory.org/2018/01/11/on-double-blind-reviews-in-theory-conferences/ ) I hope we'll eventually move to a model where all papers are on the arxiv/eccc/eprint before submission. (We're already I believe at about half, but it varies between different sub-communities of theory.) This is great for science, but it means that we need to find a different way to deal with biases aside from double-blind reviewing.
Boaz Barak
Boaz, I agree with your message. I would like to emphasize two things you said:
DeleteI believe that explicit measures to consciously correct for potential bias towards PC submissions is better than double-blinding and forgetting. Quotas are an interesting idea.
I also hope we'll eventually move to a model where all papers are online before or shortly after submission. For STOC 2018, 72% of accepted papers were online when the list came out. I would like to see conferences actively encourage it so that we can approach 100%. Alas, I think double-blinding fundamentally hurts this cause, despite assurances to the contrary.
One point to keep in mind is that best practices around DB involving a gradual deanonymization .ie it's part of a larger deliberate and conscious process that is not too dissimilar from what you and Boaz are looking for .
DeletePART I
ReplyDeleteThe drive for double-blind reviews is motivated, so I think, on the following premises:
1. A primary concern guiding PCs should be fairness to the authors.
2. The interests of authors are encoded in the one-bit information of accept/reject.
3. The identity of the authors is objectively insignificant to the review process.
4. Double-blind reviews improve fairness towards authors.
I disagree with each and every one of these premises:
1. Fairness to the authors should not be a concern of PCs. I don’t mean to say that PCs should be unfair to authors. What I mean is that PCs should not be driven by it, but rather it should be a byproduct of the PCs main driving force, which should be fairness towards the conference potential audience. PCs should be concerned with selecting an attractive program that will draw attendance and will reward attendance. Focusing on fairness towards the authors is the wrong attitude.
2. The interests of authors are a bit more complex than this one-bit information. Authors have an interest that the conference where they published in the past and/or will publish in the future will maintain its quality and prestige. Some authors have an interest that some reviewers be impressed with their work (e.g., for reference letters). The most intimate relationship a researcher has with a paper authored by others is often reviewing that paper, and that’s often when the paper and its authors get registered in the reviewer’s awareness. Some people give poor talks but write beautifully. Is it fair to conceal the identity of the authors of a well-written paper that might otherwise be overlooked by the reviewer? Our field is quite accommodating to bright young researchers primarily because of the conference system and the exposure it grants.
Yuval,
DeleteYour point 1 echoes a common misconception about fairness. It is not a distraction from the main PC goal, and it is not charity toward the authors at the expense of the audience and the quality of the program. On the contrary, its goal is to ensure that the program is selected based on parameters relevant to its quality, and not irrelevant ones like authors' gender, nationality or institutional affiliation. It may seem intuitive that simply using all available data to make "objective" decisions would lead to fairness as a byproduct, but there is vast and undeniable evidence of the opposite, that it leads to substantial implicit biases. Eliminating those biases should be a concern of PCs precisely because it aligns with their primary goal of selecting the best program.
Actually, Yuval, I tend to disagree even with this part:
Delete"PCs should be concerned with selecting an attractive program that will draw attendance and will reward attendance. Focusing on fairness towards the authors is the wrong attitude."
It is not the concern of PCs to select an "attractive" program that will draw attendance nor will reward attendance.
The sole concern of the PC is to select the most scientifically important papers within the specified scope of the conference. That is it.
"Attractiveness" has no clear meaning here. For instance, one can be attracted to a conference by the hope to meet his/her friends, another by meeting senior academics who he/she would like to make contact with. By your logic, one should always prefer to accept papers by powerful people since this would immediately increase the attractiveness of the conference.
Even if what you mean is simply the "attractiveness of the result to the audience, irrespective of the authors", then it does not make sense. By this logic, if you accept many papers in subject X, for example, then the next paper in subject X has an advantage over a paper in a different subject, just because it will be much more attractive to the expected audience (who are already interested in subject X).
Moreover, the whole premise of choosing papers towards "having a good conference/program" does not hold in CS. Since in CS conferences are one of the main venues of formal publication, the acceptance of papers is mostly a stamp of quality, a line in the CV, etc. Conferences have already lost its central place as a venue to meet and present new research. It is a place where work is being "accepted" and "archived" for most part.
Anonymous: All I meant was that PCs should focus on the quality of the program and not on the interests of the authors.
DeleteCS Prof: If indeed conferences merely give papers a stamp of quality and a resting place, what's the justification for spending so much money, mostly taxpayers money, on attending? If that's the case, let's replace conferences by an online repository with a rapidly rotating "PC" and continuous submissions, and we've done away with the issue of PC submissions.
"... let's replace conferences by an online repository with a rapidly rotating "PC" and continuous submissions, and we've done away with the issue of PC submissions."
DeleteI completely agree with Yuval here. This is what we should really be discussing.
"what's the justification for spending so much money, mostly taxpayers money, on attending?"
DeleteYuval, I tend to agree. I don't see a strong justification for this. But this is of course a different discussion.
The single responsibility principle states that every module or function should have only one reason to change. I think that also applies to CS conferences. The problem of CS conferences is that they have two roles: (1) giving quality stamps (2) being a place to meet peers. Before we bringing any changes to the conferences, shouldn’t we separate these roles at first? In fact, replacing conferences by an online repository with a rapidly rotating “PC” is a good idea. Why not let this online repository play the role of giving quality stamps and let conferences serve solely as a venue to meet peers and present new research?
Delete"(2) being a place to meet peers. "
DeleteActually this is not the role of conferences anymore in my experience. Dedicated workshops, research visits, special semesters, Dagstuhl, Simons, etc. These are the places where people collaborate. Not some four days of intense talks by hundreds of participants with different research interests.
PART II
ReplyDelete3. There are cases where the identity of the authors is important to the review process. Two obvious examples are: an author with a history of mistakes (the reviewer might want to pay extra attention to verifying the proofs), a paper proposing a new problem or model (such papers fair poorly, for good reasons that judgment of potential significance is difficult, and the past record of the author as a modeler is a good indication). The identity of the authors is often also important in the bidding phase (abstracts don’t always provide good indication), and it helps guarantee a good match between papers and reviewers.
4. Double-blind reviewing is largely a hoax. A quick web search often reveals the identify of the authors (arXiv post, title on personal web page, talk announcement, ...). The topic, style of writing, references, often yield an easy guess. The worst conflicts of interest are the hidden ones, such as poor personal relations between author and reviewer, or some vested interest of the reviewer in the paper. PC members are often aware of at least some of these issues regarding their fellow members or external reviewers. When the names are known to the PC, this is common knowledge, so people are careful with bias, and they keep an eye over suspicious situations. When the names are concealed, they might be known or discovered by everyone, but it is not common knowledge that they’re known. So people won’t be as careful with bias and won’t watch over others as carefully. If the identity of the authors is meaningfully beneficial to some, these authors will eventually make sure it’s known, with or without double-blind reviewing, but it would still not be common knowledge that it’s known.
Disclaimer: I would refuse to serve on a double-blind PC, and as a general rule I refuse to do double-blind external reviewing (unless I’m really interested in reading the paper, or I’m asked by someone I can’t say no to). If I do review double-blindly, I search the web for the authors. This frequently succeeds.
I’m currently serving on the SODA 2020 PC, where for the first time PC submissions are allowed. I don’t have a strong opinion regarding PC submissions at SODA (and wasn’t involved in the decision to do it), but I definitely don’t think that double-blind reviewing has any impact on the negatives. I think the negatives diminish with size of the conference (#submissions and #papers accepted). So the negatives of PC submissions in SODA (600 submissions, 180 accepted) are diminished compared to a similar policy being adopted, for instance, at FOCS (300 submissions, 90 accepted). The numbers are rounded estimates.
“The worst conflicts of interest are the hidden ones, such as poor personal relations between author and reviewer”
DeleteIn this case, we need extensive CoI declarations at author submission time.
Authors abuse CoI declarations. I've encountered extreme cases bordering on fraud (and perhaps on the wrong side of the border). Also, if the CoI is hidden, it might be hidden from the authors.
DeleteAt FOCS 2019, we'll be implementing the recommendations of the SafeToC ad-hoc committee, and other conferences are likely to do the same.
Unfortunately, no formal mechanism will ever solve the problem. It's the responsibility of all PC members to watch for signs of bias, and concealing the names of the authors does not help them do it.
Since no one mentioned it yet: the proposal for allowing PC submissions in SODA has been discussed during the business meeting at SODA 19, voted upon, and passed with a substantial support. This is not to say that the discussion should not continue - it should. Although most CS conferences do allow PC submissions, theory conferences are structured differently enough to make any adaptions non-trivial, and it might take a few iterations to converge on a solution.
ReplyDeleteRegarding double-blind submissions: from my point of view, SODA should adopt DB system if the majority of the conference community is supportive and willing to put up with (minor, but non-negligible) inconveniences resulting from the change. The discussion at SODA'18 business meeting indicates that this might indeed be the case, but it is important to discuss specific implementations and their consequences. The experience of ESA'19, which is experimenting with DB submissions this year, should be helpful in this context.
Piotr Indyk
The SODA rules make me more comfortable than making the conference double blind and then declaring that PC submissions are allowed and no special care needs to be taken with them. I’ve recently reviewed for a few ML conferences and I was > 90% sure I could identify some of the authors for a third of the submissions I got. This is without doing anything improper like googling around for the paper title: I just got assigned papers in my narrow subarea, and usually that’s narrow enough to know who does what. Now this may be ok for “PC members” of ML conferences, who are just glorified reviewers and there are so many of them you don’t even know who they all are. But for theory PCs, I would have little confidence in a solution which introduces DB and then handles a PC submission like any other.
ReplyDeleteSasho, what do you think of the SIGPLAN guidelines that I linked to? They are interesting in that PLDI was historically like theory venues in that they disallowed PC submissions. Now that they do allow them they have a much more detailed process to handle PC submissions. Is that something along the lines of your sense that "more is needed than just DB review"?
DeleteAfter a brief look, it sounds reasonable.
DeleteOne impact that has not been discussed at all is the effect on students. At this point, I'm fine if I skip submitting to a conference or two. But this is not OK for students, for whom, in the short-term, the timing and venue of their papers carries far more weight. I (and other young researchers I talked to) remember being frustrated by not being able to submit when our advisors were on a committee. I know I wouldn't want my service on a committee to be the difference between my student getting a job and not. I've heard that there have been advisors who selflessly removed themselves from papers to not hold back their students from submitting.
ReplyDeleteIn short, I feel like students are the ones who suffer the most from not allowing PC submissions. I would like a solution which avoids hurting students in this way, potentially even if it comes at some other cost. I refrain from opining whether the SODA 2020 system is best, or whether double blind is required (as Boaz said, I believe PC submissions and double blind are two separate issues), or something else entirely.
I am not entirely convinced this is a major problem. The chance that for a given student his/her advisor is on a given PC committee is quite small, when considering the pool of all potential students submitting to the conference. I presume that it becomes non negligible when you count only those students/advisors that have accepted papers or are PC members, but that goes to show the possible (implicit) bias in the process of acceptance.
DeleteTo CS Prof: obviously if the student and the supervisor are doing well, scientifically speaking, that increases the likelihood that the supervisor will get asked to be on the next FOCS/STOC/SODA PC. Then the supervisor has to choose between serving on the PC, which can be important for the career of young untenured faculty, or publishing a paper, which is certainly important for the career of a PhD student.
DeleteThis does not seem to contradict my claim. It is still the case that the chances are very small for a potential student submitting to a given conference to have his/her advisor on the PC.
DeleteThere is no need to restrict the sampled group of students to only those with advisors that tend to serve on all the usual conferences. As I pointed out, this may even lead to more bias in acceptance: we are changing the rules of the game in favor of those students who are already enjoying some potential implicit of explicit positive bias.
I agree with the point CS Prof is making. Notice that ESA has a guideline that someone can only be on the PC once in 5 years. This would make a PhD student subject to this issue likely only once at most. If a PhD student is frequently blocked from submitting to SODA because their advisor is on the PC, then they have the advantage of a well-connected advisor who is being invited to PCs all the time.
DeleteOne of the reasons given at the SODA 2019 business meeting for allowing PC submissions was that it is difficult to get people to accept PC invitations and this widens the pool. But does our community really do a good job of inviting all eligible people to serve on PCs ? Or do we just ask the same people all the time ? It would be good to have some statistics on this compared to other communities.
(For example, it was also mentioned at the SODA 2019 business meeting that for STOC 2019, six people who wanted to submit papers accepted the PC invitation after the rules on PC submission were changed. Is it really the case that the PC chair could not find six other people ?)
CS Prof: It is not uncommon for a paper to have half a dozen authors. A total ban on PC submissions could easily see a paper ruled out from two or three relevant conferences, forcing authors to choose between delaying publication by a year or picking a less relevant conference.
DeleteThomas, again, the fact that there are some papers with many authors who are potentially also PC members does not contradict my claim. The claim is quite simple: the chances that this happens persistently for a given (random) student submitting to a conference are slim.
DeleteThey are slightly less slim when you sample from the pool of those students who are already enjoying a potential implicit or explicit bias by having their advisor well connected (which will tend to correlate well with serving on PC committees). But this is not an argument in favor of changing the rules of the game. On the contrary.
Sure, if I sample a random PhD student in TCS, the probability I land on one whose advisor is on the FOCS PC is small. This is just a weird way to restate the fact that there are many more TCS PhD students than there are FOCS PC members, and that most people don't have that many PhD students. But a large fraction of these students also would not submit to FOCS, so we are not saying very much with this statement, are we? To get a meaningful statement, you need to condition on students who have results that are publishable in FOCS, and then the chances would look, I conjecture, differently. To CS Prof this is simply evidence of bias, because they apparently live in a world where the quality of a student's work is independent of their advisor being invited to multiple PCs. But even in an ideally unbiased world, you would like the PC to consist of active people doing cutting edge work, and they would do that work with their students.
DeleteActually, I'm focusing only on those students who submit to FOCS/STOC. The odds that a random student out of those students submitting to FOCS/STOC, has his/her advisor on the PC is very small.
DeleteFor the rest of your comment, I don't see how it relates to my comments.