tag:blogger.com,1999:blog-65559472024-03-14T01:32:43.741-06:00The GeomblogRuminations on computational geometry, algorithms, theoretical computer science and lifeSuresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.comBlogger1332125tag:blogger.com,1999:blog-6555947.post-89663149393060623042021-05-17T08:33:00.002-06:002021-05-17T08:33:37.528-06:00Transitions<p> I've been at the U of Utah and Salt Lake City for 14 years (14.5 really). It was my first academic job and the longest time I've spent anywhere (throughout my whole life). So it's a little hard to accept that I'm moving to my next adventure. </p><p>It's a two-part adventure, because why make one move when you can make two. </p><p>Firstly, as of today, I'm going to working with Alondra Nelson at the White House Office of Science and Technology Policy, advising on matters relating to fairness and bias in tech systems. This is a scary and exciting new position, and I hope to help to nudge things along just a bit further in the direction of tech that can help more than it harms, especially for those who've been left behind in our rush to an algorithmically controlled future. </p><p>Secondly, I'm moving to Brown University to join the CS department there as well as their Data Science Initiative. Together with Seny Kamara and others, I'm going to start a new center on Computing for the People, to help think through what it means to do computer science that truly responds to the needs of people, instead of hiding behind a neutrality that merely gives more power to those already in power. </p><p>Lots of changes, and because of the pandemic, all this will happen in slow machine, but it's a whirlwind of emotions (and new clothes - apparently tech conference T-shirts don't work in formal settings - WHO KNEW!!!). </p><p><br /></p>Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com11tag:blogger.com,1999:blog-6555947.post-10343286088799921492020-12-25T11:47:00.002-07:002020-12-25T11:52:04.970-07:00Lars Arge. <p><i> Not a post I'd have wanted to make on Christmas day, but that's how it goes sometimes. </i></p><p>Lars Arge just passed away, on Dec 23. For those of us who've been following his battles with cancer, this might not come as a total shock, but there was always hope, and that's no longer an option. </p><p>It's hard to imagine this in 2020, but there was a time not that long ago (at least in my mind) when "big data" wasn't really a thing. Companies were acquiring lots of data, and "GIGA byte" was a thing, but there was no real appreciation of the computational challenge associated with big data. </p><p><a href="https://hal.inria.fr/inria-00075827">A paper by Aggarwal and Vitter</a> in 1998 made the first step towards changing that, introducing the <a href="https://en.wikipedia.org/wiki/External_memory_algorithm#:~:text=The%20model%20was%20introduced%20by,size%20and%20the%20cache%20size.&text=Both%20the%20internal%20and%20external,into%20blocks%20of%20size%20B.">external memory model</a> as a way to think about computations when you have memory access that are cheap (in RAM) and expensive (on disk). </p><p>It's a diabolically simple model: all main memory access is free, and any disk access costs 1 unit (but you can get a block of data of size B for that one unit of access). It's not meant to be realistic, but like the best computational models, it's meant to isolate the key operations that are expensive so that we can study how algorithm design needs to change. </p><p>Lars was one of the foremost algorithm designers for this new world of external memory. His Ph.D thesis laid out ideas for how to build data structures that are external memory efficient, and his research over the next many decades, in true Tarjan/Hopcroft form, built the fundamental structures and concepts one would need to even think about efficient algorithm design, with many clever ideas around batching queries, processing data in main memory to prepare for queries, and streaming access to disk when appropriate. </p><p>Formal algorithmic models are often misunderstood. They look simplistic, miss many of the details that seem relevant in practice, and appear to encourage theoretical game playing divorced from reality. But a formal model at its best does its work invisibly. It shifts the way we think about a framework. It fosters the design of new paradigms for efficient algorithms, and it allows us to layer optimizations on that move a system from theory to practice without ever having to compromise the underlying design principles.</p><p>Lars was a force of nature in this area. I first remember meeting him in 1998 at AT&T Labs when I was interning and he was visiting there. He had boundless energy for this space, and seemingly wanted to turn everything into an external memory algorithm, whether it was geometry, data structures, or even the most basic algorithms like sorting. His intuition was the best kind of algorithmic intuition: build up the core primitives, and the rest would follow. </p><p>And this is exactly what happened. The field exploded. For a while, "big data algorithms" WERE external memory algorithms. There was no other way to even talk about big data. And that spawned even more models. Streaming algorithms were inspired by external memory and the realization that a one pass stream was an effective way to work with large data. Cache-oblivious algorithms asked about what would happen if we took the same two-part hierarchy with main memory and disk and extended it to the cache. Semi-external memory models asked how we might modify the base model for graph computations. The MapReduce framework from the early 2000s generalized the external memory model to handle newer kinds of streaming/memory-limited architectures, in turn to be followed by Spark and so many other models. </p><p>I'd go as far as to say this: all of the conceptual developments we see today in big data computations at some level can be traced back to work on external memory algorithms, and that was driven by Lars (and his collaborators). </p><p>It wasn't just the papers he wrote. Lars was a leader in shaping the field. Early in the 2000s he moved back from Duke University to Aarhus University, and from there started to build what would become one of the foremost institutes for thinking about big data, first as a BRICS center and then as the appropriately named <a href="https://cs.au.dk/research/centers/madalgo/">MADALGO</a> Institute. </p><p>Many of us who had anything to do with big data visited MADALGO at some point in our careers. I spent one of the best summers of my life being hosted by him during my sabbatical - my children still remember that summer we spent in Aarhus and wish we could go back each year. He instinctively knew that the best way to foster the area was to facilitate a generation of researchers who would bring their own ideas to Aarhus, mix and exchange them, and then go away and share them with the world. </p><p>And he wasn't merely content with that. He wanted to demonstrate the power of his perspective beyond just the realm of academia. He started a company <a href="https://scalgo.com/">SCALGO</a> that applied the principles of external memory algorithms (and so much more) to help with modeling geospatial data. I remember distinctly him telling me the first time he demonstrated SCALGO products in a forum with other companies doing GIS work and how the performance of their system blew the other products out of the water. For someone (at the time) deeply embedded in the theory of computer science, I was astounded and encouraged by this validation of formal thinking. </p><p>Lars was a giant in our field (his email address was always large@..., and this worked more appropriately than one would ever dream of). But he was also a giant both in real life and in his personality. He was the warmest, most fun person to be around. He seemed almost ego-free, and often downplayed his own accomplishments, claiming that his main talent was hanging around with smarter people. He was extremely generous with his time and resources (which is why so many of us were able to visit Aarhus and benefit from being at MADALGO)</p><p>He was the life of any party -- I still remember when he hosted the Symposium on Computational Geometry in Denmark. It felt like we were at a post-battle Viking celebration (and yes he got up on a table and shouted "SKÅL" over and over again while an actual pig was roasting on a spit nearby). I remember him taking me to a Denmark-Sweden soccer game and warning me not to wear anything with blue on it. I remember us going for go-kart racing and his stream of trash talking. </p><p>Lars was the entire package: a great person, a great researcher, a visionary leader, and a canny entrepreneur. I will miss him greatly. </p>Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com4tag:blogger.com,1999:blog-6555947.post-91229504915343395382019-04-11T14:00:00.001-06:002019-04-11T14:00:12.109-06:00New conference announcement<div class="tr_bq">
Martin Farach-Colton asked me to mention this, which is definitely NOT a pox on computer systems. </div>
<blockquote>
ACM-SIAM Algorithmic Principles of Computer Systems (APoCS20) </blockquote>
<blockquote>
<a class="" href="https://www.siam.org/Conferences/CM/Main/apocs20"><span style="color: black;">https://www.siam.org/Conferences/CM/Main/apocs20</span></a>January 8, 2020<br />
Hilton Salt Lake City Center, Salt Lake City, Utah, USA<br />
Colocated with SODA, SOSA, and Alenex </blockquote>
<blockquote>
The First ACM-SIAM APoCS is sponsored by SIAM SIAG/ACDA and ACM SIGACT. </blockquote>
<blockquote>
<b class="">Important Dates:</b><span class="Apple-tab-span" style="white-space: pre;"> </span> </blockquote>
<blockquote>
August 9: Abstract Submission and Paper Registration Deadline<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>August 16: Full Paper Deadline<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>October 4: Decision Announcement </blockquote>
<blockquote>
<b class="">Program Chair: </b>Bruce Maggs, Duke University and Akamai Technologies </blockquote>
<blockquote>
<b class="">Submissions: </b>Contributed papers are sought in all areas of algorithms and architectures that offer insight into the performance and design of computer systems. Topics of interest include, but are not limited to algorithms and data structures for: </blockquote>
<blockquote>
<br />
<ul>
<li>Databases</li>
<li>Compilers</li>
<li>Emerging Architectures</li>
<li>Energy Efficient Computing</li>
<li>High-performance Computing</li>
<li>Management of Massive Data</li>
<li>Networks, including Mobile, Ad-Hoc and Sensor Networks</li>
<li>Operating Systems</li>
<li>Parallel and Distributed Systems</li>
<li>Storage Systems</li>
</ul>
<br />
A submission must report original research that has not previously or is not concurrently being published. Manuscripts must not exceed twelve (12) single-spaced double-column pages, in addition the bibliography and any pages containing only figures. Submission must be self-contained, and any extra details may be submitted in a clearly marked appendix. </blockquote>
<blockquote>
<b class="">Steering Committee:</b> </blockquote>
<blockquote>
<br />
<ul>
<li>Michael Bender</li>
<li>Guy Blelloch</li>
<li>Jennifer Chayes</li>
<li>Martin Farach-Colton (Chair)</li>
<li>Charles Leiserson</li>
<li>Don Porter</li>
<li>Jennifer Rexford</li>
<li>Margo Seltzer
</li>
</ul>
</blockquote>
Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-37212242412524870822019-03-26T08:00:00.000-06:002019-03-28T23:44:20.737-06:00On PC submissions at SODA 2020SODA 2020 (in SLC!!) is experimenting with a new submission guideline: PC members will be allowed to submit papers. I had a conversation about this with Shuchi Chawla (the PC chair) and she was kind enough (thanks Shuchi!) to share the guidelines she's provided to PC members about how this will work.<br />
<br />
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px 'Fira Sans'; -webkit-text-stroke: #000000}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px 'Fira Sans'; -webkit-text-stroke: #000000; min-height: 20.0px}
span.s1 {font-kerning: none}
</style>
<br />
<blockquote class="tr_bq">
<span class="s1">SODA is allowing PC members (but not the PC chair) to submit papers this year. To preserve the integrity of the review process, we will handle PC member submissions as follows.</span> </blockquote>
<blockquote class="tr_bq">
<span class="s1">1. PC members are required to declare a conflict for papers that overlap in content with their own submissions (in addition to other CoI situations). These will be treated as hard conflicts. If necessary, in particular if we don't have enough confidence in our evaluation of a paper, PC members will be asked to comment on papers they have a hard conflict with. However, they will not have a say in the final outcome for such papers. </span> </blockquote>
<blockquote class="tr_bq">
<span class="s1">2. PC submissions will receive 4 reviews instead of just 3. This is so that we have more confidence on our evaluation and ultimate decision.</span> </blockquote>
<blockquote class="tr_bq">
<span class="s1">3. We will make early accept/reject decisions on PC members submissions, that is, before we start considering "borderline" papers and worrying about the total number of papers accepted. This is because the later phases of discussion are when subjectivity and bias tend to creep in the most.</span> </blockquote>
<blockquote class="tr_bq">
<span class="s1">4. In order to be accepted, PC member submissions must receive no ratings below "weak accept" and must receive at least two out of four ratings of "accept" or above.</span> </blockquote>
<blockquote class="tr_bq">
5. PC member submissions will not be eligible for the best paper award.</blockquote>
<br />
My understanding is that this was done to solve the problem of not being able to get people to agree to be on the PC - this year's PC has substantially more members than prior years.<br />
<br />
And yet....<br />
<br />
Given all the discussion about conflicts of interest, implicit bias, and double blind review, this appears to be a bizarrely retrograde move, and in fact one that sends a very loud message that issues of implicit bias aren't really viewed as a problem. As one of my colleagues put it sarcastically when I described the new plan:<br />
<br />
<blockquote class="tr_bq">
"why don't they just cut out the reviews and accept all PC submissions to start with?"</blockquote>
and as another colleague pointed out:<br />
<br />
<blockquote class="tr_bq">
"It's mostly ridiculous that they seem to be tying themselves in knots trying to figure out how to resolve COIs when there's a really easy solution that they're willfully ignoring..."</blockquote>
<br />
Some of the arguments I've been hearing in support of this policy frankly make no sense to me.<br />
<br />
First of all, the idea that a more heightened scrutiny of PC papers can alleviate the bias associated with reviewing papers of your colleagues goes against basically all of what we know about implicit bias in reviewing. The most basic tenet of human judgement is that we are very bad at filtering our own biases and this only makes it worse. The one thing that theory conferences (compared to other venues) had going for them regarding issues of bias was that PC members couldn't submit papers, but now....<br />
<br />
Another claim I've heard is that the scale of SODA makes double blind review difficult. It's hard to hear this claim without bursting out into hysterical laughter (and from the reaction of the people I mentioned this to, I'm not the only one). Conferences that manage with double blind review (and PC submissions btw) are at least an order of magnitude bigger (think of all the ML conferences). Most conference software (including easy chair) is capable of managing the conflicts of interest without too much trouble. Given that SODA (and theory conferences in general) are less familiar with this process, I’ve recommended in the past that there be a “workflow chair” whose job it is to manage the unfamiliarity associated with dealing the software. Workflow chairs are common at bigger conferences that typically deal with 1000s of reviewers and conflicts.<br />
<br />
Further, as a colleague points out, what one should really be doing is "aligning nomenclature and systems with other fields: call current PC as SPC or Area Chairs, or your favorite nomenclature, and add other folks as reviewers. This way you (i) get a list of all conflicts entered into the system, and (ii) recognize the work that the reviewers are doing more officially as labeling the PC members. "<br />
<br />
<br />
Changes in format (and culture) take time, and I'm still hopeful that the SODA organizing team will take a lesson from <a href="https://algo2019.ak.in.tum.de/index.php/menue-esa/esa-call">ESA 2019</a> (and their own resolution to look at DB review more carefully that was passed a year or so ago) and consider exploring DB review. But this year's model is certainly not going to help.<br />
<div>
<br />
<b>Update: </b><a href="https://twitter.com/stevemblackburn/status/1111068299796705280?s=20">Steve Blackburn outlines how PLDI handles PC submissions</a> (in brief, double blind + external review committee)<br />
<br />
<b>Update: </b><a href="https://md.ekstrandom.net/blog/2019/03/anonymous-reviewing">Michael Ekstrand</a> takes on the question that Thomas Steinke asks in the comments below: "How is double blind review different from fairness-through-blindness?".<br />
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px 'Fira Sans'; -webkit-text-stroke: #000000}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px 'Fira Sans'; -webkit-text-stroke: #000000; min-height: 20.0px}
span.s1 {font-kerning: none}
</style></div>
Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com46tag:blogger.com,1999:blog-6555947.post-2678836853981203782019-02-19T09:00:00.000-07:002019-02-19T09:00:08.249-07:00OpenAI, AI threats, and norm-building for responsible (data) scienceAll of twitter is .... atwitter?... over the <a href="https://blog.openai.com/better-language-models/">OpenAI announcement</a> and partial non-release of code/documentation for a language model that purports to generate realistic-sounding text from simple prompts. The system actually addresses many NLP tasks, but the one that's drawing the most attention is the deepfakes-like generation of plausible news copy (<a href="https://blog.openai.com/better-language-models/#sample3">here's one sample</a>).<br />
<br />
Most consternation is over the rapid PR buzz around the announcement, including somewhat breathless headlines (that OpenAI is not responsible for) like<br />
<br />
<blockquote class="tr_bq">
<a href="https://techcrunch.com/2019/02/17/openai-text-generator-dangerous/">OpenAI built a text generator so good, it’s considered too dangerous to release</a></blockquote>
or<br />
<blockquote class="tr_bq">
<a href="https://arstechnica.com/information-technology/2019/02/researchers-scared-by-their-own-work-hold-back-deepfakes-for-text-ai/">Researchers, scared by their own work, hold back “deepfakes for text” AI</a></blockquote>
There are concerns that OpenAI is overhyping solid but incremental work, that they're disingenuously allowing for overhyped coverage in the way they released the information, or worse that they're deliberately controlling hype as a publicity stunt.<br />
<br />
I have nothing useful to add to the discussion above: indeed, see posts by <a href="https://anima-ai.org/2019/02/18/an-open-and-shut-case-on-openai/">Anima Anandkumar,</a> <a href="https://towardsdatascience.com/should-i-open-source-my-model-1c109188b164">Rob Munro</a>, <a href="http://approximatelycorrect.com/2019/02/17/openai-trains-language-model-mass-hysteria-ensues/">Zachary Lipton</a> and <a href="https://medium.com/@lowe.ryan.t/openais-gpt-2-the-model-the-hype-and-the-controversy-1109f4bfd5e8?sk=bc319cebc22fe0459574544828c84c6d">Ryan Lowe</a> for a comprehensive discussion of the issues relating to OpenAI. Jack Clark from OpenAI has been engaging in a lot of twitter discussion on this as well.<br />
<br />
But what I do want to talk about is the larger issues around responsible science that this kerfuffle brings up. Caveat, as Margaret Mitchell puts it in this searing thread.<br />
<blockquote class="twitter-tweet">
<div dir="ltr" lang="en">
It's really hard to watch the GPT-2 conversations unfold like so much else in tech. 1/</div>
— MMitchell (@mmitchell_ai) <a href="https://twitter.com/mmitchell_ai/status/1097626427048964098?ref_src=twsrc%5Etfw">February 18, 2019</a></blockquote>
<br />
To understand the kind of "norm-building" that needs to happen here, let's look at two related domains.<br />
<br />
In computer security, there's a fairly well-established model for finding weaknesses in systems. An exploit is discovered, the vulnerable entity is given a chance to fix it, and then the exploit is revealed , often simultaneously with patches that rectify it. Sometimes the vulnerability isn't easily fixed (see <a href="https://meltdownattack.com/">Meltdown and Spectre</a>). But it's still announced.<br />
<br />
A defining characteristic of security exploits is that they are targeted, specific and usually suggest a direct patch. The harms might be theoretical, but are still considered with as much seriousness as the exploit warrants.<br />
<br />
Let's switch to a different domain: biology. Starting from the sequencing of the human genome through the <a href="https://allofus.nih.gov/">million-person precision medicine project </a>to CRISPR and cloning babies, genetic manipulation has provided both invaluable technology for curing disease as well as grave ethical concerns about misuse of the technology. And professional organizations as well as the NIH have (sometimes slowly) risen to the challenge of articulating norms around the use and misuse of such technology.<br />
<br />
Here, the harms are often more diffuse, and the harms are harder to separate from the benefits. But the harm articulation is often focused on the individual patient, especially given the shadow of abuse that darkens the history of medicine.<br />
<br />
The harms with various forms of AI/ML technology are myriad and diffuse. They can cause structural damage to society - in the concerns over bias, the ways in which automation affects labor, the way in which fake news can erode trust and a common frame of truth, and so many others - and they can cause direct harm to individuals. And the scale at which these harms can happen is immense.<br />
<br />
So where are the professional groups, the experts in thinking about the risks of democratization of ML, and all the folks concerned about the harms associated with AI tech? Why don't we have the equivalent of the <a href="https://en.wikipedia.org/wiki/Asilomar_Conference_on_Recombinant_DNA">Asilomar conference on recombinant DNA</a>?<br />
<br />
I appreciate that OpenAI has at least raised the issue of thinking through the ethical ramifications of releasing technology. But as the furore over their decision has shown, no single imperfect actor can really claim to be setting the guidelines for ethical technology release, and "starting the conversation" doesn't count when (again as Margaret Mitchell points out) these kinds of discussions have been going on in different settings for many years already.<br />
<br />
Ryan Lowe suggests workshops at major machine learning conferences. That's not a bad idea. But it will attract the people who go to machine learning conferences. It won't bring in the journalists, the people getting SWAT'd (and one case <a href="https://en.wikipedia.org/wiki/2017_Wichita_swatting">killed</a>) by fake news, the women being harassed by trolls online with deep-fake porn images. <br />
<br />
News is driven by news cycles. Maybe OpenAI's announcement will lead to us thinking more about issues of responsible data science. But let's not pretend these are new, or haven't been studied for a long time, or need to have a discussion "started".<br />
<br />
<br />Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-23167411098950402532019-02-02T00:46:00.000-07:002019-02-02T00:46:06.800-07:00More FAT* bloggingSession 3: <a href="https://algorithmicfairness.wordpress.com/2019/01/30/fat-papers-profiling-and-representation/">Representation and Profiling</a><br />
<br />
Session 4: <a href="https://algorithmicfairness.wordpress.com/2019/02/01/fat-papers-fairness-methods/">Fairness methods. </a>Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-60620563581714392652019-01-28T23:48:00.002-07:002019-01-28T23:48:16.276-07:00FAT* Session 2: Systems and Measurement.Building systems that have fairness properties and monitoring systems that do A/B testing on us.<br />
<br />
<a href="https://algorithmicfairness.wordpress.com/2019/01/28/fat-papers-systems-and-measurement/">Session 2 of FAT*</a>: my opinionated summary.Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-81692167496978222902019-01-27T21:09:00.003-07:002019-01-27T21:09:55.785-07:00FAT* bloggingI'll be blogging about each session of papers from the FAT* Conference. So as not to clutter your feed, the posts will be housed at the fairness blog that I co-write along with Sorelle Friedler and Carlos Scheidegger.<br />
<br />
The first post is on <a href="https://algorithmicfairness.wordpress.com/2019/01/27/fat-papers-framing-and-abstraction/">Session 1: Framing and Abstraction</a>.Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-7536473904215694932018-12-20T09:25:00.003-07:002018-12-20T09:25:55.336-07:00The theoryCS blog aggregator REBORN(<i>will all those absent today please email me)</i><div>
<i><br /></i></div>
<div>
<i>(if you can't hear me in the back, raise your hand)</i></div>
<div>
<i><br /></i></div>
<div>
The theoryCS blog aggregator is back up and running at its new location -- <a href="http://cstheory-feed.org/">cstheory-feed.org</a> -- which of course you can't know unless you're subscribed to the new feed, which....</div>
<div>
<br /></div>
<div>
More seriously, we've announced this on the <a href="https://twitter.com/cstheory">cstheory twitter feed</a> as well, so feel free to repost this and spread the word so that all the theorists living in caves plotting their ICML, COLT and ICALP submissions will get the word. </div>
<div>
<br /></div>
<div>
Who's this royal "we"? <a href="https://www.comp.nus.edu.sg/~arnab/">Arnab Bhattacharyya</a> and myself (well mostly Arnab :)). </div>
<div>
<br /></div>
<div>
For anyone interested in the arcana of how the sausage (SoCG?) gets made, read on: </div>
<div>
<br /></div>
<div>
<a href="http://randomwalker.info/">Arvind Narayanan</a> had set up an aggregator based on the <a href="http://www.intertwingly.net/code/venus/">Planet Venus</a> software for feed aggregation (itself based on python packages for parsing feeds). The two-step process for publishing the aggregator works as follows:</div>
<div>
<ol>
<li>Run the software to generate the list of feed items and associated pages from a configuration file containing the list of blogs</li>
<li>Push all the generated content to the hosting server. </li>
</ol>
<div>
Right now, both Arnab and I have git access to the software and config files and can edit the config to update blogs etc. The generator is run once an hour and the results are pushed to the new server. </div>
</div>
<div>
<br /></div>
<div>
So if you have updates or additions, either of us can make the changes and they should be reflected fairly soon on the main page. The easiest way to verify this is to wait a few hours, reload the page and see if your changes have appeared. </div>
<div>
<br /></div>
<div>
The code is run off a server that Arnab controls and both of us have access to the domain registry. I say this in the interest of transparency (<a href="http://fatconference.org/">PLUG</a>!!) but also so that if things go wonky as they did earlier, the community knows who to reach. </div>
<div>
<br /></div>
<div>
Separately, I've been pleasantly surprised at the level of concern and anxiety over the feed -- mainly because it shows what a valuable community resource the feed is and that I'm glad to be one of the curators. </div>
<div>
<br /></div>
<div>
If you've read this far, then you really are interested in the nitty gritty, and so if you'd like to volunteer to help out, let us know. It would be useful for e.g to have a volunteer in Europe so that we have different time zones covered when things break. And maybe our central Politburo (err. I mean the <a href="https://thmatters.wordpress.com/catcs/">committee to advance TCS</a>) might also have some thoughts, especially in regard to their mission item #3:</div>
<blockquote class="tr_bq">
<i>To promote TCS to and increase dialog with other research communities, including facilitating and coordinating the development of materials that educate the general scientific community and general public about TCS.</i></blockquote>
<br />
Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-20317253404433678102018-12-06T00:31:00.000-07:002018-12-06T00:31:09.198-07:00The theoryCS aggregatorAs you all might now, the cstheory blog aggregator is currently down. Many people have been wondering what's going on and when it will be back up so here's a short summary.<br />
<br />
The aggregator has been thus far maintained by Arvind Narayanan who deserves a HUGE thanks for setting up the aggregator, lots of custom code and the linked twitter account. Arvind has been planning to hand it over and the domain going down was a good motivator for him to do that.<br />
<br />
Currently I have all the code that is used to generate the feed, as well as control over the twitter feed. <a href="https://www.comp.nus.edu.sg/~arnab/">Arnab Bhattacharyya</a> has kindly volunteered to be the co-manager of the aggregator. What remains to be done now is<br />
<br />
<ul>
<li>set up a new location to run the aggregator code from</li>
<li>set up hosting for the website</li>
<li>link this to the twitter account. </li>
</ul>
<div>
None of these seem too difficult and the main bottleneck is merely having Arnab and I put together a few hours of work to get this all organized (we have a domain registered already). We hope to have it done fairly soon so you can all get back to reading papers and blogs again. </div>
Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-41489335742598185822018-11-24T15:00:00.000-07:002018-11-24T15:00:05.616-07:00Should credit scores be used for determining residency? It's both exhilarating and frustrating when you see the warnings in papers you write play out in practice. Case in point, <a href="https://www.dhs.gov/publication/proposed-rule-inadmissibility-public-charge-grounds">the proposal by DHS</a> to use credit scores to ascertain whether someone should be granted legal residence.<br />
<br />
<a href="https://slate.com/technology/2018/11/dhs-credit-scores-legal-resident-assessment.html">Josh Lauer at Slate does a nice analysis</a> of the proposal and I'll extract some relevant bits for commentary. First up: what does the proposal call for? (emphasis mine)<br />
<blockquote class="tr_bq">
The new rule, contained in a <a href="https://www.dhs.gov/publication/proposed-rule-inadmissibility-public-charge-grounds">proposal</a> signed by DHS Secretary Kirstjen Nielsen, is designed to help immigration officers identify applicants likely to become a “public charge”—that is, a person primarily dependent on government assistance for food, housing, or medical care. <b>According to the proposal, credit scores and other financial records (including credit reports, the comprehensive individual files from which credit scores are generated) would be reviewed to predict an applicant’s chances of “self-sufficiency.”</b></blockquote>
So what's the problem with this? What we're seeing is an example of the <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3265913">portability trap (from our upcoming FAT* paper)</a>. Specifically, scores designed in a different context (for deciding who to give loans to) are being used in this context (to determine self-sufficiency). Why is this a problem?<br />
<blockquote class="tr_bq">
Unfortunately, this is not what traditional credit scores measure. They are <a href="https://www.consumer.ftc.gov/articles/0152-credit-scores#how.">specialized algorithms</a> designed for <b>one</b> purpose: to predict future bill-paying delinquencies, for any reason. This includes late payments or defaults caused by insurmountable medical debts, job loss, and divorce—three leading causes of personal bankruptcy—as well as overspending and poor money management.</blockquote>
That is, the reason the portability trap is a problem is because you're using one predictor to train another system. And if you're trying to make any estimations about the validity of the resulting process, then you have to know whether the thing you're observing (in this case the credit score) has any relation to the thing you're trying to observe (the construct of "self-sufficiency"). And this is something we harp on a lot in <a href="https://arxiv.org/abs/1609.07236">our paper on axiomatic considerations of fairness</a> (and ML in general)<br />
<br />
And in this case there's a clear disconnect:<br />
<blockquote class="tr_bq">
Credit scores do not predict whether an individual will become a public charge. And they do not predict financial self-sufficiency. They are only useful in this context if one believes credit scores reveal something about a person’s character. In other words, if one believes that people with low credit scores are moochers and malingerers. <b>Given the Trump administration’s hostility toward (brown-skinned) immigrants, this conflation of credit scores and morality is not surprising.</b></blockquote>
And this is a core defining principle of our work: that beliefs about the world control how we choose our representations and learning procedures: the procedures cannot be justified except in the context of the beliefs that underpin them. <br />
<br />
I think that if you read anything I've written, it will be clear where I stand on the normative question of whether this is a good idea (tl;dr: NOT). But as a researcher, it's important to lay out a principled reason for why, and this sadly merely confirms that our work is on the right track.<br />
<br />
<br />Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-69497590698300303092018-11-02T01:42:00.001-06:002018-11-02T01:42:58.112-06:00What do I work on ? <div style="text-align: center;">
<i>So, what do you work on? </i></div>
<div style="text-align: center;">
<i><br /></i></div>
<div style="text-align: left;">
As questions go, this is one of the most rudimentary. It's the conference equivalent of "Nice weather we're having", or "How about them Broncos!". It's a throat-clearer, designed to start a conversation in an easy non-controversial way. </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
And yet I'm always having to calculate and calibrate my answers. There's a visible pause, a hesitation as I quickly look through my internal catalog of problems and decide which one I'll pull out. On the outside, the hesitation seems strange: as if I don't quite know <b>what</b> I work on, or if I don't know how to explain it. </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
It's an occupational hazard that comes from living on the edge of many different areas. I go to data mining conferences, machine learning conferences, theory/geometry conferences, and (now) conferences on ethics, society and algorithms. And in each place I have a different circle of people I know, and a different answer to the question</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: center;">
<i>So, what do you work on? </i> </div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: left;">
It makes me uncomfortable, even though it shouldn't. I feel like I can only share a part of my research identity because otherwise my answer will make no sense or (worse!) seem like I'm trying to impress people with incomprehensible words. </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I don't doubt that most people share some form of this feeling. As researchers, none of us are one-dimensional, and most of us work on many different problems at a time. Probably the easiest answer to the question is the problem that one has most recently worked on. But I sense that my case is a little unusual: not the breadth per se, but the range of topics (and styles of problem solving) that I dabble in. </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: center;">
<i>So, what do you work on? </i></div>
<div style="text-align: center;">
<i><br /></i></div>
<div style="text-align: left;">
I often joke that my research area is a random walk through computer science and beyond. I started off in geometry, dabbled with GPUs (alas, before they were popular), found my way into information theory and geometry (and some differential geometry), slipped down the rabbit hole into data mining, machine learning, and a brief side foray into deep learning, and then built a nice little cottage in algorithmic fairness, where I spend more time talking to social scientists and lawyers than computer scientists.<br />
<br />
Being an academic nomad has its virtues: I don't really get bored with my work. But it also feels like I'm always starting from square one with my learning and that there are always people who know way more about every topic than I do. And my academic roamings seem to mirror my actual nomadic status. I'm a foreigner in a land that gets stranger and less familiar by the day, and the longest time I've spent in any location is the place I'm in right now.<br />
<br />
<br />
<div style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: center; text-decoration-color: initial; text-decoration-style: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">
</div>
<br />
<div style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: center; text-decoration-color: initial; text-decoration-style: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;">
<div style="font-style: normal; margin: 0px;">
<i>So, what do you work on? </i></div>
<div style="font-style: normal; margin: 0px;">
<i><br /></i></div>
<div style="font-style: normal; margin: 0px; text-align: left;">
Maybe, in a way that's so American, "What do you work on" is really a question of "Who are you" in the way we bind together our work and our identity. When my students come and ask me what they should work on, what they're really asking me is to tell them what their research identity is, and my answer usually is, "whatever you want it to be right now". It's a frustrating answer no doubt, but I feel that it lowers the import of the question to a manageable level. </div>
<div style="font-style: normal; margin: 0px; text-align: left;">
<br /></div>
<div style="margin: 0px; text-align: center;">
<i>So, what DO you work on?</i></div>
<div style="margin: 0px; text-align: center;">
<i><br /></i></div>
<div style="margin: 0px; text-align: left;">
I do algorithmic fairness, and think about the ethics of automated decision-making. I bring an algorithmic (and geometric) sensibility to these questions. I'm an amateur computational philosopher, a bias detective, an ML-translator for lawyers and policy folk, and my heart still sings when I see a beautiful lemma. </div>
<div style="font-style: normal; margin: 0px; text-align: left;">
<br /></div>
<div style="font-style: normal; margin: 0px; text-align: left;">
<br /></div>
</div>
</div>
Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-30048437055593228482018-10-22T08:05:00.000-06:002018-10-22T08:05:10.656-06:00On teaching ethics to tech companiesKara Swisher (who is unafraid to call it like it is!) has a new op-ed in the NYT titled "<a href="https://www.nytimes.com/2018/10/21/opinion/who-will-teach-silicon-valley-to-be-ethical.html">Who will teach Silicon Valley to be ethical</a>". She asks<br />
<blockquote class="tr_bq">
How can an industry that, unlike other business sectors, persistently promotes itself as doing good, learn to do that in reality? Do you want to not do harm, or do you want to do good? These are two totally different things. </blockquote>
<blockquote class="tr_bq">
And how do you put an official ethical system in place without it seeming like you’re telling everyone how to behave? Who gets to decide those rules anyway, setting a moral path for the industry and — considering tech companies’ enormous power — the world.</blockquote>
<br />
There are things that puzzle me about this entire discussion about ethics and tech. It seems like an interesting idea for tech companies to incorporate ethical thinking into their operations. Those of us who work in this space are clamoring for more ethics education for budding technologists.<br />
<br />
There is of course the cynical view that this is merely window dressing to make it look like Big Tech (is that a phrase now?) cares without actually having to change their practices.<br />
<br />
But let's put that aside for a minute. Suppose we assume that indeed tech companies are (in some shape of form) concerned about the effects of technology on society and that their leaders do want to do something about it.<br />
<br />
What I really don't understand is the idea that we should teach Silicon Valley to be ethical. This seems to play into the overarching narrative that tech companies are trying to do good in the world and slip up because they're not adults yet -- a problem that can be resolved by education that will allow them to be good "citizens" with upstanding moral values.<br />
<br />
This seems rather ridiculous. When chemical companies were dumping pesticides on the land by the ton and Rachel Carson wrote Silent Spring, we didn't shake our heads sorrowfully at companies and sent them moral philosophers. We founded the EPA!<br />
<br />
When the milk we drink was being adulterated with borax and formaldehyde and all kinds of other horrific additives that Deborah Blum documents so scarily in her new book '<a href="https://www.amazon.com/gp/product/1594205140/ref=as_li_qf_asin_il_tl?ie=UTF8&tag=gastropod-20&creative=9325&linkCode=as2&creativeASIN=1594205140&linkId=e32674164044af30a6d9284204848e4c">The Poison Squad</a>', we didn't shake our heads sorrowfully at food vendors and ask them to grow up. We passed a law that led eventually to the formation of the FDA.<br />
<br />
Tech companies are companies. They are not moral agents, or even immoral agents. They are amoral profit-maximizing vehicles for their shareholders (and this is not even a criticism). Companies are supposed to make money, and do it well. Facebook's stock price didn't slip when it was discovered how their systems had been manipulated for propaganda. It slipped when they proposed changes to their newsfeed ratings mechanisms to address these issues.<br />
<br />
It makes no sense to rely on tech companies to police themselves, and to his credit, Brad Smith of Microsoft made exactly this point <a href="https://blogs.microsoft.com/on-the-issues/2018/07/13/facial-recognition-technology-the-need-for-public-regulation-and-corporate-responsibility/">in a recent post on face recognition systems</a>. Regulation, policing and whatever else we might imagine, has to come from the outside. While I don't claim that regulation mechanisms all work as they are currently conceived, the very idea of checks and balances seems more robust than merely hoping that tech companies will get their act together on their own.<br />
<br />
Don't get me wrong. It's not even clear what has to be regulated here. Unlike with poisoned food or toxic chemicals, it's not clear how to handle poisonous speech or toxic propaganda. And that's a real discussion we need to have.<br />
<br />
But let's not buy into Silicon Valley's internal hype about "doing good". Even Google has dropped its "Don't be evil" credo.<br />
<br />Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-65827353708012936682018-10-11T09:06:00.000-06:002018-10-11T09:13:19.493-06:00Google's analysis of the dilemma of free speech vs hate speechBreitbart just acquired a leaked copy of an internal google doc taking a cold hard look at the problems of free speech, fake news and censorship in the current era. I wrote a <a href="https://twitter.com/geomblog/status/1050400879474401281">tweet storm</a> about it, but also wanted to preserve it here because tweets, once off the TL, cease to exist.<br />
<div>
<br /></div>
<blockquote class="tr_bq">
Breitbart acquired an internal google doc discussing the misinformation landscape that the world finds itself in now: <a href="https://t.co/KXdyttcuFs">https://www.scribd.com/document/390521673/The-Good-Censor-GOOGLE-LEAK#from_embed …</a> </blockquote>
<blockquote class="tr_bq">
I almost wish that Google had put out this document to read in public. It's a well thought out exploration of the challenges faced by all of us in dealing with information dissemination, fake news, censorship and the like. And to my surprise, it (mostly) is willing to point figures backwards at Google and other tech companies for their role in it. (although there are some glaring omissions like the building of the new censored search tool in China). It's not surprising that people inside Google are thinking carefully about these issues, even as they flail around in public. And the analysis is comprehensive without attempting to provide glib solutions<br />
<br />
Obviously, since this is a doc generated within Google, the space of solutions is circumscribed to those that have tech as a major player. For e.g the idea of publicly run social media isn't really on the table, or even better ways to decentralize value assignment for news, or alternate models for search that don't require a business model. But with those caveats in mind, the analysis of the problems is reasonable. </blockquote>
Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-73062661760347504152018-10-08T09:00:00.000-06:002018-10-08T09:00:02.107-06:00A new sexual harassment policy for TCS conferences. One of my most visited posts is the anonymous post by a <a href="http://blog.geomblog.org/2018/02/a-metoo-testimonial-that-hits-close-to.html">theoryCS colleague describing her own #metoo moments inside the TCS conference circuit</a>. It was a brutal and horrific story to read.<br />
<br />
Concurrently (I don't know if the blog post had an effect, but one can but hope it helped push things along), a committee was set up under the auspices of TCMF (FOCS), ACM, SIAM, and EATCS to<br />
<div class="page" title="Page 1">
<div class="layoutArea">
<div class="column">
<blockquote class="tr_bq">
<span style="font-family: "cmr10"; font-size: 11pt;">Draft a proposal for joint ToC measures to combat discrimination, harassment, bullying, and retaliation, and all matters of ethics that might relate to that.</span></blockquote>
That committee has now completed its work, and <a href="https://www.ics.uci.edu/%7Eirani/safetoc.html">a final report is available</a>. The report was also endorsed at the FOCS business meeting this week. The report is short, and you should read it. The main takeaways/recommendations are that every conference should<br />
<ul>
<li>adopt a code of conduct and post it clearly. </li>
<li>recruit and train a group of advocates to provide confidential support to those facing problems at a conference</li>
<li>have mechanisms for authors to declare a conflict of interest without needing to be openly specific about the reasons. </li>
</ul>
<div>
There are many useful references in the report, as well as more concrete suggestions about how to implement the above recommendations. This committee was put together fast, and generated a very useful report quickly. Well done!</div>
<div>
<br /></div>
</div>
</div>
</div>
Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-75572693873241142452018-09-10T00:19:00.000-06:002018-09-10T07:18:33.367-06:00Hello World: A short review<em>A short review of Hannah Fry's new book 'Hello World'</em><br />
<em><br /></em>
Starting wth Cathy O'Neill's Weapons of Math Destruction, there's been an onslaught of books sounding the alarm about the use of algorithms in daily life. My Amazon list that collects these together is even called 'Woke CS'. These are all excellent books, calling out the racial, gender, and class inequalities that algorithmic decision-making can and does exacerbate and the role of Silicon Valley in perpetuating these biases. <br />
<br />
Hannah Fry's new book "<a href="https://www.penguin.co.uk/books/1114076/hello-world/">Hello World</a>" is not in this category. Not exactly, anyway. Her take is informative as well as cautionary. Her book is as much an explainer of <em>how</em> algorithms get used in contexts ranging from justice, to medicine, to art, as much as it is a reflection on what this algorithmically enabled world will look like from a human perspective. <br />
<br />
And in that sense it's a far more optimistic take on our current moment than I've read in a long time. In a way it's a relief: I've been mired for so long in the trenches of bias and discrimination, looking at the depressing and horrific ways in which algorithms are used as tools of oppression, that it can be hard to remember that I'm a computer scientist for a reason: I actually do marvel at and love the idea of computation as a metaphor, as a tool, and ultimately as a way to (dare I say it) do good in the world. <br />
<br />
The book is structured around concepts (Power, data) and domains (justice, medicine, cars, crime and art). After an initial explainer on how algorithms function (and also how models are trained using machine learning), and how data is used to fuel these algorithms, she very quickly gets into specific case studies of both the good and the bad in algorithmically mediated decision making. Many of the case studies are from the UK and were unknown to me before this book. I quite liked that: it's easy to focus solely on examples in the US, but the uses (and misuse) of algorithms is global (Vidushi Mardia's <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3240384">article on AI policy in India</a> has similar locally-sourced examples). <br />
<br />
If you're a layman looking to get a general sense of <em>how</em> algorithms tend to show up in decision making systems, how they hold out hope for a better way of solving problems and where they might go wrong, this is a great book. It uses a minimum of jargon, while still beiing willing to wade into the muck of false positives and false negatives in a very nice illustrative example in the section on recidivism prediction and COMPAS, and also attempting to welcome the reader into the "Church of Bayes". <br />
<br />
If you're a researcher in algorithmic fairness, like me, you start seeing the deeper references as well. Dr. Fry alludes to many of the larger governance issues around algorithmic decision making that we're wrestling with now in the FAT* community. Are there better ways to integrate automated and human decision-making that takes advantage of what we are good at? What happens when the systems we build start to change the world around them? Who gets to decide (and how) what level of error in a system is tolerable, and who might be affected by it? As a researcher, I wish she had called out these issues a little more, and there are places where issues she raises in the book have actually been addressed (and in some cases, <a href="https://arxiv.org/abs/1706.09847">answered</a>) by researchers. <br />
<br />
While the book covers a number of different areas where algorithms might be taking hold, it takes very different perspectives on the appropriateness of algorithmic decision-making in these domains. Dr. Fry is very clear (and rightly so) that criminal justice is one place where we need very strong checks and balances before we can countenance the use of any kind of algorithmic decision-making. But I feel that maybe she's letting off the medical profession a little easy in the chapter on medicine. While I agree that biology is complex enough that ML-assistance might lead us to amazing new discoveries, I think some caution is needed, especially since there's ample evidence that the benefits of AI in medicine might only accrue to the (mostly white) populations that dominate the clinical trials. <br />
<br />
Similarly, the discussion of creativity in art and what it means for an algorithm to be creative is fascinating. The argument Dr. Fry arrives at is that art is fundamentally human in how it exists in transmission -- from artist to audience -- and that art cannot be arrived at "by accident" via data science. It's a bold claim, and of a kind with many claims about the essential humanness of certain activities that have been pulverized by advances in AI. Notwithstanding, I find it very appealing to posit that art is essentially a human endeavour by definition. <br />
<br />
But why not extend the same courtesy to the understanding of human behavior or biology? Algorithms in criminal justice are predicated on the belief that we can predict human behavior and how our interventions might change it. We expect that algorithms can pierce the mysterious veil of biology, revealing secrets about how our body works. And yet the book argues not that these systems are fundamentally flawed, but that precisely <em>because</em> of their effectiveness they need governance. I for one am a lot more skeptical about the basic premise that algorithms can predict behavior to any useful degree beyond the aggregate (and perhaps <a href="https://en.wikipedia.org/wiki/Psychohistory_(fictional)">Hari Seldon</a> might agree with me). <br />
<br />
Separately, I found it not a little ironic, in a time when Facebook is constantly being yanked before the US Congress, Cambridge Analytica might have swayed US elections and Brexit votes, and Youtube is a dumpster fire of extreme recommendations, that I'd read a line like "Similarity works perfectly well for recommendation engines" in the context of computer generated art. <br />
<br />
The book arrives at a conclusion that I feel is JUST RIGHT. To wit, algorithms are not authorities, and we should be skeptical of how they work. And even when they might work, the issues of governance around them are formidable. But we should not run away from the potential of algorithms to truly help us, and we should be trying to frame the problem away from the binary of "algorithms good, humans bad" or "humans good, algorithms bad" and towards a deeper investigation of how human and machine can work together. I cannot read <br />
<blockquote>
Imagine that, rather than exlcusively focusing our attention on designing our algorithm to adhere to some impossible standard of perfect fairness, we instead designed them to facilitate redress when they inevitable erred; that we put as much time and effort into ensuring that automatic systems were as easy to challenge as they are to implement. </blockquote>
without wanting to stand up and shout "<b><i>HUZZAH</i></b>!!!". (To be honest, I could quote the entire conclusions chapter here and I'd still be shouting "<b><i>HUZZAH</i></b>"). <br />
<br />
It's a good book. Go out and buy it - you won't regret it. <br />
<br />
<em>This review refers to an advance copy of the book, not the released hardcover. The advance copy had a glitch where a fragment of latex math remained uncompiled. This only made me happier to read it.</em><br />
Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-36259862461756382282018-08-30T05:31:00.000-06:002018-08-30T05:31:10.147-06:00Clustering: a draft of a part!For the last X years (X being a confidential and never to be revealed number, but large enough that AI was more than just deep learning at the time), Sergei Vassilvitskii and I have been toiling away at a book on clustering.<br />
<br />
The book isn't ready yet, but we do have a draft of part I (the core of the book). <a href="http://clustering.cc/">Check it out</a>, and send any comments you might have to <a href="mailto:clusteringbook@gmail.com">clusteringbook@gmail.com</a>.Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-59368870986380757962018-02-14T07:00:00.000-07:002018-02-14T07:00:37.576-07:00A #metoo testimonial that hits close to home...<i>This is a guest post by a colleague in the TCS community, a person I know. If you read other TCS blogs you might come across this there. This is by design. Please do read it. </i><br />
<i><br /></i>
Every #MeToo story over the last several months has made me pause. My heart races and my concentration fails. The fact that the stories have largely focused on the workplace adds to my difficulty.<br /><br />Do I speak out too?<br /><br />I have shared a few stories with colleagues about things that have happened to me in school and at work. But these stories have been somewhat lighthearted events that have been easy to share without outing the perpetrators.<br /><br />For example, I have told a story about a university employee telling me, in so many words, that I should be barefoot and pregnant and not in the office. What I didn't share is that the same employee, later that year -- despite the fact that our common boss knew about this story because I did indeed report it -- was awarded a best employee award. How do you think that made me feel? Like my experience didn't matter and that such comments are condoned by our department. Why didn't I share that information widely? Because I was worried that folks would then be able to figure out who the culprit was. And isn't that even worse? Shouldn't it be the sexist who is worried and not the woman who, yet again, is made to feel like she doesn't belong?<br /><br />---<br /><br />Let me tangent a bit. For years I have not flown. Ostensibly I stopped flying because of the contribution to the climate crisis. When I travel, I go by train. It takes longer, but has been surprisingly pleasant. And when travel takes 3-4 times as long, you don't do it as often, further reducing your carbon footprint. Of course, that means that I don't go to conferences unless they are nearby.<br /><br />But when I really think about it, is this really the reason I stopped going to conferences? A conference I would normally go to was held nearby a few years ago and I didn't go. Sure, I suffered a grievous injury two weeks before, but I hadn't even registered. I had planned to not go long before that injury.<br /><br />So, really, why do I no longer attend conferences? Partly I don't feel that I need to anymore, now that I have tenure. When I stopped attending conferences, I was able to "coast into" tenure. Letter writers would remember me. I essentially stopped going to conferences and workshops as soon as I possibly could. <br /><br />---<br /><br />Back to the beginning, or close to.<br /><br />I was nervous at the first conference I attended as a graduate student. One of the reasons I was nervous was that I was athletic at the time and planned on daily runs while I was attending -- I was worried that it might be viewed as a waste of time. My advisor, who also went to the conference, found out about my athleticism and suggested we run together. This was a relief to me. That is, until we were running and he started talking about his lackluster sex life with his wife. I responded by picking up the pace and feigning an illness on the remaining days. On the last day of the conference we were out for dinner with a large group of people and dinner went late into the night. I excused myself, as I had a 4AM bus to catch. My advisor walked me out of the restaurant and awkwardly said something about wanting me to stay and that we should talk. I stuck to leaving, knowing that I needed some sleep before the long trip home the next day. He said we should talk when we were back in the office. Honestly, at the time I thought he was going to complain about my talk or my professional performance in some way. I worried about it all through the weekend until we met next. I brought it up at the end of our meeting, asking what he wanted to talk about, naively expecting professional criticism. When he said I must surely know, in a certain voice, I knew he wasn't talking about work. I feigned ignorance, and he eventually brushed it off and said not to worry. In the coming months, he would cancel meetings and otherwise make himself unavailable. After a half year I realized I wouldn't be able to be successful without having a supportive advisor and, despite first planning to quit grad school, found a new advisor and moved on. That former advisor barely made eye contact with me for the remainder of my time in graduate school.<br /><br />Fast forward many years. I was at a small workshop as a postdoc. A senior and highly respected researcher invited me to dinner. I was excited at the opportunity to make a stronger connection that would hopefully lead to a collaboration. However, at dinner he made it very clear that this was not professional by reaching across the table and stroking my hands repeatedly. I don't even recall how I handled it. Perhaps I should have expected it -- a grad school friend of mine had a similar, and probably worse, interaction with this same researcher. Shortly after I got to my room at the hotel, my hotel room phone rang. It was him. He wanted to continue our conversation. I did not.<br /><br />Perhaps a year later, still as a postdoc, I was at a party and a colleague from another university was there too. At the end of the party, we were alone. We flirted, mutually. Flirting led to kissing, kissing led to him picking me up in a way that asserted how much stronger he is than me, which led to my utter discomfort, which led to me saying no, stop, repeatedly. Which he didn't listen to. Which led to a calculation in my head. I could either resist and risk physical injury or I could submit. I chose to submit, without consent.<br /><br />For the record, that is called rape.<br /><br />For a long while, I suppressed it. I pretended in my own head that it didn't happen that way, that it was consensual. I even tried to continue working with him -- always in public places, mind you. The wall in my mind gradually broke down over the years until several years later, we were at the same workshop where the doors of the rooms didn't have locks. You could lock them from the inside, but not the outside. I remember worrying that he would be lurking in my room and took to making sure I knew where he was before I ventured back to sleep.<br /><br />---<br /><br />So why would I continue to go to workshops and conferences when that is the environment I know I will face? Even if I felt safe, if 95% of the attendees are men, how many look at me as a colleague and how many look at me as a potential score? When I was going up for tenure, I thought long and hard about listing the senior-and-highly-respected researcher on a do-not-ask-for-a-letter list. But where would it stop? Do I include all the people who hit on me? All the people who stared at my breasts or commented on my body? All the people who I had been given clear signals that they didn't see me as a colleague and equal member of the research community, but as a woman meant to be looked at, hit on, touched inappropriately.<br /><br />Should I have quit grad school when I had the chance? We all know it isn't any better in industry. Should I have pursued another discipline? <a href="https://www.chronicle.com/article/A-Complete-Culture-of/242040">No discipline, it seems, is immune to sexualization of women</a>. But I think the situation is uniquely horrible in fields where there are so few women. At conferences in theoretical computer science, 5-10% of the attendees are women, as a generous estimate. The numbers aren't in women's favor. The chances that you will get hit on, harassed, assaulted are much higher. There is a greater probability that you will be on your own in a group of men. You can't escape working with men. It is next to impossible to build a career when you start striking men off your list of collaborators in such a field. That is not to say there aren't wonderful men to work with. There are many men in our field that I have worked with and turned to for advice and spent long hours with and never once had detected so much as a creepy vibe. But you can't escape having to deal with the many others who aren't good. When you meet someone at a conference, and they invite you for a drink or dinner to continue the conversation, how do you know that they actually want to talk about work, or at least treat you as they would any colleague? How do you make that decision?<br /><br />I hung on until I no longer needed to go to conferences and workshops to advance my career to the stability of tenure. But surely my career going forward will suffer. My decision is also hard on my students, who go to conferences on their own without someone to introduce them around. It is hard on my students who can't, for visa difficulties, go to the international conferences that I am also unwilling to go to, so we roll the dice on the few domestic conferences they can go to.<br /><br />And now I am switching fields. Completely. I went to two conferences last summer. The first, I brought the protective shield of my child and partner. The second, I basically showed up for my talk and nothing else. I wasn't interested in schmoozing. It'll be difficult, for sure, to establish myself in a new field without fully participating in the expected ways.<br /><br />Is all this why I am switching fields? Not entirely, I'm sure, but it must have played a big role. If I enjoyed conferences as much as everyone else seems to, and didn't feel shy about starting new collaborations, I might be too engrossed to consider reasons to leave. And certainly, the directions I am pursuing are lending themselves to a much greater chance of working with women.<br /><br />Why am I speaking out now? The #MeToo moment is forcing me to think about it, of course. But I have been thinking about this for years. I hope it will be a relief to get it off my chest. I have been <a href="https://www.theguardian.com/commentisfree/2018/jan/26/germaine-greer-metoo-sexual-harassment-whingeing">"getting on with it"</a> for long enough. <a href="https://en.wikipedia.org/wiki/Rape_in_the_United_States#Statistics">1 in 5 women will deal with rape in their lifetime.</a> 1 in 5! You would think that I would hear about this from friends. But I hadn't told anyone about my rape. And almost no one has told me about theirs. I think it would help, in the very least therapeutically, to talk about it.<br /><br />---<br /><br />I thought about publishing this somewhere, anonymously, as a "woman in STEM". I considered publishing it non-anonymously, but was shy to deal with the trolls. I didn't want to deal with what many women who speak out about their experiences face: have their life be scrutinized, hear excuses being made on behalf of the predators, generally have their experiences denied. But I think by posting it here, many people in theoretical computer science will read it, rather than a few from the choir. I am hoping that you will talk to each other about it. That you will start thinking of ways to make our community better for others. In all my years of going to conferences and workshops, of all the inappropriate comments and behaviors that others have stood around and witnessed, never once did any of the good ones call that behavior out or intervene. Maybe they did so in private, but I think it needs to be made public. Even the good ones can do better.<br /><br />What can you do?<br /><br />While you couldn't have protected me from being raped, you can think about the situations we are expected to be in for our careers -- at workshops in remote locations, where we're expected to drink and be merry after hours. I hope not many of us have been raped by a colleague, but even if you haven't, it doesn't take many instances of being hit on or touched inappropriately to begin to feel unsafe.<br /><br />I remember being at a conference and, standing in a small group, an attendee interrupted a conversation I was having to tell me that my haircut wasn't good, that I shouldn't have cut my hair short. I tried to ignore it, and continue my conversation, but he kept going on about it. Saying how I would never attract a man with that haircut. No one said anything. Speak up. Just say -- shut up! -- that's not appropriate. Don't leave it up to the people who have to deal with this day in day out to deal with it on their own. Create a culture where we treat each other with respect and don't silently tolerate objectification and worse.<br /><br />I regret never reporting my first graduate advisor's behavior, but is it my fault? I had no idea who to report it to. I had no idea either in undergrad who I would report such behavior to. Where I am now is the first place I've been that has had clear channels for reporting sexual harassment and other damaging situations. The channels are not without problems, but I think the university is continuing to improve them. Perhaps we should have a way of reporting incidents in our field. I have a hard time believing, given that myself and a grad school friend had similar experiences with the same senior-and-highly-respected researcher, that others in the field don't know that he is a creep. It is up to you to protect the vulnerable of our community from creeps and predators. Keep an eye on them. Talk to them. Don't enable them. As a last resort, shame and isolate them.Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-1173328529291126182018-01-22T09:00:00.000-07:002018-01-22T09:00:18.345-07:00Double blind review: continuing the discussionMy first two posts on double blind review triggered good discussion by <a href="http://mybiasedcoin.blogspot.com/2018/01/double-blind-alenex.html">Michael Mitzenmacher</a> and <a href="https://windowsontheory.org/2018/01/11/on-double-blind-reviews-in-theory-conferences/#comments">Boaz Barak</a> (see the comments on these posts for more). I thought I'd try to synthesize what I took away from the posts and how my own thinking has developed.<br />
<br />
First up, I think it's gratifying to see that the the basic premise: "single blind review has the potential for bias, especially with respect to institutional status, gender and other signifiers of in/out groups" is granted at this point. There was a time in the not-so-distant past that I wouldn't be able to even establish this baseline in conversations that I'd have.<br />
<br />
The argument therefore has moved to one of tradeoffs: <i>does the installation of DB review introduce other kinds of harm while mitigating harms due to bias?</i><br />
<br />
Here are some of the main arguments that have come up:<br />
<h4>
Author identity carries valuable signal to evaluate the work. </h4>
<div>
This argument manifested itself in comments (and I've heard it made in the past). <a href="https://windowsontheory.org/2018/01/11/on-double-blind-reviews-in-theory-conferences/#comment-43910">One specific version of it that James Lee articulate</a>s is that all reviewing happens in a resource-limited setting (the resource here being time) and so signals like author identity, while not necessary to evaluate the correctness of a proof, provide a prior that can help focus one's attention. </div>
<div>
<br /></div>
<div>
My instinctive reaction to this is "you've just defined bias". But on reflection I think James (and others people who've said this) are pointing out that abandoning author identity is not for free. I think that's a fair point to make. But I'd be hard pressed to see why this increase in effort negates the fairness benefits from double blind review (and I'm in general a little uncomfortable with this purely utilitarian calculus when it comes to bias).<br />
<br />
As a side note, I think that focusing on paper correctness is a mistake. As Boaz points out, this is not the main issue with most decisions on papers. What matters much more is "interestingness", which is very subjective and much more easily bound up with prior reactions to author identity. </div>
<h4>
Some reviewers may be aware of author identity and others might not. This inconsistency could be a source of error in reviewing.</h4>
<div>
Boaz makes this point in his argument against DB review. It's an interesting argument, but I think it also falls into the trap of absolutism: i.e imperfections in this process will cause catastrophic failure. This point was made far more eloquently <a href="https://acl2017.wordpress.com/2017/02/19/arxiv-and-the-future-of-double-blind-conference-reviewing/#comment-131">in a comment</a> on a blog post about ACL's double blind policy (emphasis mine). </div>
<div>
<br /></div>
<div>
<blockquote class="tr_bq">
I think this kind of all-or-nothing position fails to consider one of the advantages of blind review. Blind review is not only about preventing positive bias when you see a paper from an elite university, <i>it’s also about the opposite: preventing negative bias when you see a paper from someone totally unknown</i>. Being a PhD student from a small group in a little known university, the first time I submitted a paper to an ACL conference I felt quite reassured by knowing that the reviewers wouldn’t know who I was. </blockquote>
<blockquote class="tr_bq">
In other words, under an arXiv-permissive policy like the current one, authors still have the *right* to be reviewed blindly, even if it’s no longer an obligation because they can make their identity known indirectly via arXiv+Twitter and the like. I think that right is important. So the dilemma is not a matter of “either we totally forbid dissemination of the papers before acceptance in order to have pure blind review (by the way, 100% pure blind review doesn’t exist anyway because one often has a hint of whom the authors may be, and this is true especially of well-known authors) or we throw the baby out with the bathwater and dispense with blind review altogether”. I think blind review should be preserved at least as a right for the author (as it is know), and the question is whether it should also be an obligation or not.</blockquote>
</div>
<h4>
Prepublication on the arXiv is a desirable goal to foster open access and the speedy dissemination of information. Double blind review is irrevocably in conflict with non-anonyous pre-print dissemination.</h4>
<div>
This is perhaps the most compelling challenge to implementing double blind review. The arXiv as currently constructed is not designed to handle (for e.g) anonymous submissions that are progressively blinded. <a href="https://acl2017.wordpress.com/2017/02/19/arxiv-and-the-future-of-double-blind-conference-reviewing/">The post that the comment above came from</a> has an <b>extensive</b> discussion of this point, and rather than try to rehash it all here, I'd recommend that you read the post and the comments. </div>
<div>
<br />
But the comments also question the premise head on: specifically, "does it really slow things down" and "so what?". Interestingly, <a href="https://nlpers.blogspot.com/2015/10/a-small-observation-for-prepubs-on-arxiv.html">Hal Daumé made an attempt to answer the "really?" question</a>. He looked at arXiv uploads in 2014-2015 and correlated them with NIPS papers. The question he was trying to ask was: is there evidence that more papers uploaded to the arXiv before submission to NIPS in the interest of getting feedback from the community? His conclusion was that there was little evidence to support the idea that the arXiv had radically changed the normal submit-revise cycle of conferences. I'd actually think that theoryCS might be a little better in this regard, but I'd also be dubious of such claims without seeing data.<br />
<br />
In the comments, even the question of "so what?" is addressed. And again this boils down to tradeoffs. While I'm not advocating that we ban people from putting their work on the arXiv, ACL has done precisely this, by asserting that the relatively short delay between submission and decision is worth it to ensure the ability to have double blind review.<br />
<br />
<h3>
Summary</h3>
</div>
<div>
I'm glad we're continuing to have this discussion, and talking about the details of implementation is important. Nothing I've heard has convinced me that the logistical hurdles associated with double blind review are insurmountable or even more than inconveniences that arise out of habit, but I think there are ways in which we can fine tune the process to make sense for the theory community. </div>
Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-81242473592102285772018-01-09T00:41:00.000-07:002018-01-09T00:41:39.915-07:00Double blind review at theory conferences: More thoughts. I've had a number of discussions with people both before and after the report that Rasmus and I wrote on the double-blind experiment at ALENEX. And I think it's helpful to lay out some of my thoughts on both the purpose of double blind review as I understand it, and the logistical challenges of implementing it.<br />
<br />
<b>What is the purpose of double blind review? </b><br />
<b><br /></b>
The goal is to mitigate the effects of the unconscious, implicit biases that we all possess and that influence our decision making in imperceptible ways. It's not a perfect solution to the problem. But there is now a large body of evidence suggesting that<br />
<br />
<ul>
<li>All people are susceptible to implicit biases, whether it be regarding institutional status, individual status, or demographic stereotyping. And what's worse that we are incredibly bad at assessing or detecting our own biases. At this point, a claim that a community is <i>not </i>susceptible to bias is the one that needs evidence. </li>
<li>Double blind review can mitigate this effect. Probably the most striking example of this is the case of orchestra auditions, where requiring performers to play behind a screen dramatically increased the number of women in orchestras. </li>
</ul>
<div>
<b>What is NOT the purpose of double blind review? </b></div>
<div>
<b><br /></b></div>
<div>
Double blind review is not a way to prevent anyone from ever figuring out the author identity. So objections to blinding based on scenarios where author identity is partially or wholly revealed are not relevant. Remember, the goal is to eliminate the <i>initial </i>biases that come from the first impressions. </div>
<div>
<br /></div>
<div>
<b>What makes DB review hard to implement at theory venues? </b></div>
<div>
<b><br /></b></div>
<div>
Theory conferences do two things that are different from other communities. We</div>
<div>
<ul>
<li>require that PC members do NOT submit papers</li>
<li>allow PC members to initiate queries for external subreviewers. </li>
</ul>
<div>
These two issues are connected. </div>
<div>
<ol>
<li>If you don't allow PC members to submit papers, you need a small PC. </li>
<li>If you have a small PC, each PC member is responsible for many papers. </li>
<li>If each PC member is responsible for many papers, they need to outsource the effort to be able to get the work done. </li>
</ol>
<div>
As we mentioned earlier, it's not possible to have PC members initiate review requests if they don't know who might be in conflict with a paper whose authors are invisible. So what do we do? </div>
</div>
</div>
<div>
<br /></div>
<div>
There's actually a reasonably straightforward answer to this. </div>
<br />
<br /><ul>
<li>We construct the PC as usual with the usual restrictions.</li>
<li>We construct a list of “reviewers”. For example, "anyone with a SODA/STOC/FOCs paper in the last 5 years” or something like that. Ideally we will solicit nominations from the PC for this purpose.</li>
<li>We invite this list of people to be reviewers for SODA, and do this BEFORE paper submission</li>
<li>authors will declare conflicts with reviewers and domains (and reviewers can also declare conflicts with domains and authors) </li>
<li>at bidding time, the reviewers will be invited to bid on (blinded) papers. The system will automatically assign people. </li>
<li>PC members will also be in charge of papers as before, and it’s their job to manage the “reviewers” or even supply their own reviews as needed. </li>
</ul>
Any remaining requests for truly external sub reviewing will be handled by the PC chairs. I expect this number will be a lot smaller.<div>
<br /></div>
<div>
Of course all of this is pretty standard at venues that implement double blind review. </div>
<div>
<br /></div>
<div>
<b>But what if a sub-area is so small that all the potential reviewers are conflicted</b></div>
<div>
<b><br /></b></div>
<div>
well if that's the case, then it's a problem we face right now. And DB review doesn't really affect it. </div>
<div>
<br /></div>
<div>
<b>What about if a paper is on the arXiv? </b></div>
<div>
<b><br /></b></div>
<div>
We ask authors and reviewers to adhere to double blind review policies in good faith. Reviewers are not expected to go hunting for the author names, and authors are expected to not draw attention to information that could lead to a reveal. Like with any system, we trust people to do the right thing, and that generally works. </div>
<div>
<br /></div>
<div>
<b>But labeling CoI for so many people is overwhelming.</b></div>
<div>
<b><br /></b></div>
<div>
It does take a little time, but less time than one expects. Practically, many CoIs are handled by institutional domain matching, and most of the rest are handled by explicit listing of collaborators and looking for them in a list. Most reviewing systems allow for this to be automated. </div>
<div>
<br /></div>
<div>
<b>But how am I supposed to know if the proof is correct if I don't know who the authors are. </b></div>
<div>
<b><br /></b></div>
<div>
Most theory conferences are now comfortable with asking for full proofs. And if the authors don't provide full proofs, and I need to know the authors to determine if the result is believable, isn't that the very definition of bias? </div>
<div>
<br /></div>
<div>
And finally, from the business meeting....</div>
<div>
<br /></div>
<div>
Cliff Stein did an excellent job running the discussion on this topic, and I want to thank him for facilitating what could have been, but wasn't, a very fraught discussion. He's treading carefully, but forward, and that's great. I was also quite happy to see that in the straw poll, there was significant willingness for trying double blind review (more than the ones opposed). There were still way more abstentions, so I think the community is still thinking through what this might mean.<br /><div>
<br /></div>
<b><br /></b></div>
Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-12888832026635820652018-01-07T13:54:00.000-07:002018-01-07T13:54:24.304-07:00Report on double blind reviewing in ALENEX 2018<a class="g-profile" href="https://plus.google.com/116794393152584688705" target="_blank">+Rasmus Pagh</a> and I chaired <a href="http://www.siam.org/meetings/alenex18/">ALENEX 2018</a>, and we decided to experiment with double blind review for the conference. What follows is a report that we wrote on our experiences doing this. There are some useful notes about logistics, especially in the context of a theoretically-organized conference on experimental algorithms.<br />
<div>
<br /></div>
<div>
<div style="text-align: center;">
<b><u>ALENEX 2018 Double Blind Review</u></b></div>
<span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"><br /></span>
<span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">For ALENEX 2018, we decided to experiment with a double blind review process i.e one in which authors </span><span style="font-family: "georgia"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">and</span><span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"> reviewers were unaware of each others’ identity. While double blind review is now almost standard in most computer science conferences, it is still relatively uncommon in conferences that focus on theoretical computer science and related topics. </span><br />
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 16pt; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">The motivation</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">In the original argument we presented to the ALENEX Steering Committee, we presented the following reasons for why we wanted double blind review:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">1. </span><span style="color: #323333; font-family: "georgia"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Eliminating bias.</span><span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"> </span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-indent: 36pt;">
<span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">Andrew Tomkins did an experiment for WSDM this year and wrote a report on it: </span><a href="https://arxiv.org/abs/1702.00502" style="text-decoration: none;"><span style="color: #1155cc; font-family: "georgia"; font-size: 11pt; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">https://arxiv.org/abs/1702.00502</span></a></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-indent: 36pt;">
<span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">One particular observation:</span></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-indent: 36pt;">
<span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">"Reviewers in the single-blind condition typically bid for 22% fewer papers, and preferentially bid for papers from top institutions. Once papers were allocated to reviewers, single-blind reviewers were significantly more likely than their double-blind counterparts to recommend for acceptance papers from famous authors and top institutions. The estimated odds multipliers are 1.66 for famous authors and 1.61 and 2.10 for top universities and companies respectively, so the result is tangible”</span></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">2. </span><span style="color: #323333; font-family: "georgia"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Common practice</span><span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">.</span></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">Virtually every CS community except theory is doing double blind review, including most of ML (NIPS, ICML), DB (VLDB, SIGMOD), Systems (NSDI), etc. While theory papers have their own idiosyncrasies, we argued that ALENEX is much closer in spirit and paper structure to more experimental venues like the ones listed.</span></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">3. </span><span style="color: #323333; font-family: "georgia"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Limited burden on authors and reviewers for an experiment</span></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">There was no real logistical burden. We were not blocking people from posting on the arXiv, or talking about their work. We’re merely requiring submissions be blinded (which is easy to do). For reviewers also, this is not a problem - typically you merely declare conflicts based on domains and that takes care of the problem of figuring out who’s conflicted with what paper (</span><span style="color: #323333; font-family: "georgia"; font-size: 11pt; font-style: italic; vertical-align: baseline; white-space: pre-wrap;">but more on this later)</span><span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">.</span></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">4. </span><span style="color: #323333; font-family: "georgia"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Prototyping</span><span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">.</span></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">While theoryCS conferences in general do not make use of double blind review, ALENEX is a small but core venue where such an experiment might reveal useful insights about the viability of double blind overall. So we don’t have to advocate changes at SODA/STOC/FOCS straight up without first learning how it might work.</span></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">5. </span><span style="color: #323333; font-family: "georgia"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">PC submissions</span><span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">.</span></div>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="color: #323333; font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">We are allowing PC members to submit papers, and this has been done before at ALENEX. In this case double blind review is important to prevent even the appearance of conflict.</span></div>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 16pt; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">The process</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Before submission: </span><span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">We provided a submission template for authors that suppressed author names. We also instructed authors on how to refer to prior work or other citations that might leak author identity - in brief, they were asked to treat these as any third-party reference. We also asked authors to declare conflicts with PC members. </span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">After submission/before reviews: </span><span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">We recognized that authors might not be used to submitting articles in double blind mode and so went over each submission after it was submitted and before we opened up bidding to PC members to make sure that the submissions were properly blinded. In a few cases (less than 10/49) we had to ask authors to make modifications. </span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 11pt; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">During review process: </span><span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">The next step was to handle requests for subreviewers. Since PC members could not determine CoIs (conflicts of interest) on their own, all such requests were processed through the PC chairs. A PC member would give us a list of names and we would pick one. (so more private information retrieval than a zero knowledge protocol!)</span></div>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 16pt; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Issues</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">A number of issues came up that appear to be unique to the theory conference review process. We document them here along with suggested mitigations. </span></div>
<ol style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="font-family: Georgia; font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; font-weight: 700; list-style-type: decimal; vertical-align: baseline; white-space: pre;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 18pt;">
<span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">Managing the CoI process: </span><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">In theoryCS conferences, subreviewing happens outside the umbrella of the PC. PC members have the power to request any number of subreviewers for papers, and this process happens </span><span style="font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">after the papers are submitted. </span><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">In contrast, in other venues, subreviewers essentially function as members of the PC - they are invited to be reviewers ahead of time, and are listed when the author declare conflicts of interest. This means that under the process we used, PC members cannot determine for themselves whether a subreviewer has a CoI with a paper, whereas in the alternate process, this is taken care of automatically. One possible mitigation is to ask PC members to list potential reviewers ahead of time and have them registered in the system for authors to declare CoI with. While this might generate a long list of subreviewers for authors to review, this process is customarily handled by a) allowing authors to declare conflicts by affiliation (domain name) and then b) presenting them with a filtered set of reviewers to mark conflicts with. Domain-based filtering is probably the most effective method for handling conflicts based on current or recent affiliation: it allows for reviewers to be added after the fact, and systems like Microsoft’s CMT allow for it. </span></div>
</li>
<li dir="ltr" style="font-family: Georgia; font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; font-weight: 700; list-style-type: decimal; vertical-align: baseline; white-space: pre;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">The difficulty of hiding identity based on prior work: </span><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">In experimental work, a group will often write a series of papers that builds on infrastructure that they’ve developed. The relative difficulty of building such infrastructure also means that groups become “known” for working in certain areas. This made it a little difficult for authors to honestly blind their identity, because their papers clearly built on software that wasn’t publicly available and therefore had to be part of their group. The solution of blinding that reference itself does not always work because then it is hard to evaluate the quality of the work described in the paper. We note that this problem occurs in other, more experimental parts of CS. The typical solution is to continue with the blinding effort in any case, and make an effort to release code publicly, so anyone could have used the code being built on. In our view, this is a less significant problem than the first point. To this end, here are some guidelines from </span><a href="https://chi2016.acm.org/wp/chi-anonymization-policy/" style="text-decoration: none;"><span style="color: #1155cc; font-size: 11pt; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">CHI</span></a><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;"> and </span><a href="http://cscw.acm.org/2018/submit.html#Call%20for%20Papers" style="text-decoration: none;"><span style="color: #1155cc; font-size: 11pt; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">CSCW</span></a><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;"> (both ACM conferences). </span></div>
</li>
<li dir="ltr" style="font-family: Georgia; font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; font-weight: 700; list-style-type: decimal; vertical-align: baseline; white-space: pre;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 0pt;">
<span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">Is the paper provably double blind? </span><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">A common complaint about double blind review is that it is not perfect -- that it’s possible with some effort to determine some subset of the authors with some degree of certainty. The response that we gave when asked, and that is usually given, is that the goal of double blind review is not to provide a zero knowledge protocol, but to prevent the immediate implicit bias that comes from directly seeing the author names prior to reading the paper. We note that this is a common complaint from people in the theory community: however our experience with double blind review in other venues has been that after a while, one gets accustomed to reviewing papers without attempting to first determine the authors and the process works as intended. </span></div>
</li>
</ol>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 16pt; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">Feedback</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">We also solicited feedback from the program committee after the review process was complete. We asked them three questions:</span></div>
<ol style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="font-family: Georgia; font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; list-style-type: decimal; vertical-align: baseline; white-space: pre;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 18pt;">
<span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">What did you like (and what worked) about the double blind review process instituted this year for ALENEX?</span></div>
</li>
<li dir="ltr" style="font-family: Georgia; font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; list-style-type: decimal; vertical-align: baseline; white-space: pre;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">What in your opinion did NOT work? </span></div>
</li>
<li dir="ltr" style="font-family: Georgia; font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; list-style-type: decimal; vertical-align: baseline; white-space: pre;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 0pt;">
<span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-ligatures: normal; font-variant-position: normal; vertical-align: baseline; white-space: pre-wrap;">Is there any other feedback you'd like to provide about the double blind review process? </span></div>
</li>
</ol>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">The responses we got for question 1 were uniformly of the form of “I’m glad that we did it, and felt that papers got a fairer shake than they would have otherwise”. One PC member even said unequivocally that the double blind review process made it more likely that they would submit to ALENEX in the future. </span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">On question 2, PC members brought up the issues we raised above, recommending that we make it clearer to authors how they need to blind their submissions and also mentioning the difficulty of assigning subreviewers. </span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">On question 3, the feedback was uniformly positive in favor of continuing with double blind review, inspite of the issues raised. </span></div>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 16pt; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Summary</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">On balance, our experience with double blind review was positive. While there were logistical issues, many of these can be resolved by the methods we describe above. Further, there is now a wealth of knowledge accumulated in other areas of computer science that we can learn from. </span><a href="https://popl18.sigplan.org/track/POPL-2018-papers#Submission-and-Reviewing-FAQ" style="text-decoration: none;"><span style="color: #1155cc; font-family: "georgia"; font-size: 11pt; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">SIGPLAN has built a comprehensive FAQ</span></a><span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"> around this issue that answers many of the questions that arise. </span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="font-family: "georgia"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;">We recommend continuing with double blind review for at least the next two years at ALENEX, firstly because this brings us in line with common practice in most other parts of computer science, and secondly because many of the logistical issues people face with DB review will go away with habituation. At that point, the potential inconvenience of the process reduces and will be outweighed by the benefits in transparency and lack of bias. </span></div>
<br />
<br /></div>
Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-70880950641996003932017-10-22T23:25:00.000-06:002017-10-22T23:49:41.985-06:00Cake cutting algorithms in prisonThis past Friday, I gave a lecture on cake cutting algorithms at the Timpanogos Women's Facility as part of a lecture series organized by my Utah colleague Erin L. Castro and her <a href="https://www.facebook.com/utahpep/">Utah Prison Education Project</a>. The project's mission is to<br />
<blockquote class="tr_bq">
... provide quality, sustained, and meaningful higher educational opportunities to individuals incarcerated in Utah state prisons. Through embodying the mission of the University of Utah, the Project assists incarcerated students and non-incarcerated volunteers to live lives of impact, both in prison and post-incarceration, by fostering leadership, civic engagement, and critical inquiry. UPEP aims to create lasting impact in our state and our communities by investing in people and providing them the tools necessary for empowerment and lifelong learning.</blockquote>
I think this is incredibly important work. We don't need to get into a much larger discussion about rehabilitation versus punishment theories of justice to appreciate how providing access to education might allow incarcerated students the ability to turn their life around, or even find opportunities for work once they leave prison so that they have a way to support themselves without falling back into criminal activities. Maybe the amount of education they get in prison might even one day be a predictive factor in <a href="https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing">deciding whether they will reoffend</a>!<br />
<br />
When I first mentioned this on Facebook, many people were curious about what it was like. I can report here that my lecture was.... more or less exactly like a lecture would happen in any of the other places I lecture. Students came in with a lot of fear of math (which is why I thought I'd talk about recreational math). There was an actual cake and lots of nervousness when I had students run the algorithms on the cake. There were lots of questions about the different models of fair division, and some confusion about whether we could trust the results of the process.<br />
<br />
In other words, every kind of question one might expect in any setting. The students were engaged and interested. They hadn't had too much math experience except what they did in high school, but they were able to follow along quite well and come up with their own algorithms as we advanced further into the lecture. I enjoyed myself, and I hope they did too!<br />
<br />
But of course this was at a prison, so there were some other details.<br />
<br />
<ul>
<li>Arranging the cake (and a whiteboard) took some work, and Erin and her group (as well as the lieutenant at the prison) did their magic to make it happen. There was some discussion about who would use the plastic butter knife: eventually the students did. </li>
<li>I couldn't bring my laptop in, and used a whiteboard (which worked fine for my talk). But I hadn't planned on using one, and maybe if I did want to do a slide presentation that would have been arranged as well. </li>
<li>There was some discussion about where the students would sit, and how much spacing was needed. There was even some minor discussion about whether we could all get together for a group photo at the end (we did!). </li>
</ul>
<div>
<br /></div>
Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-20075007696967777722017-09-29T09:00:00.000-06:002017-09-29T09:00:34.916-06:00On music, mathematics and teaching. I'm a perpetual student when it comes to my guitar-playing. I started off learning acoustic guitar, and taught myself a little bass in college. When I was in the college band our music advisor played some classical guitar and that got me hooked.<br />
<br />
I've had a number of teachers through grad school and beyond, but I've always plateaued out at a level where I'm competent but no better. At some point I realized that what motivated me to play was the right kinds of music (this I also learnt when watching my children learn an instrument), and that inexorably led me to my new quest: learning flamenco guitar.<br />
<br />
Flamenco is a very passionate style of playing - and classical guitar can seem bloodless and sedate in comparison. It also requires many different right hand techniques that are not common in classical guitar problem.<br />
<br />
The net result is that I'm back to being a beginning student again - struggling with mechanics, hand position and note playing. It's a lot of frustration with the occasional moment of transcendence. I whine at my teacher in the way students whine at me, and he's sneaky enough that now he just asks me "so what would you tell your own students" and that shuts me up.<br />
<br />
Which brings me to the point of this post (<i>what??? posts need a point?). </i>We spent a lesson last week talking about extracting expression and feeling from the instrument. I kept asking him about what tools I could use (beyond the usual tone control by moving up and down the fretboard and using volume) to express more emotion, and what emotion that would be. His response was first to show me this beautiful video of an interviewer "talking" to Paco De Lucia's guitar<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/KLtlM5r8rdc/0.jpg" src="https://www.youtube.com/embed/KLtlM5r8rdc?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
and then explain to me that I have to dig deep within myself to find the way I can relate to the music. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
And then it hit me (painfully). <a href="http://www.cs.utah.edu/~bhaskara/">Aditya Bhaskara</a> and I are running a theory seminar on reading classic theory papers where (<a href="http://blog.geomblog.org/2016/01/reading-with-purpose-grand-experiment.html">much like my previous seminar</a>) there's a strong emphasis on getting to the core ideas and intuitions that drive a result. I'm constantly exhorting students (even more so than Aditya - I think it's interesting to see how different people absorb messages from a paper) to find the core intuition in the paper and be able to express it in a short "story" almost. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
And that's essentially what my teacher is exhorting me to do. In both cases, the expert is trying to get the student to transcend the purely mechanical aspects of (reading the paper/playing the instrument) and get to the deeper (mathematical/emotional) truth of the (paper/piece). And it's hard precisely because the student in either case is still struggling with the mechanical, and doesn't yet have the facility with the tools to let them fall away. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Does this mean I'll be a more enlightened teacher? I doubt it :). But I do have a little more sympathy for my students. </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<br />Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-5525549603318539432017-09-29T00:02:00.000-06:002017-09-29T00:02:06.205-06:00"X is a social construct" and the perils of mining behavior.After the infamous Google memo (and frankly for much longer if you work in algorithmic fairness), the idea of something being a "social construct" has popped up again, and I will admit that I've struggled with trying to understand what that means (damn you, focused engineering education!)<br />
<br />
<a href="https://www.theatlantic.com/national/archive/2013/05/what-we-mean-when-we-say-race-is-a-social-construct/275872/">Ta-Nehisi Coates' article about race</a> is a short and excellent read. But I also want to highlight something much closer to home. BYU Radio's Julie Rose did an interview with Jacqueline Chen (at the U) on her recent work on perceptions of race in the US vs Brazil.<br />
<br />
<a href="https://www.byuradio.org/episode/ae082c7f-2298-4e09-a28e-1688ee5202ab?playhead=1201&autoplay=true">The interview is here</a> (and it's short - starting at around 20 minutes in) and in it Prof. Chen very masterfully lays out the way in which race is perceived and how it changes based on changes in context. The interview is based on a <a href="http://journals.sagepub.com/doi/abs/10.1177/1948550617725149">recently published paper</a> ($$).<br />
<br />
One important takeaway: the way in which one's racial identity is perceived varies greatly between the US (which appears to be influenced by parental information) vs Brazil (where skin color appears to be the dominant factor). More importantly, the idea of race as immutable vs changeable, a categorical attribute versus a continuous one, all vary.<br />
<br />
And that's what we mean by saying that X (here, race) is a social construct. It's not saying that it's fictitious or less tangible. But that it's defined by the way we talk about it in society.<br />
<br />
Why is this important? When we collect data as a way to predict behavior, we're making an implicit claim that behavior can be predicted (and explained) by intrinsic and often immutable descriptors of an individual. We use (or don't use) "race" as a feature when building models.<br />
<br />
But this itself is a huge assumption! It assumes that we can intelligently ascribe features to individuals that capture these notions, and that they are defined solely by the individual and not by context. <a href="https://medium.com/@blaisea/physiognomys-new-clothes-f2d4b59fdd6a">The brilliant Medium article about the paper that claimed to predict criminality from facial features</a> makes this point very well.<br />
<br />
But do we capture the entire history of environmental factors that make up the story of an individual. Of course not. We essentialize an individual into a collection of features that we decide captures all their relevant traits for the purpose of prediction, and then we build a model that rests on this extremely problematic idea.<br />
<br />
Much of the work I do on fairness can be reduced to "check your data, and check your algorithm". What we're also thinking about (and that directly speaks to this issue) is "check your features".<br />
<br />
It turns out that way back in 1921, Walter Lippman had something interesting to say about all of this. <a href="https://www.thinkpragati.com/history/2376/world-outside-pictures-heads/">From a longer essay that he wrote</a> on the importance of frames as mediating how we perceive the world (and it says something about fake news and "true facts" as well):<br />
<blockquote class="tr_bq">
And so before we involve ourselves in the jungle of obscurities about the innate differences of men, we shall do well to fix our attention upon the extraordinary differences in what men know of the world. I do not doubt that there are important biological differences. Since man is an animal it would be strange if there were not. But as rational beings it is worse than shallow to generalize at all about comparative behavior until there is a measurable similarity between the environments to which behavior is a response.<br /><br /><br /></blockquote>
<br />
<br />
<br />Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0tag:blogger.com,1999:blog-6555947.post-51799446114465814012017-08-23T09:00:00.000-06:002017-08-23T09:00:00.176-06:00On free speech, gerrymandering and self-contradictory rule systemsIn the light of the wave of racist and neo-Nazi bile being slung around in Charlottesville and beyond, Karl Popper's <a href="https://en.wikipedia.org/wiki/Paradox_of_tolerance">Paradox of Tolerance</a> has been doing the rounds. Paraphrased, it can be phrased as<br />
<br />
<blockquote class="tr_bq">
In a tolerant society, one must be tolerant of everything except intolerance. </blockquote>
There's an interesting self-referential element there that's reminiscient of <a href="https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems">Gödel's incompleteness theorems</a>. To wit,<br />
<br />
We can either have a system of rules that is consistent, but cannot account for all phenomena that are consistent with the rules, or have a system that is inconsistent.<br />
<br />
In other words, the principle of tolerance in society cannot incorporate its own negation and still survive. One nuance here is that Popper doesn't necessarily advocate for restricting intolerant <i>speech</i> as much as speech that leads to incitement, which is somewhat in line with 1st Amendment protections in the US.<br />
<br />
This reminds me of another situation where self-contradictory rule systems cause problems: <a href="https://en.wikipedia.org/wiki/Gerrymandering">gerrymandering</a>.<br />
<br />
The Supreme Court is soon to hear arguments in a <a href="http://www.scotusblog.com/case-files/cases/gill-v-whitford/">case of partisan gerrymandering</a> from Wisconsin. Roughly speaking, a partisan gerrymander (as opposed to a racial gerrymander) is one in which districts are drawn to favor a certain political party (the "partisan" aspect") as opposed to favoring a certain race.<br />
<br />
While racial gerrymanders have been repeatedly struck down in court (the latest being a case from Texas), partisan gerrymanders have a much more mixed record, which is why many are watching this new case with great interest.<br />
<br />
One potential argument in favor of allowing partisan gerrymandering is that if a party wins, their victory should allow them the power to redraw districts -- the "elections have consequences" principle. But it seems to me that this is again an example of Popper's paradox of tolerance.<br />
<br />
That is to say, if we allow the party that wins an election to do partisan gerrymandering, then we are allowing through the democratic process an action that will serve to directly <i>reduce</i> the ability of the democractic process to represent the people. And for that reason a partisan gerrymander should not be allowed.<br />
<br />
I wonder if there are other settings where this principle might help clarify the limits of a permissive rule structure.Suresh Venkatasubramanianhttp://www.blogger.com/profile/15898357513326041822noreply@blogger.com0