On various aspects of academia, research, and peer review
Collusions are a major problem in peer review: two or more participants make a deal that they will try to get assigned each others’ papers and give good reviews. Many peer-review venues employ bidding, where reviewers indicate their expertise for submitted papers, and papers are assigned to reviewers based on this information. Bidding is thus … Continue reading Randomized Transparency to Mitigate Collusions
Conferences ask for various pieces of information from authors and reviewers. One may wonder why this information is needed: is it actually important or simply a bureaucratic requirement? If authors or reviewers don’t provide this information, program chairs may have to spend considerable time and effort chasing them (which is especially burdensome in large-scale conferences), … Continue reading Why do conferences ask for this information?
A sub-area of computer science, where a large part of my own work lies, is on theoretical topics in human computation. A broad class of questions here involve a set of items, and data in the form of evaluations of these items made by humans. The goal is to infer certain properties about these items … Continue reading Visual Demos for Theoretical Results (in Human Computation)
with Jingyan Wang. In this post, we evaluate the gender distribution of best paper awardees in various top CS conferences. The inspiration to do so comes from an interview of Andrea Goldsmith in the IEEE Information Theory Society (ITSoc) newsletter [1] in which she lays out some interesting statistics: "Going by the names of authors, … Continue reading Gender Distributions of Paper Awards
In computer science, conferences are often the final publication venue for research. However, conference papers are not always reviewed very carefully in terms of objective criteria like correctness, and the focus of the review is often on subjective criteria such as interestingness and perceived impact. Moreover, sub-fields such as Machine Learning (and AI) have recently … Continue reading Double Decker Peer Review
with Jingyan Wang. (based on a folklore meme) It is common in some academic fields such as theoretical computer science to order the authors of a paper according to the alphabetical order of their last names. Alphabetical ordering is also employed in other contexts like listing of names of people on the web, for instance, … Continue reading There’s Lots in a Name (Whereas There Shouldn’t Be)
Research On Research 