There’s Lots in a Name (Whereas There Shouldn’t Be)

with Jingyan Wang.

(based on a folklore meme)

It is common in some academic fields such as theoretical computer science to order the authors of a paper according to the alphabetical order of their last names. Alphabetical ordering is also employed in other contexts like listing of names of people on the web, for instance, to order the participant list and pictures on the ITA conference website.

Although alphabetical ordering mitigates some issues with other ordering approaches (e.g., possible conflicts among authors under contribution-based ordering), it causes its own biases. These biases form the focus of this post.

What are these biases?

A number of papers have empirically studied the effects of the convention of alphabetically-ordered authorship, which reveal biases associated to this convention. Here is an excerpt from the study [1] by Einav and Yariv:

“We begin our analysis with data on faculty in all top 35 U.S. economics departments. Faculty with earlier surname initials are significantly more likely to receive tenure at top ten economics departments, are significantly more likely to become fellows of the Econometric Society, and, to a lesser extent, are more likely to receive the Clark Medal and the Nobel Prize. These statistically significant differences remain the same even after we control for country of origin, ethnicity, religion or departmental fixed effects. All these effects gradually fade as we increase the sample to include our entire set of top 35 departments.

We suspect the ‘alphabetical discrimination’ reported in this paper is linked to the norm in the economics profession prescribing alphabetical ordering of credits on coauthored publications. As a test, we replicate our analysis for faculty in the top 35 U.S. psychology departments, for which coauthorships are not normatively ordered alphabetically. We find no relationship between alphabetical placement and tenure status in psychology.”

Various other studies make similar observations and draw similar conclusions (e.g., see [2], [3] and references therein).

What is the source of these biases?

There are at least two types of bias effects.

Implicit bias – Primacy effects: Primacy effects describe the human cognitive bias that people are more likely to remember and choose items showing up earlier in a list than items later in the list — in short, “first is best” [4]. Primacy effects have been widely studied in psychology, and observed in many laboratory and field settings, e.g., people are more likely to recall words earlier in a list [5]; people are more likely to choose the first candidate on the ballot for an election [6]. In the context of author ordering, primacy effect suggests that authors whose names show up earlier in the author list are likely to receive more attention from the reader.

Explicit bias – “First author et al.”: A more conspicuous bias arises when papers use a “First author et al.” format in its text to refer to other papers. Now, it may be argued that communities which use alphabetical-ordering conventions do not use the “First author et al.” format. So we put this hypothesis to the test. Publication venues in computer science that primarily follow alphabetical orderings include STOC, FOCS and EC. A search on Google Scholar reveals the following number of papers in these conferences which use the “First author et al.” format in their own text:

Conference	#Total papers	#Papers using “First author et al.” in its text
STOC 2017	99	70
STOC 2016	79	59
FOCS 2017	79	48
FOCS 2016	73	43
EC 2017	75	48
EC 2016	99	87

So, what are alternative solutions?

For ordering authors in papers, a contribution-based arrangement is a popular alternative. However, this manner of ordering can cause conflicts between authors regarding their contributions. An alternative is to employ a technique that computer scientists use extensively in their research — randomization! Under such a randomized arrangement, authors could be ordered uniformly at random. Or otherwise the authors could be arranged as a combination of contribution-based and randomized methods, where contributions can determine a partial order and then a total order is selected uniformly at random from among all total orders consistent with the partial order. In this case, symbols or footnotes can be used to distinguish authors whose orders are contribution-based and whose orders are random. See, for instance, the paper [7] for a more detailed discussion on randomized author ordering.

Likewise for lists of names on the web, one could randomize the order whenever feasible. This randomization could be dynamic (a new ordering whenever the page is loaded) or static (permute once and fix the permutation). Now, if we were dealing with listing names in some printed material, searching for any particular individual would have been difficult. But on the browser, one can always use Ctrl/Cmd+F to search.

Updates after publication:

• (Jun 18, 2019) We reached out to the program chairs of ACM EC 2019, Nicole Immorlica and Ramesh Johari. They kindly agreed to change the submission style file with numbered references as default from the “First author et al.” format, and also keep numbered references in the camera ready versions. (Jingyan helped out with the style files).
• (Nov 14, 2019) Taking cognizance of these biases, starting October 24, 2019, the Machine Learning Department at CMU has randomized the ordering of students and faculty on its webpages. One concern was that users may get confused since the standard practice is to order alphabetically. To this end, we put a small bar on top of the page indicating these biases and a link to this post for details. Our webmaster tells us that the user experience has been same as before (along with a lot of positive feedback that this was the right thing to do). Thanks to Robeto Iriondo, Aaditya Ramdas and Roni Rosenfeld!
• (Jul 19, 2020) The CMU Theory group website also uses dynamic ordering now http://theory.cs.cmu.edu/ Thanks to Anupam Gupta!
• (Jun 15, 2021) We had reached out to Virginia Vassilevska Williams, the program chair of STOC 2021. Taking cognizance of these issues, the call for papers for the conference added: “Recommended best practices for citations: Authors are asked to avoid “et al.” in citations in favor of an equal mention of all authors’ surnames (unless the number of authors is very large, and if it is large, consider just using \cite{} with no “et al.”).” Thanks a lot to Virginia and all STOC organizers!
• (Oct 15, 2022) Taking cognizance of these issues, the call for papers for the STOC 2022 conference added: “Randomization of Author Name Ordering: Alphabetical orderings of author names can lead to biases, so authors may consider randomizing author orderings for your papers. Randomization tools include this or this. Randomization of the author order can be indicated using \textcircled{r} instead of a comma as a name delimiter, or using a footnote on the title page using \thanks{}.” Thanks a lot to the program chair Anupam Gupta and all STOC organizers!

References

[1] “What’s in a surname? The effects of surname initials on academic success,” L. Einav and L. Yariv. Journal of Economic Perspectives, 2006.

[2] “The Benefits of Being Economics Professor A (rather than Z),” C. van Praag and B. van Praag. Economica, 2008.

[3] “How Do Journal Quality, Co-Authorship, and Author Order Affect Agricultural Economists’ Salaries?” C. Hilmer and M. Hilmer. American Journal of Agricultural Economics, 2005.

[4] “First Is Best,” D. Carney and M. Banaji. PLOS ONE, 2012.

[5] “The serial position effect of free recall,” B. Murdock. Journal of Experimental Psychology, 1962.

[6] “The impact of candidate name order on election outcomes in North Dakota,” E. Chen, G. Simonovits, J. Krosnick, J. Pasek. Electoral Studies, 2014.

[7] “Certified Random: A New Order for Coauthorship,” D. Ray and A. Robson. American Economic Review, 2018.