Abstract
We consider peer review under a conference setting
where there are conflicts between the reviewers and
the submissions. Under such conflicts, reviewers
can manipulate their reviews in a strategic manner
to influence the final rankings of their own papers.
Present-day peer-review systems are not designed
to guard against such strategic behavior, beyond
minimal (and insufficient) checks such as not assigning a paper to a conflicted reviewer. In this
work, we address this problem through the lens of
social choice, and present a theoretical framework
for strategyproof and efficient peer review. Given
the conflict graph which satisfies a simple property,
we first present and analyze a flexible framework
for reviewer-assignment and aggregation for the reviews that guarantees not only strategyproofness but
also a natural efficiency property (unanimity). Our
framework is based on the so-called partitioning
method, and can be treated as a generalization of
this type of method to conference peer review settings. We then empirically show that the requisite
property on the (authorship) conflict graph is indeed satisfied in the ICLR-17 submissions data, and
further demonstrate a simple trick to make the partitioning method more practically appealing under
conference peer-review settings. Finally, we complement our positive results with negative theoretical
results where we prove that under slightly stronger
requirements, it is impossible for any algorithm to
be both strategyproof and efficient