Abstract
The PAN series of shared tasks is well known
for its continuous and high quality research
in the field of digital text forensics. Among
others, PAN contributions include original corpora, tailored benchmarks, and standardized
experimentation platforms. In this paper we
review, theoretically and practically, the authorship verification task and conclude that the
underlying experiment design cannot guarantee pushing forward the state of the art—in
fact, it allows for top benchmarking with a surprisingly straightforward approach. In this regard, we present a “Basic and Fairly Flawed”
(BAFF) authorship verifier that is on a par with
the best approaches submitted so far, and that
illustrates sources of bias that should be eliminated. We pinpoint these sources in the evaluation chain and present a refined authorship
corpus as effective countermeasure