Unsupervised Information Extraction: Regularizing DiscriminativeApproaches with Relation Distribution Losses
Abstract
Unsupervised relation extraction aims at extracting relations between entities in text. Previous unsupervised approaches are either generative or discriminative. In a supervised
setting, discriminative approaches, such as
deep neural network classifiers, have demonstrated substantial improvement. However,
these models are hard to train without supervision, and the currently proposed solutions are
unstable. To overcome this limitation, we introduce a skewness loss which encourages the
classifier to predict a relation with confidence
given a sentence, and a distribution distance
loss enforcing that all relations are predicted
in average. These losses improve the performance of discriminative based models, and enable us to train deep neural networks satisfactorily, surpassing current state of the art on
three different datasets.