Abstract

A linear restriction of a function is the same function with its domain restricted to points on a given line. This paper addresses the problem of computing a succinct representation for a linear restriction of a piecewise-linear neural network. This primitive, which we call E XACT L INE, allows us to exactly characterize the result of applying the network to all of the infinitely many points on a line. In particular, E XACT L INE computes a partitioning of the given input line segment such that the network is affine on each partition. We present an efficient algorithm for computing E XACT L INE for networks that use ReLU, MaxPool, batch normalization, fullyconnected, convolutional, and other layers, along with several applications. First, we show how to exactly determine decision boundaries of an ACAS Xu neural network, providing significantly improved confidence in the results compared to prior work that sampled finitely many points in the input space. Next, we demonstrate how to exactly compute integrated gradients, which are commonly used for neural network attributions, allowing us to show that the prior heuristic-based methods had relative errors of 25-45% and show that a better sampling method can achieve higher accuracy with less computation. Finally, we use E XACT L INE to empirically falsify the core assumption behind a well-known hypothesis about adversarial examples, and in the process identify interesting properties of adversarially-trained networks.