Abstract Feature-to-feature matching is the key issue in the Bag-of-Features model. The baseline approach employs a coarse feature-to-feature matching, namely, two descriptors are assumed to match if they are assigned the same quantization index. However, this Hard Assignment strategy usually incurs undesirable low precision. To fifix it, Multiple Assignment and Soft Assignment are proposed. These two methods reduce the quantization error to some extent, but there are still a lot of room for improvement. To further improve retrieval precision, in this paper, we propose a novel feature matching strategy, called local-restricted Soft Assignment (lrSA), in which a new feature matching function is introduced. The lrSA strategy is evaluated through extensive experiments on fifive benchmark datasets. Experiments show that the results exceed the retrieval performance of current quantization methods on these datasets. Combined with postprocessing steps, we have achieved competitive results compared with the state-of-the-art methods. Overall, our strategy shows notable benefifit for retrieval with large vocabularies and dataset size.