Abstract In this work, we propose a novel adaptive spatiallyregularized correlation fifilters (ASRCF) model to simultaneously optimize the fifilter coeffificients and the spatial regularization weight. First, this adaptive spatial regularization scheme could learn an effective spatial weight for a specifific object and its appearance variations, and therefore result in more reliable fifilter coeffificients during the tracking process. Second, our ASRCF model can be effectively optimized based on the alternating direction method of multipliers, where each subproblem has the closed-from solution. Third, our tracker applies two kinds of CF models to estimate the location and scale respectively. The location CF model exploits ensembles of shallow and deep features to determine the optimal position accurately. The scale CF model works on multi-scale shallow features to estimate the optimal scale effificiently. Extensive experiments on fifive recent benchmarks show that our tracker performs favorably against many state-of-the-art algorithms, with real-time performance of 28fps.