资源论文Stochastic Multi-Armed-Bandit Problem with Non-stationary Rewards

Stochastic Multi-Armed-Bandit Problem with Non-stationary Rewards

2020-01-19 | |  109 |   69 |   0

Abstract

In a multi-armed bandit (MAB) problem a gambler needs to choose at each round of play one of K arms, each characterized by an unknown reward distribution. Reward realizations are only observed when an arm is selected, and the gambler’s objective is to maximize his cumulative expected earnings over some given horizon of play T . To do this, the gambler needs to acquire information about arms (exploration) while simultaneously optimizing immediate rewards (exploitation); the price paid due to this trade off is often referred to as the regret, and the main question is how small can this price be as a function of the horizon length T . This problem has been studied extensively when the reward distributions do not change over time; an assumption that supports a sharp characterization of the regret, yet is often violated in practical settings. In this paper, we focus on a MAB formulation which allows for a broad range of temporal uncertainties in the rewards, while still maintaining mathematical tractability. We fully characterize the (regret) complexity of this class of MAB problems by establishing a direct link between the extent of allowable reward “variation” and the minimal achievable regret, and by establishing a connection between the adversarial and the stochastic MAB frameworks.

上一篇:Fast Training of Pose Detectors in the Fourier Domain

下一篇:Decomposing Parameter Estimation Problems

用户评价
全部评价

热门资源

  • A Mathematical Mo...

    Direct democracy, where each voter casts one vo...

  • Learning to Predi...

    Much of model-based reinforcement learning invo...

  • The Variational S...

    Unlike traditional images which do not offer in...

  • Hierarchical Task...

    We extend hierarchical task network planning wi...

  • Shape-based Autom...

    We present an algorithm for automatic detection...