Abstract
Interactive search, where a set of tags is recommended to users together with search results at each
turn, is an effective way to guide users to identify their information need. It is a classical sequential decision problem and the reinforcement
learning based agent can be introduced as a solution. The training of the agent can be divided
into two stages, i.e., offline and online. Existing
reinforcement learning based systems tend to perform the offline training in a supervised way based
on historical labeled data while the online training is performed via reinforcement learning algorithms based on interactions with real users. The
mis-match between online and offline training leads
to a cold-start problem for the online usage of the
agent. To address this issue, we propose to employ
a simulator to mimic the environment for the offline
training of the agent. Users’ profiles are considered
to build a personalized simulator, besides, modelbased approach is used to train the simulator and is
able to use the data efficiently. Experimental results
based on real-world dataset demonstrate the effectiveness of our agent and personalized simulator