Abstract We investigate the emergence and stability of social conventions for ef?ciently resolving con?icts through reinforcement learning. Facilitation of coordination and con?ict resolution is an important issue in multi-agent systems. However, exhibiting coordinated and negotiation activities is computationally expensive. In this paper, we ?rst describe a con?ict situation using a Markov game which is iterated if the agents fail to resolve their con?icts, where the repeated failures result in an inef?cient society. Using this game, we show that social conventions for resolving con?icts emerge, but their stability and social ef?ciency depend on the payoff matrices that characterize the agents. We also examine how unbalanced populations and small heterogeneous agents affect ef?ciency and stability of the resulting conventions. Our results show that (a) a type of indecisive agent that is generous for adverse results leads to unstable societies, and (b) sel?sh agents that have an explicit order of bene?ts make societies stable and ef?cient.