Effect of update rule transition triggered by Q-learning algorithm in evolutionary prisoner's dilemma game involving extortion

¹ School of Information Science and Engineering, Hebei University of Science and Technology Shijiazhuang 050018, PRC
² Institute of Translational Medicine, Medical College Yangzhou University - Yangzhou, 225001, PRC

^(a) E-mail: mzjinlong@163.com (corresponding author)
^(b) E-mail: cailiangliang765@163.com (corresponding author)

Received: 12 May 2023
Accepted: 4 July 2023

Abstract

Most studies have shown that the heterogeneity of update rules has an important impact on evolutionary game dynamics. In the meanwhile, Q-learning algorithm has gained attention and extensive study in evolutionary games. Therefore, a mixed stochastic evolutionary game dynamic model involving extortion strategy is constructed by combining imitation and aspiration-driven updating rules. During the evolution of the model, individuals will use the Q-learning algorithm which is a typical self-reinforcement learning algorithm to determine which update rule to adopt. Herein, through numerical simulation analyses, it is found that the mixed stochastic evolutionary game dynamic model affected by the Q-learning algorithm ensures the survival of cooperators in the grid network. Moreover, the cooperators cannot form a cooperation cluster in the grid network but will form a chessboard-like distribution with extortioners to protect cooperators from the invasion of defectors. In addition, a series of results show that, before the evolution turns into steady state, our model increases the number of nodes utilizing the average aspiration-driven update rule, thereby promoting the emergence of chessboard-like distribution. Overall, our study may provide some interesting insights into the development of cooperative behavior in the real world.