Role of delay-based reward in the spatial cooperation

Highlights

• We introduce delay-based reward into spatial cooperation, in which individuals can select various reward amounts on the basis of delay.

• We find that the intermediate differences of reward among kinds of delays are most important contributions to cooperation.

• We show that the individuals who select the fastest and smallest reward in evolving have strongest competition.

Abstract

Strategy selection in games, a typical decision making, usually brings noticeable reward for players which have discounted value if the delay appears. The discounted value is measure: earning sooner with a small reward or later with a delayed larger reward. Here, we investigate effects of delayed rewards on the cooperation in structured population. It is found that delayed reward supports the spreading of cooperation in square lattice, small-world and random networks. In particular, intermediate reward differences between delays impel the highest cooperation level. Interestingly, cooperative individuals with the same delay time steps form clusters to resist the invasion of defects, and cooperative individuals with lowest delay reward survive because they form the largest clusters in the lattice.

Introduction

Game theory that seeks to understand the strategy selection of individuals to maximize the payoffs in specific condition, has attracted more ​attention  [1], [2]. Various researches about evolutionary games have focused on strategy selection mechanisms in repeated games  [3], [4]. Individual within evolutionary games framework, selects the strategy of the next round by referring to the past payoff and opponent’s strategy, such as cooperation or defection. For instance, tit-for-tat [5], [6], [7], [8], [9], win–stay and lose–shift  [10], [11] have been explored. Games in structured population, it is found that spatial topology favors cooperation, where individual interacts with its direct neighbors and gains a payoff  [12], [13], [14], [15], [16]. By means of strategy adoption rules, such as deterministic  [17], [18] and pairwise-comparison  [19], [20], [21], [22], [23], individual adjusts its strategy by comparing the performance of neighbors’ and itself. Some features, including memory  [24], reputation  [25], [26], diversity  [27], [28], [29], [30] have been incorporated into spatial games. In addition, punishment and reward  [31], [32], including antisocial pool reward  [33], rewarding with links  [34] and adaptive rewarding  [35] have also been examined from both experimental and theoretical aspects  [36], [37]. Previous results also show that incorporating of punishment and reward groups into the original game model can produce interesting phase diagram  [38].

Because individuals are driven by payoffs during the decision making process, the emerging of delayed and uncertain rewards may affect their strategies selection  [39], [40], [41], [42]. For example, when an individual is required to choose between a smaller reward available sooner and a larger reward available later, where the decision usually involves reward amount which is so called temporal discounting  [43]. That is, individual discounts the value of reward based on the delay. The general formula of discounting in neurobiological and economic science is exponential, while the alternative form in psychology experiments is a hyperbolic $V=1/(1+Kd)$, where $V$ is the subject value for delay $d$, and $K$ marks the discounting factor  [44].

Players participating in evolutionary games, are also required to make decisions in each round, such as cooperation or defection. Especially, when cooperative players are rewarded on the basis of cooperative rounds. Then, from the perspective of temporal discounting in neurobiological and economic science, incorporating of delay will lead to varied strategy selections  [45], [46], such as cooperating once to obtain the soonest but smallest reward, or largest but delayed reward by cooperating with several rounds. And how this decision making affects the spatial cooperation, especially when individuals can adopt the strategy and reward selection of neighbors simultaneously. To address this problem, we explore the role of delay-based reward for decision making in spatial cooperation. Here, individuals can select either the larger but available later or smaller and sooner reward based on cooperative rounds. We assume individuals select the reward choices and strategies of neighbors at the same time. We show that, delay-based reward spreads of cooperation on lattice, random and small-world networks. The results prove that, temporal discounting impels the intermediate reward variation for different cooperative rounds to be most important contribution to the cooperation. We find that, cooperators who select the fastest and lowest reward have the strongest competition in the system, rather than who choose delayed and larger reward. This paper is organized as follows: Section  2 presents the model, Section  3 demonstrates numerical results, and the discussion is presented in Section  4.