Relative value perception in an insect: positive and negative incentive contrasts in ants

Humans tend to value things not on their absolute values, but relative to reference points such as former experience or expectations. People rate the quality of a new salary relative to their previous salary and the salaries of their peers, instead of appreciating its absolute value. Here, we demonstrate a similar effect in an insect: ants, which had previously experienced a low quality food source, showed higher acceptance of medium quality food (e.g. 0.1M then 0.5M; positive contrast) than if they had received the medium food all along (e.g. 0.5M then 0.5M; control), and vice versa for high expectations. Further experiments demonstrate that these contrast effects arise from cognitive rather than mere sensory or pre-cognitive perceptual causes. Pheromone deposition also correlates with perceived reward value, and ants showed successive contrasts in their pheromone deposition. Relative value perception can therefore be expected to have strong effects not only on individual behaviour, but also on collective decision-making. Contrast effects were also social: the quality of food received from other ants affected the perceived value of food found later. Value judgement is a key element in decision making, and thus relative value perception will strongly influence how animals interact with their environment.


33
We all compare options when making both large and small decisions, ranging from career choice to 34 the choice of an evening's entertainment. Understanding how options are compared has thus been 35 central to the study of behaviour and economics. Theories explaining the mechanisms by which 36 options are compared and decisions are made have a long tradition (Vlaev et al. 2011), with Expected 37 Utility Theory (EUT) being the most widely used theory in economic models (Mankiw 2011;von 38 Neumann and Morgenstern 1944). EUT suggests that decisions are made by evaluating and comparing 39 the expected pay-off from each option. A rational decision maker then chooses the option resulting in 40 the best end state (i.e. the option providing the greatest utility) (von Neumann and Morgenstern 1944). 41 However, over the past decades economic research on how humans make decisions has started to 42 shift away from a view of (absolute) utility maximization towards more nuanced notions of relative 43 utility, such as reference-dependent evaluations. Kahneman and Tversky (1979) made a major 44 contribution to this shift by introducing Prospect Theory, suggesting that decision making is not based 45 on absolute outcomes, but rather on relative perceptions of gain and losses. In contrast to EUT, the 46 utility attributed to options being evaluated is determined relative to a reference point, such as the 47 status quo or former experience (Kahneman and Tversky 1979;Parducci 1984; Tversky and Kahneman 48 1992; Ungemach, Stewart, and Reimers 2011; Vlaev et al. 2011). Various examples of relative value 49 perception have been described. For example, satisfaction gained from income is perceived not 50 absolutely, but relative to the income of others in the social reference group -such as one's colleagues 51 (Boyce, Brown, and Moore 2010). Overall, Prospect Theory has enriched our understanding of human 52 decision making by conceptualizing it as more nuanced than previously assumed (Tversky and 53 Kahneman 1974Kahneman , 1981. 54 A similar relativistic pattern can be found in sensory judgements: Humans rated drinks containing the 55 same sucrose concentration sweeter when they were presented with a range of lower concentrations 56 and less sweet when higher concentrations were presented more frequently (McBride 1982;Riskey, 57 Parducci, and Beauchamp 1979). However, these findings also match well with predictions from 58 psychophysics, in which the link between a given stimulus strength and it's sensation is studied 59 (Zwislocki 2009). A key psychophysical finding is that identical stimuli are perceived as more or less 60 intense depending on the strength of reference stimuli. 61 The concept of malleable value perception is not just relevant to humans. Value judgments in animals 62 are also influenced by factors apparently independent of the absolute value of options. For example, 63 capuchin monkeys refuse otherwise acceptable pay (cucumber) in exchanges with a human 64 experimenter if they had witnessed a conspecific obtain a more attractive reward (grape) for equal 65 effort (Brosnan and de Waal 2003). Rats, starlings, and ants, like humans, place greater value on things 66 they work harder for (Aw, Vasconcelos, and Kacelnik 2011;Czaczkes, Brandstetter, et al. 2018;Lydall, 67 Gilmour, and Dwyer 2010), and fish and locusts demonstrate state-dependent learning, wherein they 68 show a preference for options experienced when they were in a poor condition (Aw et al. 2009;69 Pompilio, Kacelnik, and Behmer 2006). Roces and Núñez aimed to show that in leaf cutting ants 70 perceived value can be influenced by other ants. Ants recruited to higher quality food sources ran 71 faster, deposited more pheromone, but cut smaller leaf fragments, even if the food source the recruits 72 find is replaced by a standardised food source (Roces 1993;Roces and Núñez 1993). However, in these 73 experiments the absolute value and nature of the reference remains unclear, and indeed pheromone 74 presence may have caused the observed behaviours without influencing the ants' expectations or 75 value perception at all. Critically missing from this body of work is a systematic description of value 76 judgment relative to a reference point. 77 A common way in which value is judged is by either comparing two options to each other or by 78 comparing one option to an option experienced in the past. Thus, the perceived value of an option is 79 likely to depend strongly on the strength of contrast between both options and on whether the new 80 option results in a relative gain or a loss. Such value-distortion by comparison effects have been studied 81 for decades using the successive contrasts paradigm. In such experiments, animals are trained to a 82 quality or quantity of reward which is then suddenly increased (positive incentive contrast) or 83 decreased (negative incentive contrast) ( (Tinklepaugh 1928). 94 However, unlike negative contrast effects, responses to positive successive contrast have rarely been 95 found, even when searched for (Black 1968;Capaldi and Lynch 1967;Bower 1961;Dunham 1968;96 Papini et al. 2001). This may be due to three possible factors, which have the opposite effect of positive 97 contrast and may counterbalance it: ceiling effects, neophobia, and generalization decrement 98 (Annicchiarico et al. 2016;Flaherty 1999). Ceiling effects may occur when the performance of animals 99 receiving a large reward is at or near a physical limit. The absence of positive contrast may then not be 100 generated by behavioural principles, but through an artefact of experimental design (Bower 1961;101 Campbell et al. 1970). Neophobia may manifest itself through the reluctance to eat novel food -even 102 if the food is of higher quality than normal (Flaherty 1999  Contrast effects could potentially arise without differential valuation of options; other mechanisms 121 could also in principle produce these results: contrast effects in sensory tasks could derive from simple 122 psychophysical mechanisms (Fechner 1860;Zwislocki 2009), and thus arise from sensory perceptual 123 mechanisms rather than higher level cognitive processing of value. Sensory judgements are also 124 usually made relative to reference points and through constant comparisons with former stimuli 125 (Helson 1964;Vlaev et al. 2011). The position of the reference point in the range of stimuli may thus 126 bias how the stimulus, and thus the value, of a post-shift reward is perceived (Zwislocki 2009). For 127 example, the sweetness of a sucrose solution may be perceived much stronger when the reference 128 point to which it is compared is low. Sensory satiation may also result in apparent contrast effects: the 129 more sweetness receptors are blocked by a sweet reference solution, the fewer receptors will fire 130 when confronted with a post-shift reward, thus making solutions taste less sweet (Bitterman 1976). A 131 final potential driver of apparent contrast effects is related to the theoretical benefits of such 132 behaviour described above: animals may rationally expect the pre-shift reward to be available in the 133 future again and therefore rationally show lower acceptance towards the post-shift reward, because 134 they are waiting for the pre-shift reward to reoccur. 135 The finding of contrast effects in the honey bee, until now the only invertebrate for which such 136 behaviour was conclusively shown, led to a fourth explanation for contrast effects (Couvillon and 137 Bitterman 1984;Bitterman 1976;Núñez 1966). Bitterman (1976) found that honey bees which were 138 trained to a 40% sucrose solution show many feeding interruptions when experiencing a downshift to 139 20% sucrose. By contrast, bees which were fed on 20% throughout the whole experiment filled their 140 crops immediately. Bees which were shifted from 20% to 40% showed no interruptions at the post-141 shift solution either. Apart from explaining these results as negative contrast effects, Bitterman 142 suggested two alternative hypotheses: sensory saturation (see above) and changes in satiation level. concentrations. However, using an odour training paradigm, Couvillon and Bitterman (1984) found 148 negative contrast effects in honeybees and could rule out the above alternative causes. 149 In this study, we investigate positive and negative contrast effects using the successive contrasts 150 paradigm, and define the first relative value curve in an invertebrate; the ant Lasius niger. We then 151 demonstrate that relative value perception arises from non-rational cognitive effects, rather than 152 rational decision-making, physiological effects, or psychophysical phenomena. Finally, we 153 demonstrate that information flowing into the nest can influence value perception in outgoing foragers. 154 Drosophila fruit flies once a week. Water was available ad libitum. 163

Methods and Results
One sub-colony of 500 individuals was formed from each stock colony, and these eight fixed-size sub-164 colonies were used for our experiments. Sub-colonies were maintained identically to the stock colonies, 165 but did not receive any Drosophila fruit flies to prevent brood production, and were starved four days 166 prior to the experiments in order to achieve a uniform and high motivation for foraging (Mailleux,167 Detrain, and Deneubourg 2006; Josens and Roces 2000). During starvation, water was available ad 168 libitum. Any ants which died or were removed from the sub-colonies were replaced with ants from the 169 original stock colonies. 170 General setup

185
To begin an experiment, the sub-colony was connected to the runway via the drawbridge. 2-4 ants 186 were allowed onto the runway, and the first ant to reach the feeder was marked with a dot of acrylic 187 paint on its gaster. The marked ant was allowed to drink to repletion at the food source, while all other 188 ants were returned to the nest. As the ant drank at the droplet it was given one of three food 189 acceptance scores. Full acceptance (1) was scored when the ant remained in contact with the drop 190 from the moment of contact and did not interrupt drinking within 3 seconds of initial contact. Partial 191 acceptance (0.5) was scored if feeding was interrupted within 3 seconds after the first contact with the 192 food source, but the ant still filled its crop within 10 minutes (as can be seen by the distention of the 193 abdominal tergites). Lastly, rejection (0) was scored if the ant refused to feed at the sucrose solution 194 and either returned to the nest immediately or failed to fill its crop within 10 minutes. 195 When the ant had filled its crop or decided not to feed at the sucrose droplet, it was allowed to return 196 to the nest. Inside the nest, the ant unloaded its crop to its nestmates and was then allowed back onto 197 the runway for another visit. The drawbridge was now used to selectively allow only the marked ant 198 onto the runway. 199 In addition to measuring food acceptance, we also measured pheromone deposition. Individual 200 pheromone deposition behaviour correlates with the (perceived) quality of a food source (Beckers,201 Deneubourg The aim of this of experiment was to test whether Lasius niger foragers value a given absolute sucrose 222 solution concentration relative to a reference point or based on its absolute value. We used a range of 223 12 molarities as reference points in order to describe a value curve. To exclude effects of the 224 researcher's expectations on the data, the data for this experiment were collected blind to treatment 225 (Holman et al. 2015). 226

227
In the first two visits to the apparatus -termed the training visits -the ants' reference point was set by 228 allowing it to feed from a feeder at the end of the runway. The quality of the sucrose solution was 229 varied between ants, with each ant receiving the same quality twice successively. 12 different 230 molarities were used: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1, 1.5

281
To rule out the alternative non-psychological explanations for the contrast effects we described above, 282 we needed to change the expectation of the ants while exposing all ants to identical training regimes. 283 This would provide a reference point for testing relative value perception while keeping sensory 284 saturation, haemolymph-sugar levels, and psychophysical effects the same until the switch occurred. 285 To this end, we trained ants over 8 visits to associate a high sucrose molarity (1.5M) with one scent, 286 and a low molarity (0.25M) with a different scent. Then, in the 9 th testing phase, we used scents to 287 trigger an expectation of either high or low molarity, which was then contrasted with a medium (0.5M) 288 unscented solution. Finally, preference for the high-quality associated odour was tested for using a Y-289 maze. 290 On the outward journey of the 9 th (test) visit, ants walking towards the feeder while exposed to 1.5M 302 sucrose-associated cues deposited more pheromone (median=15, fig. 4D) compared to ants exposed 303 to 0.25M-associated cues (median=2, GLMM: estimate= -1.32, z= -13.51, p<0.001). Moreover, in the 304 learning probe, 87% of ants chose the 1.5M associated arm. This demonstrates that ants formed a 305 robust expectation of food molarity based on the cues learned during training. 306 Ants exposed to 1.5M-associated cues during the 9 th visit showed significantly lower food acceptance 307 towards the unscented 0.5M feeder than ants exposed to 0.25M-associated cues (CLMM: estimate= 308 1.07, z= 2.15, p= 0.03, figure 4B, table S1). Although ants exposed to high molarity associated cues on 309 their outwards journey showed a significantly higher number of pheromone depositions on their 310 return journey than ants confronted with low molarity scent (GLMM: estimate= -1.36, z= -5.

335
An ant (forager) was allowed to feed at an unscented sucrose solution droplet of either 0.16, 0.5 or 336 1.5M at the end of a 60cm long runway. Once the ant had fed and returned to the nest, we observed 337 the number of contacts with other nestmates which occurred until trophallaxis was initiated. When 338 trophallaxis began, we noted the time spent in trophallaxis with the first trophallactic partner. When 339 trophallaxis stopped, the receiving trophallactic partner (receiver) was gently moved from the nest and 340 placed onto the start of a 20cm long runway, offering unscented 0.5M sucrose solution at the end. As 341 the receiver fed, we noted its food acceptance. 342 Indeed, gains seem to be overvalued while losses are undervalued. This may be due to the 367 psychophysics of our study system: a basic tenant of psychophysics is that the Just Noticeable 368 Difference (JNDs) between two stimuli is a function of the relative difference between the stimuli 369 (Fechner 1860;Stevens 1957;Zwislocki 2009). Thus, ants shifted from 0.1M to medium (0.5M) quality 370 experience a 5-fold increase in molarity, while those down-shifted from 0.9M to 0.5M experience less 371 than a two-fold decrease, although the absolute change was of the same magnitude. This would 372 predict larger shift-changes, in terms of absolute molarity change, for gains than for losses. Indeed, the 373 fact that this is also not seen may imply that losses are indeed -relatively speaking -looming larger 374 than gains for the ants. Finally, it must be kept in mind that acceptance scores are unlikely to be linear, 375

Experiment 3 -Results
and that pheromone deposition behaviour shows large variation (Beckers, Deneubourg, and Goss 376 1992), making it difficult to use either of these factors to test for over-and undervaluation of gains and 377 losses. 378 While the results of experiment 1 can be explained using alternative, non-psychological mechanisms 379 (sensory saturation and changes in satiation) or rational behaviour based on future expectations, the 380 results of experiment 2 cannot. Ants which were expecting high molarities after scent training showed 381 lower acceptance scores when confronted with unscented medium quality food than ants which 382 expected to find low quality food ( figure 4B). This is in spite of all ants undergoing identical training 383 experiences. The only difference between the groups was the odour of the runway on the 9 th (test) 384 visit. It is thus unlikely that sensory saturation, increased haemolymph-sugar levels, simple the nest. By taking into account information gained inside the nest, recruited workers will be able to 413 evaluate newly discovered food sources in relation to other food sources available in the environment. 414 They will also be able to make better informed decisions on whether it is worth exploiting a new food 415 source or ignore it. Such a pattern would lead to individual ants being more likely to forego food 416 sources which are of lower quality than the average available food sources and thus allows colonies to 417 only exploit above-average food sources. Ants can also use this information to choose between various 418 information use strategies, such as whether to continue exploiting known food sources or be recruited 419 to follow pheromone trails leading to other food sources (Czaczkes and Beckwith 2018). Ultimately, 420 we see the nest serving as an information hub, in which information about currently available food 421 sources can be collected, synthesised, and fed back to outgoing foragers. Relative value perception can 422 therefore be expected to have strong effects not only on the individual behaviour of animals, but also 423 on the collective behaviour of insect colonies, potentially allowing colonies to ignore usually acceptable 424 options in favour of better ones 425 A broad range of behaviours relevant to behavioural economics have now been described in 426 invertebrates. These include overvaluing rewards in which more effort was invested (Czaczkes et al.