Comparative inspiration: From puzzles with pigeons to novel discoveries with humans in risky choice

Both humans and non-human animals regularly encounter decisions involving risk and uncertainty. This paper provides an overview of our research program examining risky decisions in which the odds and outcomes are learned through experience in people and pigeons. We summarize the results of 15 experiments across 8 pub- lications, with a total of over 1300 participants. We highlight 4 key ﬁ ndings from this research: (1) people choose di ﬀ erently when the odds and outcomes are learned through experience compared to when they are described; (2) when making decisions from experience, people overweight values at or near the ends of the distribution of experienced values (i.e., the best and the worst, termed the “ extreme-outcome rule ” ), which leads to more risk seeking for relative gains than for relative losses; (3) people show biases in self-reported memory whereby they are more likely to report an extreme outcome than an equally-often experienced non-extreme outcome, and they judge these extreme outcomes as having occurred more often; and (4) under certain cir-cumstances pigeons show similar patterns of risky choice as humans, but the underlying processes may not be identical. This line of research has stimulated other research in the ﬁ eld of judgement and decision making, illustrating how investigations from a comparative perspective can lead in surprising directions.


Introduction
Humans are typically more risk seeking for losses than gains, and this difference holds even when identical choices are framed as gains and losses (e.g., Kahneman and Tversky, 1979;Tversky and Kahneman, 1981). Our line of research began by examining whether this classic result from behavioural economics would also hold in pigeons, as had been found with starlings (Marsh and Kacelnik, 2002) and capuchin monkeys (Chen et al., 2006). Building from these findings led us to 'rediscover' the description-experience gap (Ludvig and Spetch, 2011), whereby people make different risky choices when the odds and outcomes are explicitly described vs. when those odds and outcomes are learned from experience (Hertwig and Erev, 2009). Since then, our journey has taken turns in other directions as we have sought to clarify how past experiences influence future decisions, and nearly all of our published work on this topic has been done in humans. Nevertheless, this line of research has comparative cognition at the heart.
Consider the following scenario: Would you rather win $20 for sure, or take a gamble with a 50% chance of winning $40 and a 50% chance of winning nothing? Most people here would choose the guaranteed win. When the same question is cast as losses, i.e., a guaranteed loss of $20 or a 50% chance of losing $40, most people instead would choose the gamble (e.g., Kahneman and Tversky, 1979). Now, how can you 'ask' a pigeon the same questions? Fig. 1A shows how, with people, odds and outcomes in these risky decisions are typically conveyed by means of language or visuals, such as a pie chart. Some studies with non-human animals, such as with monkeys Hayden, 2013, 2016), have been able to convey described odds using visual stimuli. Another approach is to instead convey odds and outcomes over successive trials using an operant procedure and have the animal, or human, learn the contingencies from their own experience. Fig. 1B shows this alternate approach, when the decision problem is posed through experienced odds and outcomes, rather than through described ones. This choice procedure, involving pairs of door pictures, was used in all of our published studies of decisions from experience in humans.
Studying decisions based on learned contingencies has a long history in operant conditioning research (e.g., Fantino, 1969;Herrnstein, 1961;Lea, 1979;Staddon and Motheral, 1978) and reflects the way animals make choices in nature, but this approach is quite different from the way decision making is often studied in humans. Indeed, the https://doi.org/10.1016/j.beproc.2018.12.009 Received 2 October 2018; Received in revised form 10 December 2018; Accepted 11 December 2018 famous studies of Kahneman and Tversky, among others, are based primarily on research that involves asking people to make choices based on explicitly described scenarios. This verbal accessibility may add to the appeal of the research program, as even the readers experience the paradoxes, but often may not represent the types of decision people regularly encounter in life. Moreover, as we will review below, people make different decisions based on descriptions than decisions based on experience, even with the same odds and outcomes.
A few years prior to our initial work, evidence had begun to accumulate showing that risk preferences in humans can change depending on whether the choices are based on description or experience (Barron and Erev, 2003;Hertwig et al., 2004). Specifically, when choosing between risky options that include rare events (i.e., 10% or lower), people overweight the rare events if the decisions are described. When the same decisions are based on repeated experience, however, people choose as though they are underweighting the rare events. For example, given a choice between a 5% chance at $100 and a guaranteed $5, people will generally take the gamble when the problem is described (overweighting the rare win), but take the sure thing when learned from experience (underweighting the rare win).
This difference in the weighting of rare events when making decisions based on described and experienced odds and outcomes has been termed the description-experience gap (Hertwig and Erev, 2009). As alluded to above, in our early work, we inadvertently uncovered another type of description-experience gap that did not involve rare events (Ludvig and Spetch, 2011). As with many advances in science (e.g., Skinner, 1956), this discovery emerged serendipitously: our initial investigations began with an attempt to re-create the framing effects in the human literature in pigeons (i.e., Tversky and Kahneman, 1981). After multiple failed attempts, we directly applied the procedure we were using with pigeons to people, now failing to yield the expected results in humans. This additional failure prompted us to directly pit with people the pigeon-inspired approach (see Fig. 1B) against the verbal approach drawn from the human literature (see Fig. 1A). With this direct comparison of people's risky choices when making decisions from experience or description (Fig. 1A), we found the opposite pattern of choices between these two approaches to conveying risk-related information. Fig. 1C shows how, as expected, people were more risk seeking for losses than for gains in decisions from description, but, contrary to the prevailing findings in the literature, they were more risk seeking for gains than for losses in decisions from experience. This pigeon-inspired approach has become the bases of our numerous subsequent studies with humans.

General procedure
As mentioned above, the decisions from experience that have become the staple of our research on risky choice in humans ( Fig. 1) were inspired by the comparative approach to studying behavior. During the task, people are only told that they should try to maximize their points to earn money, but they are not told what will happen when choosing a particular door. Instead, they learn from repeated trial-and-error experience about the odds and outcomes associated with each door. In all of our studies thus far, risk preferences are assessed in terms of choices between a fixed option that always leads to a specific outcome and a risky option that leads equally often to either a better outcome or a worse outcome; the expected value of the fixed and risky options are equal, and there are no rare events. Typically, the learning set includes two or more pairs of options that differ in value (e.g., fixed and risky gain options and fixed and risky loss options, or fixed and risky highvalue options and fixed and risky low-value options), and choices among these options are intermixed.
Relative to other studies on risky decisions from experience in the judgment and decision making (JDM) literature, our general procedure involves a few novel features, inspired from the animal literature, which are important to consider when comparing experimental designs. First, and of perhaps greatest importance, different decisions, e.g., the gain and loss decisions, are always inter-mixed within the same block of trials. This key procedural factor is critical to our main finding (see below) of greater risk seeking for relative gains than losses. In most other JDM studies, separate decisions, often referred to as 'problems', are presented one-after-another sequentially in blocks (e.g., Hertwig et al., 2004). Along similar lines, in our studies, the side of the screen on which the risky and safe options are presented is always counterbalanced. Both of these procedural details are related to our initial beginnings in the comparative cognition literature, where studies of animals often counterbalance and inter-mix different trial types. As such, this unique perspective and bridging of the JDM and comparative cognition approaches has been critical to our impact within the topics of risky decision-making and gambling.
Another important feature of the tasks is that participants make choices between a safe option and a risky option that can lead equiprobably to two potential outcomes (i.e., 50% chance of each; see Fig. 1), but the safe and risky options always have the same expected value. For example, as shown in Fig. 1, people might choose between a safe option of +20 points and a risky option that yields +40 points 50% of the time and 0 points otherwise; both these options have the same expected value (+20). This equivalence is important as many JDM studies present problems where one option, either risky or safe, has a higher expected value (e.g., Camilleri and Newell, 2011;Hertwig et al., 2004). For instance, people may be presented with a choice between a loss of 3 points with a 100% chance vs. a loss of 32 points with a 10% chance. In these cases, from a reward-maximizing perspective, there is a correct answer. When the safe and risky options have the same expected value, however, choices on these decision trials are a measure of risk preference that are not influenced by differences in reward maximization. Although behavior in such cases does not indicate the extent to which preference for risk would override differences in expected value, the choices made when expected value is equal should be sensitive to even mild variations in risk preference. Our studies, however, do include catch trials that involve a decision between options of different reward values, such as a gain vs. a loss, as a manipulation check to assess whether participants have been paying attention in the experiment and have successfully learned the outcome contingencies.
Finally, to ensure that participants adequately sample the outcomes associated with each option, some trials provide only a single option that has to be chosen. These trials limit participants from only experiencing a small sample of outcomes that inadequately represents the option, i.e., sampling biases. These single-choice trials avoid instances where a risky option is initially unlucky and is then never subsequently chosen, known as the hot-stove effect (Denrell and March, 2001). Relatedly, in most of our studies, feedback is only given for the selected options, termed partial feedback in the JDM literature (e.g., Camilleri and Newell, 2011;Hertwig and Erev, 2009).
More generally, in all of these experiments, risky choices were presented in blocks of trials, separated by a riddle to provide a brief break. Participants were neither told how many trials were included in each block, nor how many blocks comprised the experiment. Experiments typically consisted of approximately 400-600 trials and lasted 35-45 minutes. Some experiments included an honorarium based on task performance (i.e., total points earned), but others did not. When an honorarium was paid, the point-to-money conversion differed based on task procedures (e.g., both gain and loss decisions, all gains, all losses), but was not always told to participants. Nonetheless, the choice effects were robust across these procedural differences (see Figs. 6 and 7 below).

Key findings so far
Over the last few years, we have conducted a series of studies investigating risky decision making, with an emphasis on the role of memory. Here we provide an overview and summary of these studies, focusing on the bigger picture and relationship between the studies, though each individual paper included additional hypotheses and background not discussed here.

Biases in risky choice differ for description and experience
A major finding from this work is that people make different risky choices in decisions from description versus decisions from experience, even without rare events. In decisions from description-where odds and outcomes are explicitly stated-people are more risk seeking for losses than gains (Figs. 1A and C). In contrast, people are more risk seeking for gains than losses in decisions from experience (Fig. 1B and C;Ludvig and Spetch, 2011;Madan et al., 2017). Critically, this reversal appears when both types of decision are made by the same participants (in alternating blocks in the same session) and even involving the exact same reward values. Whereas the pattern of risk preferences in decisions from description is consistent with the extant literature (e.g., Kahneman and Tversky, 1979), the reversed pattern of preferences in decisions from experience was novel and has become the dominant focus of our line of research (Ludvig and Spetch, 2011).

Extreme outcomes are overweighted in choice
After the initial 2011 study, we conducted a series of experiments with the goal of understanding the conditions that lead to these differences in risk preferences across description and experience (Ludvig et al., 2014a). For this series of studies, we focused solely on the decisions-from-experience component of the task and replicated several times the finding of more risk seeking for gains than losses (see Fig. 6). This pattern was dependent on the relative range of the values experienced-participants were more risk seeking for relative gains than losses, even when all of the outcomes presented were gains or losses. For example, if people were given a set that consisted of high-value gain decisions (e.g., fixed +60 versus risky +40/+80) and low-value gain decisions (e.g., fixed +20 versus risky 0/+40), then people made more risky choices for the high-value decisions (relative gains) than for the low-value decisions (relative losses), such as in the choice behaviour shown in Fig. 5.
To explain these findings, we proposed the extreme-outcome rule, whereby the extreme outcomes-highest and lowest relative to the range of values experienced-are overweighted in the decision-making process. In the above example, 0 would be overweighted as the extreme low value and +80 would be overweighted as the extreme high value. People behave as though there is a distortion in their subjective probabilities, not treating the two outcomes for the risky option as equiprobable. Instead, people choose as though they subjectively attribute a higher probability to the value that was either the best or worst outcome within the experiment's overall decision context (see also Lieder et al., 2018).
Recently, we have further refined the extreme outcomes as being defined by proximity to the edge of the experienced distribution. To do so, we included in the decision set a second risky option that led to values that neighbor the extreme values, but were not extreme themselves . In one of the experiments, there was a lowvalue decision set (with values ranging from +5 to +45) and a highvalue decision set (with values ranging from +55 to +95). The extremes were thus +5 and +95. In the low-value set, there was a safe option that led to +25, a risky extreme option that led to +5 (the extreme) or +45 (non-extreme), as well as a risky neighbor option that leads to +6 (near the extreme) or +44. As a control group, other participants would instead have the risky neighbor option that leads to +24 or +26. In this case, proximity to the edge determined what was overweighted in the decision-making process. Both the extremes outcomes (e.g., +5) and their nearby neighbours (e.g., + 6) were overweighted, but not the remote neighbours (e.g., +24). These results also provided robust evidence against an alternative hypothesis that discriminability (due to distance from neighbouring outcomes) was the key factor in determining what counted as an extreme outcome (e.g., Brown et al., 2007).
In another study, we directly manipulated the decision process by hastening the pace of decisions. In that case, we added both a time constraint on how long participants could take to make their choices (i.e., time pressure) and shortened the inter-trial interval (Madan, Spetch, & Ludvig, 2015). Participants were generally more risk seeking when under time pressure, but the tendency to overweight the extreme outcomes remained the same. This insensitivity of the extreme-outcome effect to time pressure suggest that the bias emerges early in the decision process, rather than through a process of extensive deliberation. Here, by focusing on decisions from experience, we again extended the existing literature on time pressure and risk, which had previously only focused on decisions from description (e.g., Ben Zur and Breznitz, 1981;Kocher et al., 2013).

Extreme outcomes are overweighted in memory
Given that extreme outcome are indeed overweighted, an important open question was what psychological mechanism was driving that overweighting. In a related set of studies using an episodic-memory approach, we had found that people better recalled stimuli associated with both the highest and lowest reward values (Madan and Spetch, 2012;. Based on this confluence of results, we hypothesized that the overweighting of extremes in choice might be due to an overweighting of these outcomes in memory. Perhaps the most extremes items are more memorable and are thus more likely to be retrieved from memory and used to guide choice. In Madan et al. (2014), we tested this conjecture directly, by adding two memory tests after the risky-choice task. First, we presented pictures of each of the doors (in random order) and asked participants to type the first outcome that came to mind for that door, which we termed the 'first-outcome-reported' test. This test assessed the availability of each outcome in memory. Next, we again presented each door, but this time also presented all of the possible outcomes within that experiment (e.g., −40, −20, 0, +20, +40); participants then estimated the percentage of the time that the presented door led to each of the possible outcomes, termed the 'frequency-judgment' test. This test assessed whether there were distortions in the remembered frequency of each outcome. Fig. 2 shows how participants demonstrated similar biases in both tests-they were more likely to report the extreme outcomes (in this example −40 and +40) and attributed higher frequencies to these outcomes.
This pattern of memory results was further replicated in experiments that included only gains, only losses, and decision sets with non-overlapping values Madan et al., 2014Madan et al., , 2017. The overweighting of extremes in memory reports even occurred when the blocks of decisions from experience were intermixed with blocks of decisions from description with the same values (Madan et al., 2017). Moreover, in each of these experiments there was a correspondence between these memory biases and the risky decisions from experience. Specifically, participants who reported the extreme value in the firstoutcome-reported test for the relative gains were more risk seeking for gain decisions, and those who reported the extreme value for the relative losses were more risk averse for loss decisions, compared to those people who reported the non-extreme values. With the frequencyjudgment test, there was again a similar, consistent pattern. People who remembered a higher frequency for the relative gains were more risk seeking for those gains, whereas those who remembered a higher frequency for the relative losses were more risk averse for those losses.
As the memory tests of choice outcomes correlated with preferences in the risky decisions from experience, we asked whether these memory biases may be responsible for the differences between decisions from description and experience (Madan et al., 2017). In a large-scale replication of our initial description-experience study (Ludvig and Spetch, 2011), but with the memory tests added in, risky choice across the two information formats (description and experience) was correlated. People who were more risk seeking in decisions from description were also relatively more risk seeking in decisions from experience. In addition, as above, the memory biases correlated with peoples' risky choice in decisions from experience. This relationship between memory and decisions from experience, however, did not generalize to decisions from description. There was no reliable correlation between memory biases and risky choice in the described problems. As such, although there are some commonalities to risky decision-making as a whole (e.g., see Frey et al., 2017), decisions from experience seem uniquely related to these reward-related memory biases.
These studies only provided a correlational link between memory and choice-to go beyond that, in a further study, we attempted to establish more of a causal relation by subtly nudging participants to be more risk seeking on specific trials through explicit memory cues (Ludvig et al., 2015). Fig. 3 shows how, in this study, each reward value was matched with an outcome-unique picture (Fig. 3A), unlike previous studies (e.g., Fig. 1) where all gain reward values were associated with the same pot of gold picture. This image was used to prime participants' memories before specific decision trials in the last block of the experiment (Fig. 3B). This manipulation successfully shifted choice: participants were significantly more likely to take the risky option after being reminded of past winning outcomes, as shown in Fig. 3C. Such winning cues have also been shown to shift risky choice in a gambling task with rats (Barus and Winstanley, 2016). The reminders may have served to increase the relative availability in memory of the distinct Fig. 2. Memory results from the first-outcome reported and frequency judgment tests. In the first-outcome-reported test, participants are shown each of the choice options (i.e., doors) one at a time and asked to respond with the first outcome that came to mind. In the frequency-judgment test, participants were again shown each choice option and asked to estimate the percentage of the time that the outcome occurred. risk-related outcomes during the decision. As such, choice in these decisions from experience may have some commonalities with the availability heuristic that manifests in many choice situations (e.g., Tversky and Kahneman, 1973).

Commonalities across species
The directly comparative angle to this research line has continued throughout, and we have run several studies on risky choices in pigeons, looking for commonalities and differences with human choice. In these studies, we have mostly used an open-field procedure to have pigeons choose a 'door' that had a set number of food cups behind it, making the procedure analogous to our series of studies with humans (Ludvig et al., 2014b). As with our usual procedure with humans, pigeons chose between pairs of safer and riskier options, which had higher or lower-value possible outcomes. Fig. 4 shows a schematic of the design as well as an illustration of the experimental set-up. Critically, Fig. 5 shows how, in an initial study, we found similar patterns of risk preference across the two species-they were both more risk seeking for the relative gains than the relative losses. This behavioural convergence suggested that a similar mechanism may be involved in risky decisions from experience in both species.
In a series of follow-up experiments, we further manipulated the range of outcomes experienced by both people and pigeons (Pisklak et al., 2018). Using both the same open-field procedure and an operant variation, when the outcomes included a zero (i.e., a no-reward option), both pigeons and people showed more risk seeking for high-value than low-value options (as in Ludvig et al., 2014b). However, when the lowest outcome was non-zero (i.e., options always led to at least some reward), then behavior diverged: People continued to show behaviour congruent with the extreme-outcome rule with more risk seeking for the high-value than the low-value options, but pigeons did not, as though their behaviour was more driven by avoidance of the zero (noreward) outcome than a low extreme. This comparative divergence presents a nuanced picture of the similarities and difference in the mechanisms underlying risky decisions from experience in people and pigeons. In other species, risky choice has been frequently examined (see Weber et al., 2004 for a review) ranging all the way from bees (e.g., Anselme, 2018;Shafir et al., 1999) to monkeys (e.g., Hayden, 2013, 2016), but, to the best of our knowledge, these studies have yet to evaluate potential sensitivities to extreme outcomes (zero or otherwise) in other non-human animals.

Overview of results
Having provided an overview of this programme of research, Fig. 6 provides a comprehensive summary of the decision sets and risky choices in our published studies from this line of research. This summary chart covers 14 experiments across 7 publications, with over 1200 participants. (The priming study (Ludvig et al., 2015) is not included in the figure as it did not include multiple risky options within the experimental design.) Accompanying this review paper, we have now made the raw data available for almost all of these prior studies: https://osf.io/eagcd/.
The extreme-outcome pattern is strikingly clear across studies. In nearly every case where the extreme-outcome rule would be expected to hold (in blue in the figure), there was more risk seeking for relative gains and losses, but not where it would not be expected to apply (the cases in orange). As would be the case with any random sampling process, there are some exceptions, but the bulk of the published evidence clearly supports the main claim (aligning with the rationale behind a p-curve analysis; Simonsohn et al., 2014). Although we summarize the key results of our prior experiments across several publications here, we only make qualitative comparisons between these results, given recent demonstrations that internal meta-analyses can problematically overstate the strength of evidence for an effect (Ueno et al., 2016;Vosgerau et al., 2018).
The summary of all procedures and results at once reveals several higher-level findings that were not immediately apparent in the individual studies. For instance, though people are consistently more risk seeking for relative gains than losses, i.e., the extreme-outcome rule, this effect is larger in magnitude when all of the options in the decision set are either gains or losses, in comparison to when the decision sets involve a mixture of both gains and losses. We have suggested that this may be the case because absolute gains and losses are easy to categorize and categorical memory may overshadow memory for the exact values Fig. 3. Overview of the priming study, (A) outcome contingencies, (B) trial procedure, and (C) risk preference results. Panel A illustrates that unique pictures were associated with each outcome; panel B shows these outcomes in a single trial procedure, as well as an outcome picture being presented preceding the choice, as a prime. Figure adapted from Ludvig et al. (2015). . When all the values are either gains or losses, people may attend more to the specific values and be more sensitive to the extremes of the range. The figure also makes apparent that risk preferences were rarely much above 50%, even for high-valued gains; instead, in these decisions from experience, we typically find strong risk aversion for the relative losses and risk neutrality or weak risk preference for the relative gains. Fig. 7 provides a parallel summary of the memory results from all the studies that included memory tests. For the first-outcome-reported test, results indicate the proportion of participants who reported the more extreme value of the decision set, relative to all of those who responded with a 'valid' outcome (i.e., an outcome that was associated with the risky option, not an 'other' outcome). Frequency judgment results are treated similarly, showing the relative proportion of responses for the more extreme value. As can be observed even within the individual studies, the first-outcome measure demonstrates more pronounced biases than the frequency-judgement test. This overview, however, makes apparent a few interesting consistencies across experiments. In particular, the bias to remember extreme outcomes appears to be consistently larger for outcomes associated with loss and low-value decisions than for outcomes associated with gain and highvalue decisions, in both the first-outcome-reported tests and the frequency-judgment tests.
There is also an indication that decision sets that are within one domain (i.e., all gains or all losses) lead to stronger memory biases than instances where both gains and losses are used. This pattern may suggest that differences in outcome magnitude are more salient than differences in reward valence. This incidental finding was previously suggested in Ludvig et al. (2018, p. 12), as "attending to category information (i.e., gain or loss) may overshadow learning of specific outcomes." The summary provided here provides more direct quantitative evidence for this result. Nonetheless, further research would be needed to test this mechanism directly.

Current lines of investigation
There are several important open questions that we are attempting to answer in ongoing studies. For example, we have been pushing on   Ludvig et al. (2014b). For pigeons, the high-value decisions corresponded to a choice between fixed 3 vs. risky 2 or 4 food cups; low-value decisions corresponded to fixed 1 vs. risky 0 or 2 food cups (as shown in Fig. 4B). For humans, the high-value decision corresponded to 60 vs. 40 or 80 points; low-value decisions corresponded to 20 vs. 0 or 40 points. the comparative angle to better assess the degree to which the mechanisms overlap or diverge across species. To that end, we have run several further studies with pigeons, including with an operant touchscreen procedure, to allow for closer matched comparisons between species (see above; Pisklak et al., 2018). In addition, to more closely link our work with the existing JDM literature, we are studying the impact of the extreme-outcome rule when decisions are not intermixed or when some outcomes occur only rarely as is typically studied in decisions from experience (e.g., Hertwig & Erev, 2009). Whereas the extreme-outcome rule is based on the extremity of the reward values experienced within the decision context, the frequency (or infrequency) of these outcomes is not considered, as our procedures have always Fig. 6. Comprehensive summary of risky choices in our previously published papers. For decisions, "X" denotes the condition with extreme values, "NX" denotes non-extremes, "Ng" denotes neighbour values. "Desc" and "Exp" denote decisions from description and experience, respectively; when not stated otherwise, all decisions were made from experience. Outcome values for the risky gain and high-value options are shown in green and upward triangles, with the corresponding safe option shown as black circles; risky losses and low-value options are shown in red and downward triangles, with the corresponding safe option shown as a white square; other outcome values are shown in gray markers. The proportions of risky choices for these decisions, P(Risky), are shown correspondingly in green, red, or gray. Differences in risky choices between pairs of decisions are shown in the Diff(P(Risky)) section. Pairings where the extreme-outcome rule is thought to apply are shown in blue; other pairings are shown in orange. Studies are ordered chronologically. Error bars are 95% confidence intervals. (Note that error bars in some previously published figures were SEMs.).
used risky options that could only lead to two, equiprobable outcomes.
Another fundamental question that remains unanswered is what defines the decision context. As shown in Fig. 6, the inclusion of a higher or lower set of values within an experiment can strongly influence risky choices on a specific decision set. For instance, for the exact same decision between 100% +20 points and an option that yields 50% +40 points or 50% 0 points, people are more risk seeking when the other outcomes in the decision set involve losses than when the other decision set involves higher-valued gains. In current work, we have borrowed from the memory literature to instantiate distinct contexts within a single experiment that provide different decision sets . We have recently undertaken a series of experiments to examine how visual and temporal contexts involving distinct decision sets may affect the extreme-outcome rule.
While we have ongoing work to further this line of research, others have also recognized the utility of this approach to decisions from experience and begun to use similar paradigms with their own adjacent research questions in mind. For example, Konstantinidis et al. (in press) used the same general procedure, but manipulated the magnitude of the gains and losses. Whereas many of our studies have used choices between 100% 20 points vs 50% 40 points, Konstantinidis and colleagues examined choices across four orders of magnitude, with the safe option being either 2, 20, 200, or 2000. They found that the extreme-outcome rule, greater risk seeking for relative gains than losses, was largest in magnitude for the smaller reward values and diminished when the reward values were in the thousands.
In a further extension, St-Amand et al. (2018) used a risky-choice task based on our procedure, but preceded it with either an episodicspecificity or a general-impressions induction task. The former task was designed to increase participants' attention to specific episodic details (e.g., Madore et al., 2014;Madore and Schacter, 2016); in contrast, the latter task asked participants to focus on 'gist'-like impressions. Interestingly, St-Amand et al. found that the general impressions induction task led to decreased risk taking and no bias in memory recall. In contrast, participants given either the episodic-specificity induction task or no induction task had comparable risk preference patterns and biased memory recall. More generally, the extreme-outcome rule has also found support with varied designs (Cox and Dallery, 2018;Le Pelley et al., in press;Wispinski et al., 2017).
Recent theoretical accounts of risky choice and the underlying sampling process have also incorporated the findings of this line of work (e.g., Gershman and Daw, 2017;Lieder et al., 2018). For example, in a recent theoretical analysis, Lieder et al. (2018) developed a rational model of decision-making wherein experienced outcomes were weighted by both their probability and their extremity. Their model provided a strikingly strong fit to our pattern of empirical results (e.g., from Madan et al., 2014), while also explaining other aspects of the description-experience gap. They further showed that such an overweighting of extremes, as we have repeatedly observed, actually reflects a rational use of limited cognitive resources. Their key idea is that, with a limited number of samples to draw from memory, overemphasizing the most extreme outcome leads to less variance in utility estimates and better overall performance.

Conclusion
Across 8 publications involving over 1300 participants, we have shown how the extreme-outcome rule, in which people are more riskseeking for relative gains than for relative losses, is extremely robust and replicable. We have also shown, however, that this effect is , not including responses from participants for outcomes that did not occur for the respective risky outcome. "X" denotes the condition with extreme values, "NX" denotes non-extremes, "Ng" denotes neighbour values. Outcome values for the risky gain and high-value options are shown in green and upward triangles, with the corresponding safe option shown as black circles; risky losses and low-value options are shown in red and downward triangles, with the corresponding safe option shown as a white square; other outcome values are shown in gray markers. Studies are ordered chronologically. Error bars are 95% confidence intervals. (Note that error bars in some previously published figures were SEMs.). dependent on key procedural features. The extreme-outcome rule only manifests when outcomes are learned through experience rather than being described, and it requires the intermixing of choices involving relative gains and losses within the same context. For example, the absolute level of risk preference for a choice between a fixed option leading to +20 points and a risky option leading to either +10 or +30 points, depends on whether the choice occurs in the context of other choices that involve losses or other choices that involve higher valued gains (see Madan et al., 2018).
One of the key findings in this research is that the extremes in a range of values are overweighted in both memory and choice. This result may represent another example of a general finding that values at the ends of a distribution have a privileged status. For example, with serially presented items, the first and last items experienced are better remembered (i.e., primacy and recency effects; Murdock, 1962;Wright et al., 1985). Humans also recall items associated with the highest and lowest values in a value-association task (Madan and Spetch, 2012). In perceptual discrimination tasks such as judging line-length, people are more accurate with values that fall at the ends of the distribution than for values in the middle of the distribution (e.g., Moon et al., 2015). It may be that the edges of a distribution across numerous dimensions have ecological relevance and command attention because they provide the boundary conditions for an experience. For example, in a foraging context, it may be important to track not only the overall rate of return, but also the best and worst returns, in order to learn the range of possible outcomes for a particular decision.
There are many questions remaining about the generality of the extreme-outcome effect. From a comparative perspective, more research is needed to determine to what degree the processes underlying the effect in humans are shared with other animals. Although our first comparative study showed striking similarities in the pattern of choice behavior between pigeons and humans (Ludvig et al., 2014b), follow-up work with a wider range of outcome values suggest that differences may exist in the mechanisms, with pigeons being particularly sensitive to zero values (Pisklak et al., 2018). Research on other species is needed to determine the species generality of sensitivity to extreme outcomes or to zero values. Research on humans using consummatory reinforcers, as opposed to secondary non-consumable reinforcers, such as points or money, may also help to make stronger species comparisons (see Hayden and Platt, 2009). Though it is more difficult to probe for memory recall in animals, creative procedures are being developed in other non-human species (e.g., Crystal, 2009;Eacott and Easton, 2007). Whether the extreme-outcome rule would generalize to other features of rewards besides magnitude is also an important future research question. For example, would people or other animals overweight the extremes of delays to an outcome, the number of responses required to obtain an outcome, or the quality of the outcome (e.g., palatability of food)?
On the theoretical side, important questions also remain about how best to model the choice process in decisions from experience (e.g., Erev et al., 2017;Lieder et al., 2018). One emerging theme is that people seem to be sampling from their memories of past outcomes, which can effectively percolate biases in memory into choice (e.g., Shadlen and Shohamy, 2016;Stewart, 2009). A similar sample-based proposal has recently been forwarded in the comparative literature, to account for many challenging phenomena in animal learning, such as spontaneous recovery and latent inhibition . A second theme highlights the important role of decision context-options are always evaluated relative to others in the same context, but what defines the context is still underdetermined (Bornstein and Norman, 2017).
This line of research began with a straightforward comparative question and blossomed into a line of research that has implications for models of decision making in humans and other animals. Thus, this research provides another example of how the comparative approach-in which animals must be 'asked' using behavioral methods and learning by experience is emphasize-can be fruitfully merged with other disciplines to provide a richer understanding of important cognitive processes (e.g., see Twyman et al., 2013).