Modeling Decision-Making Under Uncertainty with Qualitative Outcomes

Modeling decision-making under uncertainty typically relies on quantitative outcomes. Many decisions, however, are qualitative in nature, posing problems for traditional models. Here, we aimed to model uncertainty attitudes in decisions with qualitative outcomes. Participants made choices between certain outcomes and the chance for more favorable outcomes in quantitative (monetary) and qualitative (medical) modalities. Using computational modeling, we estimated the values participants assigned to qualitative outcomes and compared uncertainty attitudes across domains. Our model provided a good fit for the data, including quantitative estimates for qualitative outcomes. The model outperformed a utility function in quantitative decisions. Additionally, we found an association between ambiguity attitudes across domains. Results were replicated in an independent sample. We demonstrate the ability to extract quantitative measures from qualitative outcomes, leading to better estimation of subjective values. This allows for the characterization of individual behavior traits under a wide range of conditions. Author Summary In the current study, we explored how people make decisions when the outcomes aren’t easily measured in numbers, such as in medical choices. Traditional mathematical models, which rely on numerical data, often fall short in these situations, leading to a gap in understanding how people evaluate these qualitative outcomes. Using hierarchical Bayesian modeling, we developed a model that bridges this gap by translating qualitative outcomes into individualized quantitative values, enabling us to better understand the underlying decision-making processes. Our model not only provides a better fit to real-world data than existing models with qualitative or quantitative outcomes but also allows for meaningful comparisons of how people handle uncertainty across different decision-making scenarios. This approach opens new doors for studying decision-making in areas where traditional methods struggle, offering a more nuanced view of human behavior in complex situations.


Introduction
Life is a series of decisions where most outcomes are uncertain.These decisions range from trying a new dish at our favorite restaurant to selecting a life-saving medical treatment.Often, these decisions are quantitative in nature; for example, buying a lottery ticket for $2 with a 1 in 300 million chance of winning $21 million.Many decisions, however, are qualitative, such as choosing whether to dine at an Italian or Thai restaurant.The ability to compare different qualitative outcomes implies that we can derive some form of comparable subjective value for these outcomes [1].Here, we aimed to quantify how qualitative outcomes affect uncertainty attitudes.
Uncertainty around choice outcomes can be categorized into two components: risk and ambiguity.Risk occurs when probabilities of potential outcomes are precisely known [2]; ambiguity refers to situations where these probabilities are partially or entirely unknown [3].Prior research has shown that individuals generally exhibit an aversion to both risk and ambiguity in scenarios involving potential gains [4][5][6][7], but that these attitudes vary substantially across individuals and are not strongly correlated with each other [7][8][9][10][11][12][13]13].Many studies characterized these attitudes using choices with monetary (or point) outcomes [5,8,10,12,[14][15][16].The few studies that quantified individual attitudes in other domains still employed quantitative outcomes, including numbers of M&Ms and milliliters of water [1], months of extended lifespan [17], or milligrams of medication, and minutes spent with social partners [18].Although some earlier studies did examine qualitative decision-making [19,20], there is still no straightforward way to model these kinds of decisions.As a result, such decisions are typically not addressed in neuroimaging and psychiatric research.
In this study, we estimated risk and ambiguity attitudes in two separate modalities: quantitative (monetary decisions) and qualitative (medical decisions), leveraging computational modeling to extract values from qualitative outcomes and examine how uncertainty attitudes influence decision-making across various domains.
Sixty-six in-person and 332 online participants engaged in a task of decision-making under uncertainty, where they made choices between a certain outcome and the chance for a more favorable outcome.Our objectives were to (1) estimate subjective values for a range of qualitative outcomes, (2) assess the model's fit using quantitative outcomes for comparison, and (3) explore how attitudes towards uncertainty vary across different domains.

Method Study 1 (in-person)
Participants One hundred and one adults (48 females; age range = 18-89; mean 52.97, SD ±22.41) were screened for the experiment.All participants were screened over the phone to ensure the absence of major medical conditions, including neurological illness and lifetime Axis I psychiatric disorders.Participants provided written consent after a detailed explanation of the study was provided, following the institute guidelines approved by the Yale Human Investigation Committee.Ten participants did not complete the study and were not included in the analysis, resulting in a sample of ninety-one participants (failed to complete the task n=4; failed to come to consecutive sessions n=6).
To ensure all participants were cognitively healthy, we administered the Montreal Cognitive Assessment (MoCA) [21], which can detect mild cognitive impairments.Data from participants who scored less than 26 on the test were excluded [22], resulting in a sample of seventy-one cognitively healthy participants (32 females; age range = 18-88; mean 49.68 ±22.3 SD).

Procedure
Participants in this study came for three sessions completed within one week.
Task data reported here were collected on the first session.The MoCA was completed in session 3, along with several other questionnaires.In brief, in session 1, participants completed the decision-making task and a reversal reinforcement learning task that is not reported here.In session 2, participants completed an fMRI task.Finally, in session 3, participants completed several questionnaires assessing cognitive ability and general IQ.Participants were paid for each session separately and an extra fee for successfully completing the experiment.

Risk and ambiguity in the monetary domain
The task was based on a previously developed task [12] used in multiple studies.
On each trial, participants chose between a small certain gain ($5) and a lottery that offered a larger amount.The lottery was risky in half of the trials, i.e., with known outcome probability.The risky lotteries were represented as bags containing red and blue chips.The numbers of red and blue chips were indicated by the percentage of a rectangle colored in red and blue and the numbers on the bag.Three different outcome probabilities were used (25%, 50%, and 75%).Dollar amounts ($5, $8, $12, and $25) next to each color (Figure 1a) indicated the amount of money that could be won if a chip of that color was drawn.In the remaining trials, the lottery was ambiguous, i.e., outcome probability was not precisely known.Ambiguity was achieved by occluding part of the bag (Figure 1b), rendering the probability of drawing a chip of a certain color partially unknown.Increasing the occluder size (24%, 50%, and 74%) increases the ambiguity level or the range of possible probabilities for drawing a red or blue chip.Each combination (amount, risk/ambiguity) was repeated four times.On twelve (out of 84) trials, participants were asked to choose between $5 for sure and a chance to win $5.
Those trials were used as attention checks.Participants who failed six or more attention checks (n=3) were removed from the analysis in the risk and ambiguity task.In addition, participants who chose the lottery less than two times were omitted from the analysis because their data could not be fitted with a model (n=2).This exclusion was necessary because our choice function requires response variability; a consistent choice of one option provides no data points for model estimation.The final sample included 66 participants.At the end of the experiment, the computer randomly selected one of the trials, and the participants acted out the trials by selecting a chip.However, they did not receive the additional payment.We opted for a hypothetical outcome to make the monetary and medical conditions similar to each other (see below).

Risk and ambiguity in the medical domain
Participants were presented with a hypothetical scenario in which they were involved in a car accident and, as a result, suffered a spinal injury (supplementary methods for more details).They were asked to choose between two medical treatments (Figure 1c, 1d), a known treatment with a known outcome ("slight improvement," parallel to a fixed monetary gain of $5) or the experimental treatment (Figure 1e), with outcomes varying in the level of improvement ("moderate," "major," or "complete recovery").The likelihood of the outcome of the experimental treatment varied with different levels of risk and ambiguity (parallel to playing a lottery).Outcome probabilities and ambiguity levels were the same as in the monetary task and were presented graphically and verbally.All aspects of the experimental design were similar to the monetary task.The order of the monetary and medical tasks was counterbalanced across participants.Task Design: Participants were presented with choices between an uncertain option and a certain outcome across four scenarios: risky monetary decisions (A), ambiguous monetary decisions (B), risky medical decisions (C), and ambiguous medical decisions (D).In the risky scenarios (A, C), the outcome probabilities were visually represented by red and blue rectangles, and these probabilities were fully disclosed to the participants.In the ambiguous scenarios (B, D), the probability information was partially obscured by a grey rectangle, indicating uncertainty.The outcome probabilities in risky trials were set at 25%, 50%, and 75%, while the levels of ambiguity (indicated by the grey area) were set at 74%, 50%, and 24%.There were four possible outcomes for monetary decisions ($5, $8, $12, and $25) and four potential medical outcomes (E).Each unique pairing of uncertainty and outcome levels was presented to the participants four times.

Study 2 (online)
This dataset was used to confirm and replicate the results.This is a secondary analysis of a previously published dataset [23].

Participants
Four hundred and four adults (212 females; age range = 20-80; mean 49.362, SD ±14.849) were recruited using Amazon Mechanical Turk (mTurk).Participants provided consent online after reading a detailed explanation of the study following Yale Human Investigation Committee guidelines.Seventy-two participants did not complete the study and were not included in the analysis, resulting in a sample of three-hundred and thirty-two participants.

Procedure
Participants completed tasks similar to those described above, with two exceptions.First, each lottery was presented twice instead of four times.Second, an additional condition of 100% ambiguity was introduced.For more details, please see Xu et al., [23].

Modeling approach:
We employed hierarchical Bayesian modeling to capture and examine subjects' choice behavior.By leveraging hierarchical Bayesian modeling (HBM), we were able to uncover hidden variables that offer valuable insights into the underlying mechanisms of decision-making.HBMs allow for partial pooling of data across the population, meaning that individual data points contribute to both individual and group-level estimates.This results in more robust and accurate posterior distributions compared to non-hierarchical models, particularly when dealing with small sample sizes or individual-level variability [24].Moreover, we were able to compare different models to assess which described the data better.In the context of monetary decisions, to model the subjective value assigned to each option, we utilized a power (utility) function [6] with a linear effect of ambiguity on the perceived probability (Equation 1 -Classic Utility Model) [12].Equation 1 -Classic Utility Model: ~ (0.65,1) We modeled the subjective value (SV) using the objective probability (P; for risky options P = 0.25, 0.5, or 0.75, for ambiguous options P = 0.5, and for the certain gain P = 1), ambiguity level (A; for risky or certain options A= 0, for ambiguous trials A = 0.24, 0.5, or 0.74), and potential winnings (V; 5, 8, 12, 25 dollars for the lotteries and 5 dollars for the sure bet).Risk attitude (α) and ambiguity attitude (β) were incorporated to capture individual differences in risk and ambiguity preferences.Hyperpriors are higherlevel distributions that set prior beliefs on the parameters of the model's primary prior distributions.They are beneficial as they allow for incorporating prior knowledge and uncertainty about the parameters, leading to more flexible and robust models.Importantly, the same priors were used for the two independent data sets.
Hyperpriors were chosen based on previous data [12], with slight risk (mean 0.72) and ambiguity (mean 0.65) aversion (see supplementary results for comparison with less and uninformed hyper-priors).
To estimate the probability of choosing the lottery on each trial, we fitted a logistic choice function (equation 2), in which  is the inverse temperature parameter.Equation 2: In addition, we tested a "trembling-hand" logistic choice function (Equation 3), where there is less dependency between the risk and ambiguity parameters and the slope of the logistic function [25].
Equation 3 -Trembling-Hand Model: In the medical domain, fitting the utility function is challenging due to the qualitative nature of outcomes, which lacked a quantifiable value for the value (V) component of the model.To tackle this issue, we used the model to estimate the   ~ (2, 1) In this model, we estimated the subjective value () of each outcome (i) based on the cumulative values of the preceding outcomes.We assumed the outcomes were ordinal (e.g., slight improvement < moderate improvement), and thus, we modeled the subjective value of each outcome as the sum of the subjective value of the preceding outcomes plus an additive value representing the improvement.
To evaluate the model's fit, we introduced a baseline model devoid of subjectspecific parameters, serving as a 'straw man' to establish a reference point for the estimated model's performance.In the medical domain, we utilized the category level (e.g., 1 for slight improvement, 2 for moderate improvement) as the value (V) in the model (equation 5).

Model comparison
The different models were compared using a Leave-One-Out (LOO) crossvalidation method to estimate out-of-sample predictive fit [28].This method partitions the data into train and test data and fits the training-based data on the holdout test data to evaluate the fit.This process is done repeatedly.We used the 'Arviz' implementation to compute the LOO of the models.Unlike Log Likelihood, LOO measures expected log pointwise predictive density (ELPD).Thus, higher LOO points to a better fit.

Simulations
The estimated model has more degrees of freedom.Hence, a better fit could result from overfitting [29].One way to ensure that the results represent real phenomena rather than overfit is by using simulations [30].Using the Utility function (Equation 1) and the Estimated Value (Equation 4) models, we tested several scenarios, including different numbers of participants and levels of noise.The simulation assumed that the participant uses the utility function to make choices.However, each choice introduces a unique noise level to the risk attitude (α) and ambiguity attitude (β) parameters.The noise came from a normal distribution with a mean of 0 and a standard deviation that changed based on the simulation.To ensure model convergence, the risk attitude was limited to a range between 0.1 and 1.6, and the ambiguity attitude to a range between -1.4 and 1.4.We tested the following sets of parameters (N, noise): (30, 0.1), (30, 0.3), (30, 0.5), (60, 0.3), (60, 0.5), (120, 0.5), (300, 0.5).LOO values and weight were used to compare model fit.

Results
Sixty-six participants (Table 1 for demographics) completed a computerized decision-making task under uncertainty with qualitative and quantitative outcomes.Our objectives were to determine if we could (1) extract subjective values from choices between options with qualitative outcomes, (2) assess how well these estimated values fit the observed data, and (3) explore associations across uncertainty attitudes.Data from an additional sample of 332 online participants was used to replicate the results.

Cross-domain association between uncertainty attitudes
Finally, we explored how attitudes toward uncertainty vary across different domains.Using robust regression [31,32], we found that ambiguity aversion (β) in the medical domain was strongly and positively associated with the same attitude in the monetary domain (mean slope = 0.76, 89% HDP [0.63, 0.91]; Figure 3).This association was replicated in the online sample (mean slope = 0.37, 89% HDP [0.29, 0.44]).This finding suggests that attitudes towards ambiguity are consistent across different domains.It is important to note that risk attitudes, as operationalized in our models, depend on the outcome levels; that is, the value incorporates the risk preference.Thus, it precludes a straightforward comparison of risk attitudes across domains.

Simulations
The simulation results demonstrate that model performance is influenced by noise levels.With lower noise (0.1), the classic utility model provided a better fit.
However, as noise increased to 0.3 and 0.5, the estimated value model consistently showed superior performance, regardless of the number of subjects.Specifically, at higher noise levels, the estimated value model had higher LOO values and weights across different sample sizes.For detailed results, see supplementary results.

Discussion
This When testing uncertainty attitudes, one challenge that often arises is how to treat qualitatively different outcomes [33].Theoretically, this problem stems from the difficulty in comparing outcomes that do not share a common metric [34].Methodologically, it complicates the design of experiments and the interpretation of results, as traditional approaches often rely on quantitative measures [35].Statistically, it poses challenges in modeling and analyzing data due to the lack of a consistent scale for comparison.To address this issue, researchers often quantify an aspect of the outcome (e.g., milligrams of medication) [17,18,36] or treat the outcomes as categories with a fixed distance between them [23].In this study, we utilized computational modeling to extract estimated values for each outcome, providing insights into the processes underlying participants' decision-making.This approach allows for the use of different categories without the assumptions of specific relationships among them.By estimating these values, we can examine individual differences and use them in more precise parametric analyses of brain responses to value, enhancing our understanding of the neural mechanisms involved in processing uncertainty.
Uncertainty is especially relevant in the medical domain, as it is central to health decisions across the entire continuum of medical care [37,38].While medical diagnoses and treatments are often described qualitatively, experiments assessing uncertainty attitudes typically quantify these outcomes into measures like years of life [17,39] or milligrams of medication [18].This simplification abandons the original complex qualitative outcome.In contrast, our model allows for the introduction of complete qualitative outcomes.Our focus on ambiguity -rather than risk -attitudes in this study warrants an explanation.Technically, in the Estimated Value model, risk attitudes are not quantified separately but are integrated into the estimated values (Equation 4).More broadly, any risk attitude measure relies on the outcomes and requires equating the outcomes across domains.In contrast, ambiguity attitudes measure how ambiguity affects choices within a domain, and this modulation can be compared across domains.
Previous studies suggest that subtle experimental manipulations can alter decision attitudes [40] and that ambiguity attitudes are less stable over time [11,23] and lack a known structural correlate [41,42].Nevertheless, our results indicate that ambiguity attitudes are consistent across domains when studied simultaneously.Future studies should explore disentangling the risk component from the estimated values.
In the monetary domain, we demonstrated that our Estimated Value model fits the data better than a classical utility model.While the utility model we used is constrained to follow a specific curve [43], our estimated model has more degrees of freedom and can adapt more flexibly to the data.To mitigate the risk of overfitting, we employed a leave-one-out cross-validation procedure to compare the models.Our simulations also support our modeling approach.We simulated hypothetical participants To conclude, we present a model capable of assigning quantitative values to qualitative outcomes.The model demonstrates a better fit for both qualitative and quantitative data compared to other potential models.Although more complex than a classical utility function, both model comparisons and simulations suggest that the improved fit is not due to overfitting.This model opens new avenues for exploring the relationships between different domains, outcomes that cannot be objectively quantified, and the representation of value in the brain.

Figure 2 .
Figure 2.Estimated value compared with actual values and categories.

Figure 3 .
Figure 3. Cross-domain association in ambiguity aversion (β) study aimed to quantify the values of qualitative outcomes and to characterize individual uncertainty attitudes when making choices between such outcomes.Participants made a series of choices between certain outcomes and uncertain, potentially better, outcomes.Through computational modeling, we estimated the values participants assigned to different qualitative outcomes and assessed their attitudes towards ambiguity, which were consistent across two domains.Notably, our model outperformed a classical utility model with access to the objective amounts in the monetary domain.Overall, our model demonstrates a good fit and allows for better estimation of subjective values for both qualitative and quantitative outcomes.
who made choices based on the Classical Utility Model.As expected, in scenarios with low noise levels, the Classical Utility model outperformed the Estimated Value model.This result highlights how the LOO algorithm effectively penalizes the Estimated Value model for its extra complexity.However, as noise levels increased, leading to a higher deviation from the utility function, the Estimated Value model consistently outperformed the Classical Utility model, regardless of the number of participants.This suggests that the Estimated Value model is more adaptable to noisy data, providing a better fit for real participants' behavior.The estimated model may be better equipped to capture phenomena such as the framing effect or range-frequency theory[44][45][46].Unlike the utility function, which assumes outcomes are sampled from a single curve describing a person's behavior, the estimated model allows for unique curvatures for each set of outcomes.This means that the lowest and highest amounts create a frame of reference against which all other outcomes are compared.By learning about the possibilities within this frame, participants adjust their expectations accordingly.This flexibility enables the estimated model to more accurately reflect how people perceive and evaluate different outcomes in varying contexts.Having both monetary (quantitative) and medical (qualitative) datasets allowed us to assess the Estimated Value model with quantitative data and gain confidence in the model's ability to extract values in the qualitative dataset.The model's fit on quantitative data provides evidence that the estimates for the qualitative outcomes represent participants' true values.While these estimates are on a relative scale, without specific measurement units, they open the possibility for use in future studies to examine value representation in the brain.Additionally, these estimates can be applied to evaluate the values of outcomes across different scenarios, enhancing our understanding of how people perceive and compare various types of outcomes.

Table 1 .
Demographic description of the sample Classical Utility with a Trembling-Hand choice function model (equation 3).Our analysis revealed that the Estimated Value model outperformed the other models in terms of fit to the data within the monetary domain in both samples (see of the added values people attribute to the various outcomes.The mean value for slight improvement was 6.93, with a standard deviation (SD) of 1.79.This value increased by 9.00 (SD 2.62) for moderate improvement, by an additional 7.04 (SD 2.89) for major improvement, and finally by 4.18 (SD 2.41) for complete recovery (Figure2A).For the online sample, the mean value for slight improvement was 8.63 (SD 2.54), which outcome value.Specifically, we compared the Estimated Value model against several alternatives: a No-Subjective Parameters model (equation 5), a Classical Utility model (equation 1), and a

Table 2 )
. This finding underscores the robustness of the Estimated Value model across different decision-making contexts.Difference in LOO between the model and the best model.Shows how much worse a model's LOO score is compared to the best model's LOO.Weight: Model weights based on LOO scores.Indicates the relative importance or contribution of each model in model averaging.SE: Standard error of the LOO estimate.Reflects the uncertainty in the LOO score, with larger values indicating more uncertainty.
score, with 0 being the best.LOO: A higher LOO indicates better model fit, as it reflects the model's predictive accuracy.p_loo: Effective number of parameters.Represents the model complexity, with higher values indicating more complex models.d_loo: