Reverse engineering of metacognition

The human ability to introspect on thoughts, perceptions or actions − metacognitive ability − has become a focal topic of both cognitive basic and clinical research. At the same time it has become increasingly clear that currently available quantitative tools are limited in their ability to make unconfounded inferences about metacognition. As a step forward, the present work introduces a comprehensive modeling framework of metacognition that allows for inferences about metacognitive noise and metacognitive biases during the readout of decision values or at the confidence reporting stage. The model assumes that confidence results from a continuous but noisy and potentially biased transformation of decision values, described by a confidence link function. A canonical set of metacognitive noise distributions is introduced which differ, amongst others, in their predictions about metacognitive sign flips of decision values. Successful recovery of model parameters is demonstrated, and the model is validated on an empirical data set. In particular, it is shown that metacognitive noise and bias parameters correlate with conventional behavioral measures. Crucially, in contrast to these conventional measures, metacognitive noise parameters inferred from the model are shown to be independent of performance. This work is accompanied by a toolbox (ReMeta) that allows researchers to estimate key parameters of metacognition in confidence datasets.


Introduction 30
The human ability to judge the quality of one's own choices, actions and percepts by means of 31 confidence ratings has been subject to scientific inquiry since the dawn of empirical psychology (Pierce 32 and Jastrow, 1885; Fullerton and Cattell, 1892), albeit it has long been limited to specific research 33 niches. More recently, research on human confidence, and metacognition more generally, has 34 accelerated and branched off to other domains such as mental illnesses (Rouault et  Metacognitive biases of this type have been quite extensively studied in the judgement and decision-47 consequence of the hierarchical structure is that it is essential to capture the processes underlying the 104 decision values at the type 1 level as precisely as possible, since decision values are the input to 105 metacognitive computations. In the present model, this includes an estimate of both a sensory bias 106 and a sensory threshold, both of which will influence type 1 decision values. 107 Second, recent work has demonstrated that metacognitive judgements are not only influenced by 108 sensory noise, but also by metacognitive noise (Bang et al., 2019;Shekhar and Rahnev, 2021). In the 109 present model, I therefore consider sources of metacognitive noise either during the readout of type 1 110 decision values or during report. 111 Third, human confidence ratings are often subject to metacognitive biases which can lead to the 112 diagnosis of underconfidence or overconfidence. As outlined above, there is currently no established 113 methodology to measure under-and overconfidence, let alone measure different types of such biases. 114 In the present model, I consider four parameters that can be interpreted as metacognitive biases 115 either at the level of evidence or at the level of the confidence report. The interpretation of these 116 parameters as metacognitive biases entails the assumption that observers aim at reporting probability 117 correct with their confidence ratings (statistical confidence; Hangya et al., 2016). Although I discuss 118 link functions that deviate from this assumption, in the model outlined here, the transformation of 119 sensory evidence to confidence therefore follows the logic of statistical confidence. 120 I demonstrate the issues of conventional measures of metacognitive ability and metacognitive biases, 121 in particular their dependency on type 1 performance, and show that the process model approach can 122 lead to unbiased inferences. Finally, I validate the model on a recently published empirical dataset 123 . I illustrate for this dataset how model parameters can describe different 124 facets of metacognition and assess the relationship of these parameters to conventional measures of 125 metacognitive ability and metacognitive bias. 126 This article is accompanied by a toolbox − the Reverse engineering of Metacognition (ReMeta) 127 toolbox, which allows researchers to apply the model to standard psychophysical datasets and make 128 inferences about the parameters of the model. It is available at github.com/m-guggenmos/remeta. 129 Figure 1. Computational model. Input to the model is the stimulus variable x, which codes the stimulus category (sign) and the intensity (absolute value). Type 1 decision-making is controlled by the sensory level. The processing of stimuli x at the sensory level is described by means of sensory noise (σs), bias (δs) and threshold (ϑs) parameters. The output of the sensory level is the decision value y, which determines type 1 decisions d and provides the input to the metacognitive level. At the metacognitive level it is assumed that the dominant source of metacognitive noise is either noise at the readout of decision values (noisy-readout model) or at the reporting stage (noisy-report model).
In both cases, metacognitive judgements are based on the absolute decision value |y| (referred to as sensory evidence), leading to an internal representation of metacognitive evidence * . While the "readout" of this decision value is considered precise for the noisy-report model (z = * ), it is subject to metacognitive readout noise in the noisy-readout model (z ∼ fm(z; σm), where σm is a metacognitive noise parameter). A link function transforms metacognitive evidence to confidence * . In the case of a noisy-report model, the dominant metacognitive noise source is during the report of confidence, i.e. confidence reports c are noisy expressions of the internal confidence representation * . Metacognitive biases operate at the level of sensory evidence (multiplicative evidence bias m, additive evidence bias δm) or at the level of the confidence link function (multiplicative confidence bias m, additive confidence bias m).

131
Results are structured in three parts. The first part introduces the architecture and the computational 132 model, from stimulus input to type 1 and type 2 responses. The second part provides the 133 mathematical basis for model inversion and parameter fitting and systematically assess the success of 134 parameter recovery as a function of sample size and varied ground truth parameter values. Finally, in 135 the third part, the model is validated on an empirical dataset . 136 1 Computational model 137

Computing decision values 138
For the model outlined here, the task space is restricted to two stimulus categories referred to as S − 139 and S + . Stimuli are described by the stimulus variable x, the sign of which codes the stimulus category (negative) values of δs lead to a propensity to choose stimulus category S + (S − ). In addition, the sensory 145 threshold ϑs ∈ℝ + defines the minimal stimulus intensity, which is necessary to drive the system, i.e., 146 above which the observer's type 1 choices can be better than chance level ( Figures 2B). 147 Mathematically, this means that the decision values * are zero below ϑs in the absence of bias and δs 148 in the simultaneous presence of a bias ( Figure 2C). Note that a sensory threshold parameter should 149 only be considered if the stimulus material includes intensity levels in a range at which participants 150 perform close to chance. Otherwise, the parameter cannot be estimated and should be omitted, i.e., 151 Equation 1 reduces to * = x + δs. 152 In the model described here I assume that decision values can be linearly constructed from the 153 stimulus variable x. In practice, this may often be too strong of an assumption, and it may thus be 154 necessary to allow for a nonlinear transformation of x ('nonlinear transduction', see e.g., Dosher and 155 Lu, 1998). The toolbox therefore offers an additional nonlinear transformation parameter γs (see 156 The final decision value y is subject to sources of sensory noise σs, described by a logistic distribution 158 fs(y): 159 Equation 2 is a reparameterization of a standard logistic distribution in terms of the standard deviation 160 σs using the fact that the variance of the logistic distribution is equal to s²π²/3 (where s is the 161 conventional scale parameter of the logistic distribution). Figure 2D shows psychometric functions 162 with varying levels of sensory noise σs. The logistic distribution was chosen over the more conventional 163 normal distribution due to its explicit analytic solution of the cumulative density − the logistic 164 function. In practice, both distributions are highly similar, and which one is chosen is unlikely to 165 matter. 166 Type 1 decisions d between the stimulus categories S + and S − are based on the sign of y: 167

From decision values to metacognitive evidence 168
The decision values computed at the sensory level constitute the input to the metacognitive level. I 169 assume that metacognition leverages the same sensory information that also guides type 1 decisions 170 (or a noisy version thereof). Specifically, metacognitive judgements are based on a readout of absolute 171 decision values |y|, henceforth referred to as sensory evidence or just evidence. Respecting a 172 multiplicative (m ∈ ℝ + ) and an additive (δm ∈ ℝ) evidence bias, we obtain an estimate of sensory 173 evidence at the metacognitive level (metacognitive evidence * ): 174 The multiplicative evidence bias m and the additive evidence bias δm are two different types of 175 metacognitive biases at the readout stage, which are described in more detail in section 1.4. Note that 176 the max operation is necessary to enforce positive values of metacognitive evidence. 177

The link function: from metacognitive evidence to confidence 178
The transformation from metacognitive evidence to predicted confidence * is described by a link 179 function. A suitable link function must be bounded, reflecting the fact that confidence ratings typically 180 have lower and upper bounds, and increase monotonically. 181 I assume that observers aim at reporting probability correct, leading to a logistic link function in the 182 case of the logistic sensory noise distribution (Equation 2). Without loss of generality, I use the range 183 [0;1] for confidence ratings, such that a confidence of 0 indicates chance level probability (0.5) and a 184 confidence of 1 the expectation of perfect type 1 performance. Note that I do not consider the 185 possibility that type 1 errors can be reported at the time of the confidence report, i.e., confidence 186 cannot be negative. With these constraints and using the simple mathematical relationship between 187 the logistic function and the tangens hyperbolicus, one arrives at the following link function (see 188 Appendix 1, Equation A1, for the derivation): Note that I use the variable z as opposed to * , to indicate that the metacognitive evidence that enters 190 the link function may be a noisy version of * (see the description of the noisy-readout model below). 191 I refer to * as the predicted confidence, which may be different from the ultimately reported 198 confidence c. This distinction becomes important when metacognitive noise is considered at the level 199 of the report (see Result section 1.7). 200 Figure 3. Effect of model parameters on the evidence-confidence relationship. All metacognitive bias parameters and noise parameters affect the relationship between the sensory evidence |y| and confidence, assuming the link function provided in Equation 5. (A) Effect of metacognitive bias parameters on the evidence-confidence relationship. Metacognitive noise was set to zero for simplicity. (B) Effect of metacognitive noise σm and sensory noise σs on the evidence-confidence relationship. Metacognitive noise renders confidence ratings more indifferent with respect to the level of sensory evidence. Note that, due to the absence of an analytic expression, the illustration for the effect of metacognitive noise is based on a simulation. Increasing sensory noise affects the slope of the confidence-evidence relationship, reflecting changes to be expected from an ideal metacognitive observer.

Metacognitive biases 201
Metacognitive biases describe a systematic discrepancy between objective type 1 performance and 202 subjective beliefs thereof (expressed via confidence ratings). Holding type 1 performance constant, 203 overconfident observers systematically prefer higher confidence ratings, while underconfident 204 observers systematically prefer lower confidence ratings. Importantly, metacognitive biases are 205 orthogonal to the metacognitive sensitivity of an observer. For instance, an underconfident observer 206 who consistently chooses the second-lowest confidence rating for correct choices could have high 207 metacognitive sensitivity nevertheless, as long as they consistently choose the lowest rating for 208 incorrect choices. In the present model I consider metacognitive biases either at the level of evidence 209 or at the level of confidence ( Figure 1). 210 Metacognitive evidence biases represent a biased representation of sensory evidence at the 211 metacognitive level. These biases may be either due to a biased readout from sensory channels or due 212 to biased (pre)processing of read-out decision values at the initial stages of the metacognitive level. 213 In either case, evidence biases affect the metacognitive representation z of sensory evidence and may 214 be multiplicative or additive in nature. The multiplicative evidence bias m leads to a scaling of 215 absolute sensory decision values, with m < 1 and m > 1 corresponding to under-and overconfident 216 observers, respectively. The additive evidence bias δm represents an additive bias such that 217 metacognitive evidence is systematically decreased (underconfidence) or increased (overconfidence) 218 by a constant δm. Values δm < 0 can be interpreted as a metacognitive threshold, such that the 219 metacognitive level is only 'aware' of stimuli that yield sensory evidence above δm. 220 An alternative interpretation of metacognitive evidence biases at the readout stage is that they 221 correspond to an under-or overestimation of one's own sensory noise σs. Applying this view, a value 222 of e.g. m > 1 would suggest that the observer underestimated sensory noise σs and hence shows 223 overconfidence, whereas a value of m > 1 implies that the observer overestimated σs and thus is 224 in which no metacognitive biases are present (i.e., m/λm = 1, δm/m = 0; black lines). One could assume 235 that calibration curves for bias-free observers are identical to the diagonal, such that objective and 236 subjective accuracy are identical. This is not the case − the calibration curve is tilted towards 237 overconfidence. This may seem surprising but reflects exactly what is expected for a bias-free 238 statistical confidence observer. This is best understood for the extreme case when the subjective 239 probability correct is arbitrarily close to 1. Even for very high ratings of subjective probability there is 240 − due to sensory noise − a certain finite probability that associated type 1 choices have been incorrect. 241 Hence, objective type 1 performance is expected to be below the subjective probability in these cases. 242 Importantly, relative to this bias-free observer all metacognitive bias parameters yield calibration 243 curves that resemble under-and overconfidence given appropriate choices of the parameter values 244 (underconfidence: redhish lines; overconfidence: blueish lines). 245 As mentioned previously, metacognitive sensitivity (AUROC2, meta-d') is strongly dependent on 246 type 1 performance. How do metacognitive biases perform in this regard, when measured in a model-247 free manner from choice and confidence reports? To find out, I simulated confidence biases for a 248 range of metacognitive bias parameter values and type-1-performance levels (by varying the sensory 249 noise parameter). Confidence biases were computed as the difference between subjective probability 250 correct (by linearly transforming confidence from rating space [0; 1] to probability space [0.5; 1]) and 251 objective probability correct. As shown in the middle panels of Figure 4, these results showcase the 252 limits of naively measuring confidence biases in this way. Again, the bias-free observer shows an 253 apparent overconfidence bias. In addition, this bias increases as type 1 performance decreases, 254 reminiscent of the classic hard-easy effect for confidence ; for related 255 analyses, see Soll, 1996;Merkle, 2009;Drugowitsch, 2016;Khalvati et al., 2021). At chance level 256 performance, the overconfidence bias is exactly 0.25. 257 The value of 0.25 can be understood in the context of the '0.75 signature' Adler 258 and Ma, 2018a). When evidence discriminability is zero, an ideal Bayesian metacognitive observer will 259 show an average confidence of 0.75 and thus an apparent (over)confidence bias of 0.25. Intuitively 260 this can be understood from the fact that Bayesian confidence is defined as the area under a 261 probability density in favor of the chosen option. Even in the case of zero evidence discriminability, 262 this area will always be at least 0.5 − otherwise the other choice option would have been selected, 263 but often higher. 264 The overconfidence bias leads to another peculiar case, namely that the bias of truly underconfident 265 observers (i.e., m < 1, δm < 0, λm < 1, or m < 0) can show a sign flip from over-to underconfidence as 266 performance increases from chance level to perfect performance (dotted lines in the middle panels 267 of Figure 4). Overall, the simulation underscores that metacognitive biases are just as confounded by 268 type 1 behavior as metacognitive sensitivity. 269 Is it possible to recover unbiased estimates for the metacognitive bias parameters by inverting the 270 process model? To find out, I again simulated data for a range of type-1-performance levels and true 271 values of the bias parameters. In each case, I fitted the model to the data to obtain estimates of the 272 parameters. As shown in the right panels of Figure 4, parameter recovery was indeed unbiased across 273 the type 1 performance spectrum, with certain deviations only for extremely low or high type 1 274 performance levels. This demonstrates that, in principle, unbiased inferences about metacognitive 275 biases are possible in a process model approach, assuming that the fitted model is a sufficient 276 approximation of the empirical generative model. 277 Simulations are based on a noisy-report model with a truncated normal metacognitive noise distribution. Metacognitive noise was set close to zero for simplicity. (Left panels) Calibration curves. Calibration curves compute the proportion of correct responses (objective probability correct) for each interval of subjective confidence reports. For this analysis, confidence was transformed from rating space [0; 1] to probability space [0.5; 1] and divided in 100 intervals with bin size 0.01. Calibration curves above and below the diagonal indicate under-and overconfident observers, respectively. Average type 1 performance for this simulation was around 70%. (Middle panels) Confidence bias in dependence of type 1 performance. Different levels of type 1 performance were simulated by sweeping the sensory noise parameter between 0.01 and 50. Confidence bias was computed as the difference between subjective probability correct and objective proportion correct. (Right panels) Recovery of metacognitive bias parameters in dependence of performance. Shades indicate standard deviations.
Finally, note that the parameter recovery shown in Figure 4 was performed with four separate models, 278 each of which was specified with a single metacognitive bias parameter (i.e., m, δm, λm, or m).

279
Parameter recovery can become unreliable when more than two of these bias parameters are 280 specified in parallel (see section 2.3). In practice, the researcher thus must make an informed decision 281 about which bias parameters to include in a specific model (in most scenarios one or two 282 metacognitive bias parameters are a good choice). While the evidence-related bias parameters m 283 and δm have a more principled interpretation (e.g., as an under/overestimation of sensory noise), it is 284 not unlikely that metacognitive biases also emerge at the level of the confidence report (λm, m). The 285 first step thus must always be a process of model specification or a statistical comparison of candidate 286 models to determine the final specification (see also section 3.1). 287

Confidence criteria 288
In the model outlined here, confidence results from a continuous transformation of metacognitive 289 evidence, described by a parametric link function (Equation 5). The model thus has no confidence 290 criteria. However, it would be readily possible to replace the tangens hyperbolicus with a stepwise 291 link function where each step is described by the criterion placed along the z-axis and the respective 292 confidence level (alternatively, one can assume equidistant confidence levels, thereby saving half of 293 the parameters). Such a link function might be particularly relevant for discrete confidence rating 294 scales where participants associate available confidence ratings with often idiosyncratic and not easily 295 parameterizable levels of metacognitive evidence. 296 Yet, even for the parametric link function of a statistical confidence observer it is worth considering 297 two special confidence criteria: a minimum confidence criterion, below which confidence is 0, and a 298 maximum criterion, above which confidence is 1. Indeed, the over-proportional presence of the most 299 extreme confidence ratings that is often observed in confidence datasets (cf. Confidence Database; 300 Rahnev et al., 2020) may motivate such criteria. 301 My premise here is that these two specific criteria can be described as an implicit result of 302 metacognitive biases. In general, when considering an ideal statistical confidence observer and 303 assuming continuous confidence ratings, the presence of any criterion reflects suboptimal 304 metacognitive behavior − including a minimum or maximum confidence criterion. According to 305 Equation 5, an ideal observer's confidence should never be exactly 1 (for finite sensory noise) and 306 should only ever be 0 when metacognitive evidence is exactly zero, which makes a dedicated criterion 307 for this case likewise superfluous. 308 Importantly, a minimum confidence criterion is implicit to the additive evidence bias δm. As explained 309 above, a negative value of δm effectively corresponds to a metacognitive threshold, such that 310 metacognitive evidence z (and hence confidence) is zero for decision values smaller than δm. A 311 maximum confidence criterion can be realized by the confidence bias parameters λm and m. 312 Specifically, assuming λm > 1 or m > 0, the maximum criterion is the point along the metacognitive 313 evidence axis at which a link function of the form λm·tanh(..) + m becomes exactly 1. In sum, both a 314 minimum and a maximum confidence criterion can be implemented as a form of a metacognitive bias. 315

Metacognitive noise: noisy-readout models 316
A key aspect of the current model is that the transformation from sensory decision values to 317 confidence reports is subject to sources of metacognitive noise. In this section, I first consider a model 318 of type noisy-readout, according to which the metacognitive noise mainly applies to the metacognitive 319 readout of absolute sensory decision values (i.e., * ). The final metacognitive evidence z is thus a noisy 320 version of * . By contrast, sources of noise involved in the report of confidence are considered 321 negligible and the predicted confidence estimate * resulting from the link function is equal to the 322 reported confidence c. 323 Metacognitive noise is defined by a probability distribution and a metacognitive noise parameter σm. The second scenario is that the nature of metacognitive readout noise itself makes sign flips 345 impossible, sparing the necessity of censoring. This required noise distributions that are bounded at 346 zero, either naturally or by means of truncation. I first consider truncated distributions, in particular 347 the truncated normal and the truncated Gumbel distribution ( Figure 5B). Truncating a distribution 348 means to cut off the parts of the distribution outside the truncation points (here the range below zero) 349 and to renormalize the remainder of the distribution to 1. 350 While truncated distributions behave well mathematically, compared to censored distributions it is 351 much less clear how a natural process could lead to a truncated metacognitive noise distribution. 352 Truncated distributions occur when values outside of the bounds are discarded, which clearly does 353 not apply to confidence ratings. I thus consider truncated distributions as an auxiliary construct at this 354 point that may nevertheless qualify as an approximation to an unknown natural process. 355 Finally, there are many candidates of probability distributions that are naturally bounded at zero, 356 perhaps the most prominent one being the lognormal distribution. In addition, I consider the Gamma 357 distribution ( Figure 5C), which has a more pronounced lower tail and is also the connatural 358 counterpart to the Beta distribution for noisy-report models (see next section). 359 Figure 5. Metacognitive noise. Considered noise distributions are either censored, truncated or naturally bounded. In case of censoring, protruding probability mass accumulates at the bounds (depicted as bars with a darker shade; the width of these bars was chosen such that the area corresponds to the probability mass). The parameter σm and the distributional mode was set to ⅓ in all cases. (A - C) Noisy-readout models. Metacognitive noise is considered at the level of readout, affecting metacognitive evidence z. Only a lower bound at z = 0 applies. (D - F) Noisy-report models. Metacognitive noise is considered at the level of the confidence report, affecting internal confidence representations c. Confidence reports are bounded between 0 and 1.

Metacognitive noise: noisy-report models 360
In contrast to noisy-readout models, a noisy-report model assumes that the readout noise of decision 361 values is negligible (z = * ) and that the dominant source of metacognitive noise occurs at the 362 reporting stage (c ∼ fm(c)). Reporting noise itself may comprise various different sources of noise, 363 occurring e.g. during the mental translation to an experimental confidence scale or in the form of 364 visuomotor noise (e.g. when using a mouse cursor to indicate a continuous confidence rating). 365 A hard constraint for reporting noise is the fact that confidence scales are typically bounded between 366 a minimum and a maximum confidence rating (reflecting the bounds [0; 1] for c in the present model). 367 Reported confidence cannot be outside these bounds, regardless of the magnitude of reporting noise. 368 As in the case of the noisy-readout model, one may consider either censored ( Figure 5D), truncated 369 ( Figure 5E) or naturally bounded distributions (Beta distribution; Figure 5F) to accommodate this 370 constraint. 371

Metacognitive noise as a measure of metacognitive ability 372
As outlined above, I assume that metacognitive noise can be described either as variability during 373 readout or report. In both cases, metacognitive noise is governed by the parameter σm. Higher values 374 of σm will lead to a flatter relationship between reported confidence and sensory evidence, i.e., 375 confidence ratings become more indifferent with regard to different levels of evidence ( Figure 3B). 376 The behavior of the metacognitive noise parameter is closely related to the concept of metacognitive 377 efficiency (Fleming and Lau, 2014), a term coined for measures of metacognitive ability that aim at 378 being invariant to type 1 performance (in particular, Mratio). As outlined in the introduction, the type 1 379 performance independence of Mratio has been contested to some degree, on the basis of empirical 380 data and as well as in simulations that consider the presence of metacognitive noise (Bang et  To assess the type 1 performance dependency, I simulated data with varying levels of sensory noise σs 386 and five different values of σm. In each case I computed Mratio on the data and also fitted the model to 387 recover the metacognitive noise parameter σm. As shown in the left panels of Figure 6A (noisy-report) 388 and 6B (noisy-readout), Mratio shows a nonlinear dependency with varying type 1 performance levels. 389 While this simulation was based on multiple stimulus levels, a similar nonlinear dependency is also 390 present for a scenario with constant stimuli (Figure 6-figure supplement 1). 391 By contrast, the parameter σm is recovered without bias across a broad range of type 1 performance 392 levels and at different levels of generative metacognitive noise ( Figure 6, middle panels). The 393 exception is a regime with very high metacognitive noise and low sensory noise under the noisy-394 readout model, in which recovery becomes biased. A likely reason is related to the inversion of the 395 link function, which is necessary for parameter inference in noisy-readout models (cf. section 2.2): 396 since the link function is dependent on sensory noise σs, its inversion becomes increasingly imprecise 397 as σs approaches very small (or very high) values. However, apart from these extremal cases under the 398 noisy-readout model, σm is largely unbiased and is thus a promising candidate to measure 399 metacognitive ability independent of type 1 performance. Figure 6-figure supplement 2 shows that 400 this conclusion also holds for various settings of other model parameters. 401 Figure 6. Comparison of Mratio and metacognitive noise σm. Different performance levels were induced by varying the sensory noise of the forward model. Five different levels of metacognitive noise were simulated for a truncated normal noise distribution, covering the range between low and high metacognitive noise. While Mratio showed a nonlinear dependency with varying type 1 performance levels both for A) noisy-report models and B) noisy-readout models, the recovered metacognitive noise parameter σm was largely independent of type 1 performance. Shaded areas indicate standard deviations across 100 simulated subjects. Right panels: Relationship between metacognitive noise and Mratio. Simulated data were generated with a range of varying metacognitive noise parameters σm and constant sensory noise (σs = 0.5; proportion correct responses: 0.82). Computed Mratio values show a clear negative correspondence with σm, reflecting the fact that metacognitive performance decreases with higher metacognitive noise.
Despite the fact that Mratio may not be entirely independent of type 1 performance, it is fairly likely 402 that it will capture the metacognitive ability of observers to some degree. It is thus interesting to assess 403 the relationship between the model-based measure of metacognitive noise (σm) and Mratio. To this aim, 404 I performed a second simulation which kept type 1 performance constant at around 82% correct by 405 using a fixed sensory noise parameter (σs = 0.5) and only varied the true underlying parameter σm. In 406 addition, Mratio was computed for each simulated observer. As shown in the right panels of Figures  recovery. An exception are models with both confidence bias parameters (m, m) which additionally 500 consider one of the evidence parameters ( m or δm). For these models, considerable trade-offs 501 between the bias parameters start to emerge. Finally, a model with all four considered metacognitive 502 bias essentially fails to recover its bias parameters. 503

Model recovery 522
One strength of the present modeling framework is that it allows testing whether inefficiencies of 523 metacognitive reports are better described by metacognitive noise at readout (noisy-readout model) 524 or at report (noisy-report model). To validate this type of application, I performed an additional model 525 recovery analysis which tested whether data simulated by either model are also best fitted by the 526 respective model. Figure 7-figure supplement 6 shows that the recovery probability was close to 1 527 in most cases, thus demonstrating excellent model identifiability. With fewer trials per observer, 528 recovery probabilities decrease expectedly, but are still at a very good level. 529 The only edge case with poorer recovery was a scenario with low metacognitive noise and high sensory 530 noise. Model identification is particularly hard in this regime because low metacognitive noise reduces 531 the relevance of the metacognitive noise source, while high sensory noise increases the general 532 randomness of responses. 533 3 Application to empirical data 534

On using the model framework 535
It is important to note that the present work does not propose a single specific model of 536 metacognition, but rather provides a flexible framework of possible models and a toolbox to engage 537 in a metacognitive modeling project. Applying the framework to an empirical dataset thus requires a 538 number of user decisions: which metacognitive noise type is likely more dominant? which 539 metacognitive biases should be considered? which link function should be used? These decisions may 540 be guided either by a priori hypotheses of the researcher or can be informed by running a set of 541 candidate models through a statistical model comparison. 542 As an exemplary workflow, consider a researcher who is interested in quantifying overconfidence in a 543 confidence dataset with a single parameter to perform a brain-behavior correlation analysis. The 544 concept of under/overconfidence already entails the first modeling decision, as only a link function 545 that quantifies probability correct (Equation 6) allows for a meaningful interpretation of 546 metacognitive bias parameters. Moreover, the researcher must decide for a specific metacognitive 547 bias parameter. The researcher may not be interested in biases at the level of the confidence report, 548 but, due to a specific hypothesis, rather at metacognitive biases at the level of readout/evidence, thus 549 leaving a decision between the multiplicative and the additive evidence bias parameter. Also, the 550 researcher may have no idea whether the dominant source of metacognitive noise is at the level of 551 the readout or report. To decide between these options, the researcher computes the evidence (e.g., 552 AIC) for all four combinations and chooses the best-fitting model (ideally, this would be in a dataset 553 independent from the main dataset). 554

Application to an example dataset (Shekhar and Rahnev, 2021) 555
To test the proposed model on real-world empirical data, I used a data set recently published by 556 Shekhar and Rahnev (2021) which has a number of advantageous properties for a modeling approach. 557 First, a high number of 2800 trials were measured for each of the 20 participants, enabling a precise 558 estimate of computational parameters (cf. Figure 7-figure supplement 5). Second, the task design 559 comprised multiple stimulus intensities, which is expected to improve the fit of a process model. And 560 third, participants rated their confidence on a continuous scale. While the model works well with 561 discrete confidence ratings, only continuous confidence scales harness the full expressive power of 562 the model. In each trial, participants indicated whether a Gabor patch imposed on a noisy background 563 was tilted counterclockwise or clockwise from a vertical reference and simultaneously rated their 564 confidence. The average performance was 77.7% correct responses. 565 Figure 8A visualizes the overall model fit at the sensory level. The posterior, defined as the probability 566 of choosing S + , closely matched the model fit. The average posterior probability showed a slight y-567 offset, reflecting a negative mean sensory bias towards S + (δs = −0.06 ± 0.03) and serving as a validation 568 that the model parameter sensibly captured the psychometric curve. Since no stimulus intensities 569 near chance-level performance were presented to participants, a sensory threshold parameter was 570 not fitted. 571 At the metacognitive level, I compared noisy-readout and noisy-report models in combination with 572 the metacognitive noise distributions introduced in Result sections 1.6 and 1.7. For this analysis, I 573 considered metacognitive evidence biases only (i.e., multiplicative evidence bias m and additive 574 evidence bias δm). The model evidence was computed based on the Akaike information criterion (AIC; 575 Akaike, 1974). As shown in Figure 8B, with the exception of censored distributions, all models 576 performed at a similar level. Seven of the 10 tested models were the winning model for at least one 577 participant ( Figure 8C). 578 Interestingly, there were quite clear patterns between the shapes of individual confidence 579 distributions and the respective winning model (Figure 8-figure supplement 1). For instance, a single 580 participant was best described by a noisy-report + Beta model, and indeed the confidence distribution 581 of this participant is quite unique and plausibly could be generated by a Beta noise distribution 582 (participant 7). Participants who were best fitted by noisy-readout models have quite specific 583 confidence distributions with pronounced probability masses at the extremes and very thin coverage 584 at intermediate confidence levels (participants 4-6, 8, 10, 13, 19) − except those, for which the 585 lognormal readout noise distribution was optimal (participants 9 and 11). Finally, two participants 586 were best fitted by a censored distribution (participants 14 and 16), contrary to the general tendency. 587 These participants likewise had fairly idiosyncratic confidence distributions characterized by the 588 combination of a probability mass centered at mid-level confidence ratings and a prominent 589 probability mass at a confidence of 1. While a more detailed analysis of individual differences is 590 beyond the scope of this paper, these examples may point to distinct phenotypes of metacognitive 591 noise. 592 In the next step, I inspected the winning metacognitive model (noisy report + truncated Gumbel) in 593 more detail. While the selection of this specific model is arbitrary due to the similar performance of 594 several other models, it serves the illustrative purpose and the differences between these models 595 were overall negligible. 596 I first compared confidence ratings predicted by the model with empirical confidence ratings across 597 the range of experimental stimulus intensities. As shown in Figure 8D, model-predicted confidence 598 tracked behavioral confidence quite well ( Figure 8D). This included a slight confidence bias towards 599 S + , which itself is likely a result of the general sensory bias towards S + . incorrect. I will elaborate more on this aspect in the discussion, but assert for now that metacognitive 607 efficiency in empirical data can, at least in part, be accounted for by modeling metacognitive noise in 608 a process model. 609 As outlined above, the multiplicative evidence bias m and the additive evidence bias δm can be 610 interpreted as metacognitive biases. To assess the validity of these parameters, I computed individual 611 confidence biases by subtracting a participant's objective accuracy from the subjective predicted 612 accuracy (based on confidence ratings). Positive and negative values of this confidence bias are often 613 regarded as evidence for over-and underconfidence. As shown in Figures 8F and 8G Figure 9B). This negative additive evidence bias is 628 compensated by a relatively high multiplicative evidence bias m = 1.15, resulting in an average 629 confidence of 0.488 that is close to the group average (0.477 ± 0.038). 630 While below average in terms of type 1 performance, this participant excels in terms of metacognitive 631 performance. This is both indicated by a high Mratio of 1.23 (group mean ± SEM = 0.88 ± 0.05) and a low 632 metacognitive noise parameter σm of 0.06 (group mean ± SEM = 0.10 ± 0.02). 633 It is important to note that a low metacognitive noise parameter σm does not imply that the 634 participants' confidence ratings are expected to be within a narrow range for each specific stimulus 635 intensity. This is because the uncertainty of the sensory level translates to the metacognitive level: 636 the width of decision value distributions, as determined by sensory noise σs, also affects the expected 637 width of downstream confidence distributions. Indeed, the behavioral confidence distributions in 638 Figure 9C are spread out across the entire confidence range for all difficulty levels. In Figure 9C  1, a generative model described by σm, δm and m is able to approximate a wide range of idiosyncratic 654 empirical confidence distributions. 655 Figure 9. Visualization of a model fit for a single participant from Shekhar and Rahnev (2021). The applied model was a noisy-report model with a metacognitive noise distribution of the type truncated Gumbel and metacognitive evidence biases Each stimulus category in Shekhar and Rahnev (2021) was presented with three intensity levels, corresponding to values of ±1/3, ±2/3 and ±1 in normalized stimulus space (variable x). (A) Choice probability for S + as a function of stimulus intensity. The negative sensory bias δs shifts the logistic function towards the left, thereby increasing the choice probability for S + . (B) Link function, average confidence ratings and likelihood. The link function was transformed into decision value space y, for illustratory purposes. The flat range of the link function is caused by a relatively large additive evidence bias δm. Confidence ratings from empirical data (grey) and from the generative model (orange) for each stimulus levels i are indicated by their mean and standard deviation. Note that these confidence averages derive from the whole range of possible decision values and they are anchored at the most likely decision values * of each stimulus level i only for illustratory purposes. The likelihood for confidence ratings is shown only for the most likely decision values * of each stimulus level i. (C) Confidence distributions and likelihood. Empirical confidence ratings are shown as a histograms and confidence ratings obtained from the generative model as line plots. To visualize the effect of sensory uncertainty on the metacognitive level, likelihood distributions are plotted not only for the most likely values * of the decision value distributions, but also half a standard deviation below (dashed and lighter color) and above (solid and lighter color). The width of likelihood distributions is controlled by the metacognitive noise parameter σm. Distributions colored in red indicate that a sign flip of decision values has occurred, i.e. responses based on these decision values would be incorrect.

656
The present work introduces and evaluates a process model of metacognition and the accompanying 657 toolbox ReMeta (see Materials and Methods). The model connects key concepts in metacognition 658 research − metacognitive readout, metacognitive biases, metacognitive noise − with the goal of 659 providing an account of human metacognitive responses. The model can be directly applied to 660 confidence datasets of any perceptual or non-perceptual modality. 661 As any cognitive computational model, the model can serve several purposes such as inference about 662 model parameters, inference about latent variables and as a means to generate artificial data. In the 663 present work, I focused on parameter inference, in particular metacognitive parameters describing 664 metacognitive noise (σm) and metacognitive biases ( m, δm, λm, m). Indeed, I would argue that this use 665 case is one of the most pressing issues in metacognition research: characterizing the latent processes 666 underlying human confidence reports without the confound of type 1 behavior that hampers 667 descriptive approaches. 668 In the context of metacognitive biases, I have shown that the conventional method of simply 669 comparing objective and subjective performance (via confidence ratings) is flawed not only because 670 it is biased towards overconfidence, but also because it is strongly dependent on type 1 performance. 671 Just as in the case of metacognitive performance, unbiased inferences about metacognitive biases 672 thus require a process model approach. 673 Here, I introduced four metacognitive bias parameters loading either on metacognitive evidence or 674 the confidence report. As shown through the simulation of calibration curves, all bias parameters can 675 yield under-or overconfidence relative to a bias-free observer. The fact that the calibration curves 676 and the relationships between type 1 performance and confidence biases are quite distinct between 677 the proposed metacognitive bias parameters may indicate that these are to some degree dissociable. 678 Moreover, in an empirical dataset the multiplicative evidence bias m and the additive evidence bias 679 δm strongly correlated with a conventional confidence bias measure, further validating these 680 parameters. 681 The second kind of metacognitive parameter considered in this work is metacognitive noise ( confidence, which is affected by readout noise but not by reporting noise. Conditional on finding an 699 appropriate functional relationship to metacognitive evidence, reaction times could allow for an 700 unbiased inference of metacognitive readout noise or metacognitive evidence bias parameters. 701 Second, the effects of different sources of bias and noise along the processing hierarchy may be so 702 strongly correlated that their dissociation would require unreachable amounts of confidence data. 703 This dissociation, however, is essential for many research questions in metacognition − whether the 704 goal is to derive a fundamental model of human metacognition or whether one is interested in specific 705 abberrancies in mental illness. An example for the latter is the frequent observation of overconfidence 706 in schizophrenia which is thought to reflect a more general deficit in the ability to integrate 707 disconfirmatory evidence (Speechley, 2010;Zawadzki et al., 2012) and may underlie the maintenance 708 of delusional beliefs (Moritz and Woodward, 2006b). To investigate this specific hypothesis, it is 709 central to dissociate whether metacognitive biases mainly apply at the reporting stage − which may 710 be a result of the disease − or at an earlier metacognitive processing stage, which may be involved in 711 the development of the disease. This issue likewise could be addressed by measuring behavioral, 712 physiological or neurobiological processes that precede the report of confidence. 713 Third, the demonstration of an unbiased recovery of metacognitive noise and bias parameters in a 714 process model approach comes with a strong caveat, since the data is generated with the very same 715 model that is used for parameter recovery. Yet, all models are wrong, goes the saying, and this 716 certainly applies to current models of metacognition. The question is thus: given the unknown true 717 model that underlies empirical confidence ratings, to what degree can parameters obtained from an 718 approximated model be considered unbiased? The way forward here is to continuously improve 719 computational models of metacognition in terms of model evidence, thus increasing the chances that 720 fitted parameters are meaningful estimates of the true parameters. 721 With respect to previous modeling work, a recent paper by Shekhar and Rahnev (2021) deserves 722 special attention. Here too, the authors adopted a process model approach for metacognition with 723 the specific goal of deriving a measure of metacognitive ability, quite similar to the metacognitive 724 noise parameter σm in this work. One key difference is that Shekhar and Rahnev tailored their model 725 to discrete confidence scales, such that each possible confidence rating (for each choice option) is 726 associated with a separately fitted confidence criterion (as notable precursor of this idea is Adler and 727 Ma, 2018b). This introduces maximal flexibility, as essentially arbitrary mappings from internal 728 evidence to confidence can be fitted. In addition, it requires minimal assumptions about the link 729 functions that underlies the computation of confidence, apart from an ordering constraint applied to 730 the criteria. 731 However, while this flexibility is a strength, it also comes at certain costs. One issue is the relatively 732 large number of parameters that have to be fitted. Shekhar and Rahnev note that the MLE procedures 733 for the fitting of confidence criteria often got stuck in local minima. Rather than via MLE, confidence 734 criteria were thus fitted by matching the expected proportion of high confidence trials to the observed 735 proportion for each criterion. It is thus not guaranteed that the obtained confidence criterions indeed 736 maximize the likelihood under the data. Furthermore, to make a criterion-based model compatible 737 with data from a continuous confidence scale, confidence reports have to be discretized. Apart from 738 the loss of information associated with discretization, this introduces uncertainty as to how exactly 739 the data should be binned (e.g. equinumerous versus equidistant). Another aspect worth mentioning 740 is that a criterion-based approach effectively corresponds to a stepwise link function, which is not 741 invertible. Making inferences about readout noise thus poses a challenge to such criterion-based 742 models. 743 In the present work, I assumed a mapping between internal evidence and confidence that can be 744 The disproportionate frequency of the most extreme ratings for continuous confidence scales − both 754 in the empirical dataset considered here , but also in many other datasets 755 (Rahnev et al., 2020) − may indicate that there are at least two criteria at play: a finite level of 756 metacognitive evidence below which the lowest confidence rating is given (minimum confidence 757 criterion) and a level of metacognitive evidence above which the highest confidence rating is given 758 (maximum confidence criterion). Yet, even without the postulation of explicit criteria, the present 759 model can account in two possible, non-mutually exclusive, ways for this observation. The first 760 possibility is that extremal ratings are a consequence of metacognitive noise, which pushes confidence 761 ratings to the bounds. In particular, censored noise distributions can readily account for an 762 accumulation of confidence ratings at the bounds of the scale. 763 The second possibility is that the minimum and maximum confidence criteria implicitly arise as a 764 consequence of metacognitive biases (cf. section 1.4). Indeed, the individual fits to the empirical 765 dataset (Figure 8-figure supplement 1) showed that the model accounted well even for data of 766 participants with a strong preference for extremal confidence ratings, despite not considering explicit 767 confidence criteria. An exploratory analysis (not reported here) indicated that participants with such 768 confidence distributions are not well-explained by censored noise models, which may be a tentative 769 argument for the metacognitive bias explanation instead. 770 The process model approach deviates in an important way from standard analyses of confidence 771 reports based on the type 2 receiver operating curve. As type 2 ROC analyses are solely based on 772 stimulus-specific type 1 and type 2 responses, they do not consider one of the arguably most 773 important factors in this context: stimulus intensity. This implies that such measures cannot dissociate 774 to what degree variability in confidence ratings is based on stimulus variability or on internal noise. In 775 contrast, since a process model specifies the exact transformation from stimulus intensity to decision 776 variable to confidence, this source of variance is appropriately taken into account. The metacognitive 777 noise parameter σm introduced here is thus a measure of the unexpected variability of confidence 778 ratings, after accounting for the variability on the stimulus side. Note that such stimulus variability is 779 typically present even in designs with intended constant stimulus difficulty, due to the involvement of 780 randomness in the generation of unique trial-by-trial stimuli. In many cases, the effective stimulus 781 difficulty (i.e., including this random component of stimulus variability) can likewise be quantified 782 using appropriate feature-based energy detectors (see e.g., Guggenmos et al., 2016). 783 The process model approach bears another important difference compared with type 2 ROC analyses, 784 in this case a limiting factor on the side of the process model. As the area under the type 2 ROC 785 quantifies to what degree confidence ratings discriminate between correct and incorrect responses, 786 it is important to recognize what valuable piece of information the correctness of a specific response 787 is. Over and above stimulus intensity, the correctness of a response will typically be influenced by 788 negative factors such as attentional lapses, finger errors, tiredness, and positive factors such as phases 789 of increased motivation or concentration. All of these factors not only influence type 1 performance, 790 but they also influence the type 2 response that one would expect from an ideal metacognitive 791 observer. Analyses of type 2 ROCs implicitly make use of this information insofar as they consider the 792 correctness of each individual response. 793 In contrast, the information about the objective trial-by-trial accuracy is not available in a process 794 model. The signal that enters the metacognitive level of the process model is based on observed 795 variables (in particular, the sensory decision variable), but not based on the correctness of specific 796 choices, which is unknown to the observer. Note that this is not a limitation specific to the present 797 model, but the nature of process models in general. Improving process models in this regard 798 necessitates access to additional trial-by-trial data based on objective measurements (e.g. eye-799 tracking data) or subjective reports if these are not used for model fitting (e.g. reports of attentional 800 lapses or finger errors). 801 In sum, while a type 2 ROC analysis -as a descriptive approach -does not allow any conclusions about 802 the causes of metacognitive inefficiency, it is able to capture a more thorough picture of metacognitive 803 sensitivity: that is, it quantifies metacognitive awareness not only about one's own sensory noise, but 804 also about other potential sources of error (attentional lapses, finger errors, etc.). While it cannot 805 distinguish between these sources, it captures them all. On the other hand, only a process model 806 approach will allow to draw specific conclusions about mechanisms -and pin down sources -of 807 metacognitive inefficiency, which arguably is of major importance in many applications. 808 Bayesian observer, albeit one that can be susceptible to metacognitive biases and to additional 815 sources of metacognitive noise. Thus, while the observer is Bayesian in nature, it may not be Bayes 816 optimal. At the same time, the framework and the toolbox are flexible to allow for "non-Bayesian" 817 link functions (Figure 3-figure supplement 1) that could represent certain idiosyncratic heuristics and 818 shortcuts inherent to human confidence judgements. Of note, the model proposed here does not 819 consider prior distributions over the stimulus categories (see e.g., Adler and Ma, 2018b). Instead, it is 820 assumed that the observer considers both stimulus categories equally likely which is a reasonable 821 assumption if stimulus categories are balanced. 822

Conclusion 823
The model outlined in this paper casts confidence as a noisy and potentially biased transformation of 824 sensory decision values. The model parameters that shape this transformation provide a rich account 825 of human metacognitive inefficiencies and metacognitive biases. In particular, I hope that the 826 underlying framework will allow a systematic model comparison in future confidence datasets to 827 elucidate sources of metacognitive noise, to narrow down candidate noise distributions and to 828 differentiate between different kinds of metacognitive biases. The accompanying toolbox ReMeta 829 provides the functionality for such investigations. 830

831
The ReMeta toolbox 832 The code underlying this work has been bundled in a user-friendly Python toolbox (ReMeta) which is 833 published alongside this paper at github.com/m-guggenmos/remeta. While its core is identical to the 834 framework outlined here, it offers a variety of additional parameters and settings. In particular, it 835 allows fitting separate values for each parameter depending on the sign of the stimulus (for sensory 836 parameters) or the decision value (for metacognitive parameters). Moreover, it offers various choices 837 for noise distributions and link functions, including criterion-based link functions. 838 The ReMeta toolbox has a simplified interface such that in the most basic case it requires only three 839 data vectors as input: stimuli, choices and confidence. The output is a structure containing the fitted 840 parameters, information about the goodness of fit (log-likelihood, AIC, BIC, correlation between 841 empirical confidence ratings and ratings from a generative model) and optionally vectors of all latent 842 variables (e.g. decision values, metacognitive evidence). The toolbox is highly configurable − in 843 particular each parameter can be disabled, enabled or enabled in duplex mode (i.e. sign-dependent, 844 see above). 845 Parameter fitting minimizes the negative log-likelihood of type 1 choices (sensory level) or type 2 846 confidence ratings (metacognitive level). For the sensory level, initial guesses for the fitting procedure 847 were found to be of minor importance and are thus set to reasonable default values. Data are 848 generally fitted with a gradient-based optimization method (Sequential Least Squares Programming; 849 Kraft, 1988). However, if enabled, the sensory threshold parameter can introduce a discontinuity in 850 the psychometric function, thereby violating the assumptions of gradient methods. In this case, an 851 additional gradient-free method (Powell's method; Powell, 1964