Integrative and distinctive coding of perceptual and conceptual object features in the ventral visual stream

A tremendous body of research in cognitive neuroscience is aimed at understanding how object concepts are represented in the human brain. However, it remains unknown whether and where the visual and abstract conceptual features that define an object concept are integrated. We addressed this issue by comparing the neural pattern similarities among object-evoked fMRI responses with behavior-based models that independently captured the visual and conceptual similarities among these stimuli. Our results revealed evidence for distinctive coding of visual features in lateral occipital cortex, and conceptual features in the temporal pole and parahippocampal cortex. By contrast, we found evidence for integrative coding of visual and conceptual object features in perirhinal cortex. The neuroanatomical specificity of this effect was highlighted by results from a searchlight analysis. Taken together, our findings suggest that perirhinal cortex uniquely supports the representation of fully-specified object concepts through the integration of their visual and conceptual features.


Introduction 36
Semantic memory imbues the world with meaning and shapes our understanding of the 37 relationships among object concepts. Many neurocognitive models of semantic memory 38 incorporate the notion that object concepts are represented in a feature-based manner (Rosch and 39 Mervis, 1975;Tyler and Moss, 2001; Rogers and McLelland, 2004). On this view, our 40 understanding of the concept "hairdryer" is thought to reflect knowledge of observable 41 perceptual properties (e.g., visual form) and abstract conceptual features (e.g., "used to style 42 hair"). Importantly, however, there is not always a one-to-one correspondence between how 43 something looks and what it is; a hairdryer and a comb are conceptually similar despite being 44 visually distinct, whereas a hairdryer and a gun are conceptually distinct despite being visually 45 similar. Thus, a fully-specified representation of an object concept requires integration of its 46 perceptual and conceptual features. 47 4 SD is characterized by the progressive loss of conceptual knowledge across all receptive and 59 expressive modalities (Warrington, 1975;Hodges et al., 1992). At the level of neuropathology, 60 SD is associated with extensive atrophy of the ATL, with the earliest and most pronounced 61 volume loss in the left temporal pole (Mummery et al., 2000;Galton et al., 2001). Most 62 important from a theoretical perspective, patients with SD tend to confuse conceptually similar 63 objects that are visually distinct (e.g., hairdryercomb), but not visually similar objects that are 64 conceptually distinct (e.g., hairdryergun), indicating that the temporal pole expresses 65 conceptual similarity structure (Graham et al., 1994; see Peelen and Caramazza, 2012;Chadwick 66 et al., 2016, for related neuroimaging evidence). Taken together, these findings suggest that the 67 temporal pole supports multi-modal integration of abstract conceptual, but not perceptual, 68 features. Notably, however, a considerable body of research indicates that the temporal pole may 69 not be the only ATL structure that supports feature-based integration. 70 The representational-hierarchical model of object coding emphasizes a role for perirhinal cortex 71 (PRC), located in the medial ATL, in feature integration that is distinct from that of the temporal 72 pole (Murray and Bussey, 1999). Namely, within this framework PRC is thought to support the 73 integration of conceptual and perceptual features. In line with this view, object representations in 74 part, because conceptual and perceptual features tend to vary concomitantly across stimuli (Mur, 80 2014). For example, demonstrating greater neural pattern similarity in PRC between "horse" and 81 "donkey" than between "horse" and "dolphin" may reflect differences in conceptual or 82 perceptual relatedness. Moreover, because studies linking PRC to the integration of visual 83 features have primarily used pictorial stimuli, it remains unclear whether this result will hold in 84 tasks that require assessment of visual features retrieved from semantic memory. Thus, although 85 the representational-hierarchical account was initially formalized nearly two decades ago 86 (Murray and Bussey, 1999), direct evidence of integration across conceptual and perceptual 87 features remains elusive. 88 In the current study, we used fMRI to characterize the representational structure of object 89 concepts in the brain. More specifically, we sought to determine whether and where conceptual 90 features are integrated with perceptual features, with an emphasis on visual semantics. This issue 91 was probed, for the first time, using representational similarity analysis (RSA) (Kriegeskorte and 92 Kievit, 2013) and a set of object concepts that were selected to ensure that conceptual similarity 93 was not confounded with visual similarity. In a first step, we generated behavior-based models 94 that captured the conceptual and visual similarities among these object concepts. Next, we 95 scanned participants using task contexts that emphasized processing of either the conceptual or 96 perceptual features of these objects. We hypothesized that both behavior-based models would 97 predict the neural pattern similarities between object concepts, regardless of task context, in 98 brain regions that support the integration of conceptual and perceptual features. Based on the 99 neurocognitive models reviewed above, we anticipated that this result would be uniquely 100 obtained in PRC (Murray and Bussey, 1999; Barense et al., 2011). In addition to PRC, our 101 analysis also probed regions of interest (ROIs) that have been implicated in semantic processing 102 (temporal pole and parahippocampal cortex), and visual processing (lateral occipital cortex) 103 similarity judgments between object concepts ( Figure 1A). Specifically, a pair of words was 114 presented on each trial and participants were asked to rate the visual similarity between the 115 object concepts to which they referred using a 5-point Likert scale. Similarity ratings for each 116 pair of object concepts were averaged across participants, normalized, and expressed within a 117 representational dissimilarity matrix (RDM). We refer to this RDM as the behavior-based visual 118

RDM. 119
For the purpose of constructing the conceptual similarity model, a second group of participants 120  Figure 1B). Each participant was asked to generate a list of conceptual features that characterize 122 one object concept (e.g., hairdryer: "used to style hair", "found in salons", "electrically 123 powered", "blows hot air"; comb: "used to style hair", "found in salons", "has teeth", "made 124 of plastic"). Conceptual similarity between all pairs of object concepts was quantified as the 125 cosine angle between the corresponding pairs of feature vectors. With this approach, high cosine 126 similarity between object concepts reflects high conceptual similarity. Cosine similarity values 127 were then expressed within an RDM, which we refer to as the behavior-based conceptual RDM. 128 We next performed a second-level RSA to quantify the relationship between our behavior-based 129 visual RDM and behavior-based conceptual RDM. Critically, this analysis revealed that the 130 model RDMs were not significantly correlated with one another (Kendall's tau-a = .01, p = .09), 131 indicating that differences in visual and conceptual features were not confounded across object 132 concepts. In other words, ensuring that these different types of features varied independently 133 across stimuli (e.g., hairdryergun; hairdryercomb), rather than concomitantly (e.g., horse -134 donkey; horsedolphin), allowed us to isolate the separate influence of visual and conceptual 135 features on the representational structure of object concepts in the brain. In this example, a 136 hairdryer and a gun are visually similar but conceptually dissimilar, whereas a hairdryer and a 137 comb are visually dissimilar but conceptually similar. 138 139 fMRI Task and Behavioral Results 140 We next used fMRI to obtain measurements from which we could infer the representational 141 structure of our 40 object concepts in the neural activity patterns of a third independent group of 142 participants ( Figure 2). Functional brain data were acquired over eight experimental runs, each of 143 which consisted of two blocks of stimulus presentation. All 40 object concepts were presented 144 sequentially within each block, for a total of 16 repetitions per concept. On each trial, 145 participants were asked to make a "yes / no" property verification judgment in relation to a 146 block-specific verification probe. Half of the blocks were associated with verification probes that 147 encouraged processing of visual features (e.g., "is the object angular?"), and the other half were 148 associated with verification probes that encouraged processing of conceptual features (e.g., "is 149 the object a tool?"). With this experimental design, we were able to characterize neural responses 150 to object concepts across two task contexts: a visual task context ( Figure 2A) and a conceptual 151 task context ( Figure 2B). 152 Behavioral performance on the scanned property verification task indicated that participants 153 interpreted the object concepts and property verification probes with a high degree of 154 consistency ( Figure 3). Specifically, all participants (i.e., 16/16) provided the same yes/no 155 response to the property verification task on 88.4% of all trials. Agreement was highest for the 156 "living" verification probe (96.8%) and lowest for the "non-tool" verification probe (73.2%). 157 Moreover, the proportion of trials on which all participants provided the same response did not 158 differ between the visual feature verification task context (Mean = 87.3% collapsed across all 159 eight visual probes) and the conceptual feature verification task context (Mean = 89.5% 160 collapsed across all eight conceptual probes) (z = 0.19, p = .85). Response latencies were also 161 comparable across the visual feature verification task context (Mean = 1361ms, SD = 302) and 162 the conceptual feature verification task context (Mean = 1388ms, SD = 317) (t (15) = 0.61, p = 163 .55). 164 165

ROI-Based RSA: Comparison of Behavior-Based RDMs with Brain-Based RDMs 166
We next quantified pairwise similarities between multi-voxel activity patterns evoked by specific 167 object concepts in the fMRI experiment ( Figure 2). For the purpose of conducting ROI-based 168 RSA, we focused on multi-voxel activity patterns obtained in PRC, the temporal pole, 169 parahippocampal cortex, and LOC. ROIs from a representative participant are presented in 170 Mean object-specific multi-voxel activity patterns were estimated in each ROI using general 188 linear models fit to data from the visual and conceptual task contexts, separately. Linear 189 correlation distances (Pearson's r) were calculated between all pairs of object concepts, which 190 were then expressed in two brain-based RDMs for each ROI. Specifically, the brain-based visual 191 task RDM captured the neural pattern similarities obtained between all object concepts in the 192 visual task context (i.e., while participants made visual feature verification judgments) ( Figure  193 2A), and the brain-based conceptual task RDM captured the neural pattern similarities obtained 194 between all object concepts in the conceptual task context (i.e., while participants made 195 conceptual feature verification judgments) ( Figure 2B). 196 We implemented second-level RSA to compare our behavior-based visual and conceptual RDMs 197 (i.e., independent dissimilarity models) with the brain-based visual and conceptual task RDMs 198 (i.e., neural pattern dissimilarity obtained in different verification task contexts) (solid arrows in 199 Figure 5). These analyses were conducted in each ROI using a ranked correlation coefficient 200 (Kendall's tau-a) as a similarity index (Nili et al., 2014). Significance testing was performed 201 using non-parametric permutation tests for all pertinent comparisons. A Bonferroni correction 202 was applied to compensate for multiple comparisons (4 ROIs x 2 behavior-based RDMs x 2 203 brain-based RDMs = 16 comparisons, yielding a critical alpha of .003). With this approach, we 204 revealed that object concepts are represented by three distinctive similarity codes that differed 205 across ROIs: visual similarity coding, conceptual similarity coding, and integrative coding. 206 Results from our ROI-based RSA analyses are shown in Figure 6 and discussed in turn below. 207

Lateral Occipital Cortex Represents Object Concepts with a Visual Similarity Code 208
Consistent with its well-established role in the processing of visual form, patterns of activity 209 within LOC reflected the visual similarity of the object concepts ( Figure 6). Specifically, we 210 obtained a significant correlation between the behavior-based visual RDM and the brain-based 211 visual task RDM in LOC (Kendall's tau-a = .05, p < .0001). Notably, however, the correlation 212 between the behavior-based visual RDM and the brain-based conceptual task RDM was not 213 significant (Kendall's tau-a = .01, p = .20). In other words, activity patterns in LOC expressed a 214 visual similarity structure when participants were asked to make explicit judgments about the 215 visual features that characterized object concepts (e.g., whether an object is angular in form), but 216 not when those judgments pertained to features that were conceptual in nature (e.g., whether an 217 object is naturally occurring). Conversely, the behavior-based conceptual RDM did not 218 significantly correlate with the brain-based visual task RDM (Kendall's tau-a = .002, p = .45) or 219 brain-based conceptual task RDM (Kendall's tau-a = -.016, p = .87), indicating that conceptual 220 similarities between object concepts did not capture neural pattern similarities in LOC in either 221 task context. Considered together, these results suggest that LOC represents perceptual 222 information about object concepts in a task-dependent visual similarity code that generalizes 223 across visually similar object concepts that are conceptually distinct (e.g., hairdryergun), but 224 not across conceptually similar object concepts that are visually distinct (e.g., hairdryercomb). 225

Similarity Code 227
In line with theoretical frameworks that have characterized the temporal pole as a semantic hub 228 Tranel et al., 2009), patterns of activity within this specific ATL structure 229 reflected the conceptual similarity of the object concepts ( Figure 6). Specifically, in the temporal 230 pole we revealed a significant correlation between the behavior-based conceptual RDM and the 231 brain-based conceptual task RDM (Kendall's tau-a = .06, p < .0001). The behavior-based 232 conceptual RDM was also significantly correlated with the brain-based visual task RDM 233 (Kendall's tau-a = .04, p < .0001). Thus, the temporal pole expressed a conceptual similarity 234 structure regardless of whether participants were asked to make targeted assessments of 235 conceptual features (e.g., whether the object is a tool) or visual features (e.g., whether it is 236 symmetrical). The behavior-based visual RDM was not significantly correlated with either the 237 brain-based conceptual task RDM (Kendall's tau-a = .01, p = .19) or the brain-based visual task 238 RDM (Kendall's tau-a = -.001, p = .55), suggesting that the representational structure of object 239 concepts in the temporal pole is not shaped by visual properties. 240 Patterns of activity obtained in parahippocampal cortex, which has previously been associated 241 with the processing of semantically-based contextual associations (Bar and Aminoff, 2003), also reflected the conceptual similarity of the object concepts ( Figure 6). Unlike the temporal pole, 243 however, parahippocampal cortex expressed conceptual similarity structure in a task-specific 244 manner. Specifically, the behavior-based conceptual RDM was significantly correlated with the 245 brain-based conceptual task RDM (Kendall's tau-a = .06, p < .0001), but not the brain-based 246 visual task RDM (Kendall's tau-a = .02, p = .10). The behavior-based visual RDM was not a 247 significant predictor of neural dissimilarity structure captured by either the brain-based visual 248 task RDM (Kendall's tau-a = .002, p = .42) or the brain-based conceptual task RDM (Kendall's 249 tau-a = .009, p = .22). 250 In sum, these results suggest that the temporal pole and parahippocampal cortex represent 251 conceptual information in a manner that enables efficient generalization across conceptually 252 related object concepts that are visually distinct (e.g., hairdryercomb), but not visually related 253 object concepts that are conceptually distinct (e.g., hairdryergun). That is, the degree of 254 similarity between object-evoked activity patterns in these structures reflected the degree of 255 conceptual feature overlap, but not visual feature overlap, between those object concepts. 256 Notably, the temporal pole expressed this conceptual similarity code even when the information 257 that it conveyed was orthogonal to task demands. For example, hairdryer and comb were 258 represented more similarly than were hairdryer and gun, even when task demands encouraged 259 processing of visual features in the visual task context. Conversely, our results suggest that 260 parahippocampal cortex expresses conceptual similarity structure only when task demands 261 prioritize processing of conceptual information in the conceptual feature verification task. 262

Conceptual and Visual Features 264
Results obtained in PRC support the notion that this structure integrates conceptual and visual 265 object features, as first theorized in the representational-hierarchical model of object 266 representation (Murray and Bussey, 1999). Namely, we revealed that the behavior-based visual 267 RDM and the behavior-based conceptual RDM were each significantly correlated with both the 268 brain-based visual task RDM (behavior-based visual RDM Kendall's tau-a = .07, p < .0001; 269 behavior-based conceptual RDM Kendall's tau-a = .05, p < .0001), and the brain-based 270 conceptual task RDM (behavior-based visual RDM Kendall's tau-a = .04, p < .001; behavior-271 based conceptual RDM Kendall's tau-a = .07, p < .0001) ( Figure 6). These findings indicate that 272 PRC simultaneously expressed both conceptual and visual similarity structure, and did so 273 regardless of whether participants were asked to make targeted assessments of conceptual 274 features (e.g., whether the object concept is living) or visual features (e.g., whether it is 275 elongated). In other words, activity patterns in PRC captured the conceptual similarity between 276 hairdryer and comb, as well as the visual similarity between hairdryer and gun, and did so 277 irrespective of task context. Critically, these results were obtained despite the fact that the brain-278 based RDMs were orthogonal to one another (i.e., not significantly correlated). Considered 279 together, these results suggest that, of the a priori ROIs considered, PRC represents object 280 concepts at the highest level of specificity through integration of visual and conceptual features. 281

ROI-Based RSA: Comparisons of Brain-Based RDMs with Brain-Based RDMs 283
We next implemented an additional second-level RSA in which we directly compared object-284 evoked neural similarity patterns within and across our four a priori ROIs. These analyses were 285 conducted using the same methodological procedures applied to compare behavior-based RDMs 286 with brain-based RDMs. We first sought to quantify the representational similarity between the 287 brain-based visual task RDM and brain-based conceptual task RDM obtained within each ROI. 288 This comparison is denoted by the dashed horizontal arrow in the bottom of Figure 5. Notably, 289 these brain-based RDMs were significantly correlated with one another in PRC (Kendall's tau-a 290 = .06, p < .001), but not in the temporal pole (Kendall's tau-a = .01, p = .28), parahippocampal 291 cortex (Kendall's tau-a = -.01, p = .69), or LOC (Kendall's tau-a = .02, p = .18). This result 292 suggests that PRC emphasized similar representational distinctions between object concepts 293 regardless of whether those concepts were processed in the context of a visual or conceptual task 294 context. 295 In a second set of analyses, we examined whether activity in different ROIs reflected similar 296 representational distinctions across object concepts within the same task context. To this end, we 297 first compared the brain-based visual task RDM obtained in a given ROI with those obtained in 298 all other ROIs. For example, we asked whether the brain-based visual task RDMs obtained in 299 PRC and LOC were significantly correlated with one another for the visual task context. 300 Interestingly, these analyses did not reveal any significant results between any of our ROIs (all 301 Kendall's tau-a < .029, all p > .12). These findings indicate that PRC and LOC, two regions that 302 expressed a visual similarity code as revealed through comparison with the behavior-based visual 303 RDM ( Figure 3B), emphasized different visually-based representational distinctions between 304 object concepts. 305 We next compared the brain-based conceptual task RDM obtained in a given ROI with those 306 obtained in all other ROIs. For example, we asked whether the brain-based conceptual task 307 RDMs obtained in PRC and the temporal pole, were significantly correlated with one another. 308 This set of analyses revealed a trend toward a positive correlation between PRC and 309 parahippocampal cortex (Kendall's tau-a = .05, p < .01, corrected critical alpha = .003), but no such relationship between any other ROIs (all Kendall's tau-a < .034, all p > .08). These findings 311 suggest that although the brain-based conceptual task RDMs obtained in PRC, parahippocampal 312 cortex, and the temporal pole were all significantly correlated with the behavior-based 313 conceptual RDM, they may emphasize different conceptually-based representational distinctions 314 between object concepts. 315 316

Perirhinal Cortex is the Only Cortical Region that Supports Integrative Coding of Conceptual 318 and Visual Object Features 319
We next implemented a whole-volume searchlight-based RSA to investigate the neuroanatomical 320 specificity of our ROI-based results. Specifically, we sought to determine whether object 321 representations in PRC expressed visual and conceptual similarity structure within overlapping 322 or distinct populations of voxels. If PRC does indeed support the integrative coding of visual and 323 conceptual object features, then the same subset of voxels in this structure should express both 324 types of similarity codes. If PRC does not support the integrative coding of visual and conceptual 325 object features, then different subsets of voxels should express these different similarity codes. 326 More generally, data-driven searchlight mapping allowed us to explore whether any other 327 regions of the brain showed evidence for integrative coding of visual and conceptual features in a 328 manner comparable to that observed in PRC. To this end we performed searchlight RSA using 329 multi-voxel activity patterns restricted to a 100 voxel ROI that was iteratively swept across the 330 entire cortical surface (Kriegeskorte et al., 2006;Oosterhof et al., 2011). In each searchlight ROI, 331 the behavior-based RDMs were compared with the brain-based RDMs using a procedure 332 identical to that implemented in our ROI-based RSA. These comparisons are depicted by the 333 solid black arrows in Figure 5. The obtained similarity values (Pearson's r) were mapped to the 334 center of each ROI for each participant separately. With this approach, we obtained participant-335 specific similarity maps for all comparisons, which were then standardized and subjected to a 336 group-level statistical analysis. A threshold-free cluster enhancement (TFCE) method was used 337 to correct for multiple comparisons with a cluster threshold of p < 0.05 (Smith and Nichols, 338 2009). 339 All searchlight results are depicted in Figure 7, with corresponding cluster statistics, co-340 ordinates, and anatomical labels reported in Table 1. Statistically thresholded group-level 341 similarity maps are presented in Figure 6A for comparison of both behavior-based RDMs with 342 the brain-based visual task RDM, and in Figure 6C for comparison of both behavior-based 343 RDMs with the brain-based conceptual task RDM. To determine whether PRC expressed visual 344 similarity structure and conceptual similarity structure in overlapping or distinct sets of voxels, 345 we examined the extent of voxel overlap across similarity maps. In a first step, we asked whether 346 there were any common voxels across the similarity maps obtained within each task context, 347 separately. Overlapping voxels across similarity maps obtained through comparison of behavior-348 based RDMs with the brain-based RDM derived from the visual task context are presented in 349 Figure 6B. Overlapping voxels across similarity maps obtained through comparison of behavior-350 based RDMs with the brain-based RDM derived from the conceptual task context are presented 351 in Figure 6D. Within each task context, we revealed a contiguous cluster of voxels in left PRC in 352 which both behavior-based RDMs predicted task-specific brain-based RDMs. 353 In a second step, we examined whether any voxels were common across the task-specific 354 overlapping clusters. In other words, we asked whether both behavior-based RDMs were able to 355 describe the both brain-based RDMs derived from a common set of voxels (as depicted by the black arrows in Figure 6E). Critically, left PRC was the only region in the entire scanned volume 357 in significant clusters of voxels overlapped across all similarity maps ( Figure 6E). This result 358 indicates that a subset of voxels within PRC simultaneously expressed both visual and 359 conceptual similarity structure, suggesting that this structure does indeed support integration of 360 the visual and conceptual features that define an object concept. has been hindered by the fact that such features tend to vary concomitantly across object 369 concepts. Here, we used a data-driven approach to systematically select a set of object concepts 370 in which visual and conceptual features varied independently (e.g., hairdryercomb, which are 371 conceptually but not visually similar; hairdryergun, which are visually but not conceptually 372 similar). By comparing behavior-based models of the visual and conceptual similarity structure 373 of these object concepts with corresponding brain-based similarity structure we revealed novel 374 evidence for an integrative coding process that binds conceptual object features with observable 375 perceptual features in a task-invariant manner. This integrative coding, which we uniquely found 376 in PRC, may guide complex behavior through the representation of objects and object concepts 377 at the highest level of specificity. Moreover, we also revealed a representational distinction 378 between PRC and the temporal pole as they relate to semantic memory. Namely, whereas PRC 379 showed evidence of integrative coding across conceptual and visual features, neural activity 380 patterns in the temporal pole were best understood in relation to a purely conceptual code. Taken 381 together, these findings provide a first step toward filling a theoretically important gap in the 382 cognitive neuroscience of semantic memory and object representation, more broadly. 383 Our central finding is that patterns of activity within PRC reflected both the visual and 384 conceptual similarities between object concepts. We interpret this result as evidence for 385 integration for reasons directly related to our experimental design. First, the behavior-based 386 visual RDM and behavior-based conceptual RDM (i.e., the models) used in the current study 387 were not correlated with one another, indicating that these models accounted for different 388 sources of variability in the relationships among the object concepts. For example, the behavior-389 based conceptual RDM captured a relationship between "hairdryer" and "comb", where none 390 existed in the behavior-based visual RDM. Second, and despite the fact that these behavior-based 391 RDMs were orthogonal to one another, they could each be used to describe the brain-based 392 RDMs derived from both the visual and conceptual task contexts. Critically, across our ROI-393 based RSA and our searchlight analysis, PRC was the only region in which we obtained this 394 pattern of results. At the level of interpretation, the importance of these points is perhaps best 395 illustrated with an example from our experiment. Specifically, our results indicated that while 396 participants made conceptual judgments about objects in the fMRI scanner, such as whether a 397 "hairdryer" is man-made or a "gun" is pleasant, the corresponding degree of neural pattern 398 similarity between "hairdryer" and "gun" could be captured by their perceptual similarity, as 399 indexed by behavioral ratings from an independent group of observers. Likewise, when 400 participants made perceptual judgments about object concepts in the fMRI task, such as whether 401 a "hairdryer" is angular or a "comb" is elongated, the corresponding degree of neural pattern 402 similarity between "hairdryer" and "comb" could be captured by their conceptual similarity, as 403 derived from responses provided by an independent group of participants. In both cases, PRC 404 carried information about semantic features that were neither required to perform the immediate 405 task at hand, nor correlated with the features that did in fact have task-relevant diagnostic value. 406 Moreover, results from our RSA-based searchlight mapping analysis indicated that a contiguous 407 cluster of voxels in left PRC was the only region in the brain that showed this effect. Thus, 408 despite the fact that we disentangled conceptual and perceptual feature overlap across objects 409 and imposed task demands that biased processing toward one class of feature or the other, both 410 types of information were ubiquitous and inseparable in PRC. When considered together, these 411 results suggest that, at the level of PRC, it may not be possible to fully disentangle conceptual 412 and perceptual information. convergence zone that abstracts conceptual information from the co-occurrence of features 458 otherwise represented in a distributed manner across modality-specific cortical nodes. Consistent 459 with this idea, we have shown that a behavior-based conceptual similarity model predicted the 460 similarity structure of neural activity patterns in the temporal pole, irrespective of task context. 461 Specifically, neural activity patterns associated with conceptually similar object concepts that are 462 visually distinct (e.g., "hairdryer" -"comb") were more comparable than were conceptually 463 dissimilar concepts that are visually similar (e.g., "hairdryer" -"gun"), even when task demands 464 In summary, we used fMRI to characterize the representational structure of object concepts in 473 the brain. Specifically, we generated behavior-based models that independently captured the 474 conceptual and visual similarities among a targeted set of object concepts and used these models 475 to predict brain-based neural similarities across two task contexts. Using this approach we 476 revealed three distinct types of coding of object concepts. First, we found that LOC represented 477 object concepts in a visually-based similarity code. Second, we found that the temporal pole and 478 parahippocampal cortex represented object concepts in a conceptually-based similarity code, but 479 that the temporal pole did so in a task invariant manner, whereas parahippocampal cortex only 480 did so in the context of explicit conceptual feature judgments. Critically, and despite the fact that 481 our visual and conceptual similarity models were not correlated with one another, we found that 482 PRC uniquely supported the integrative coding of perceptual and conceptual features in a task 483 invariant manner. At a broad level, our results suggest that PRC supports the representation of 484 fully-specified object concepts in which perceptual and conceptual information is integrated. to technical problems, we were unable to obtain data from one experimental run in two different 505 participants. No participants were removed due to excessive motion using a criterion of 1.5mm 506 of translational displacement. All participants gave informed consent, reported that they were 507 native English speakers, free of neurological and psychiatric disorders, and had normal or 508 corrected to normal vision. Participants were compensated $50. This study was approved by the 509 Baycrest Hospital Research Ethics Board. 510 511 Stimuli 512 As a starting point, we chained together a list of 80 object concepts in such a way that adjacent 513 items in the list alternated between being conceptually similar but visually distinct and visually 514 similar but conceptually distinct (e.g., bulletgunhairdryercomb; bullet and gun are 515 conceptually but not visually similar, whereas gun and hairdryer are visually but not 516 conceptually similar, and hairdryer and comb are conceptually but not visually similar, etc.). Our 517 initial stimulus set was established using the authors' subjective impressions. The visual and 518 conceptual similarities between all pairs of object concepts were then quantified by human 519 observers in the context of a visual similarity rating task and a conceptual feature generation 520 task, respectively. Results from these behavioral tasks were then used to select 40 object 521 concepts used throughout the current study. 522 Participants who completed the visual similarity rating task were presented with 40 pairs of 523 words and asked to rate visual similarity between the object concepts to which they referred 524 ( Figure 1A). Responses were made using a 5-point scale (very dissimilar, somewhat dissimilar, 525 neutral, somewhat similar, very similar). Each participant was also presented with four catch 526 trials on which an object concept was paired with itself. Across participants, 95.7% of catch trials 527 were rated as being very similar. Data were excluded from 28 participants who did not rate all 528 four catch trials as being at least 'somewhat similar'. Every pair of object concepts from the 529 initial set of 80 object concepts (3160) was rated by 15 different participants. 530 We next quantified conceptual similarities between object concepts based on responses obtained 531 in a conceptual feature generation task ( Figure 1B We used a data-driven approach to select a subset of 40 object concepts from the initial 80-item 547 set. These 40 object concepts are reflected in the behavior-based visual and conceptual RDMs, 548 and were used as stimuli in our fMRI experiment. Specifically, we first ensured that each object 549 concept was visually similar, but conceptually dissimilar, to at least one other item (e.g., 550 hairdryergun), and conceptually similar, but visually dissimilar, to at least one different item 551 (e.g., hairdryercomb). Second, in an effort to ensure that visual and conceptual features varied 552 independently across object concepts, stimuli were selected such that the corresponding 553 behavior-based visual and conceptual similarity models were not correlated with one another. 554 555

Behavior-Based RDMs 556
Behavior-Based Visual RDM 557 A behavior-based model that captured visual dissimilarities between all pairs of object concepts 558 included in the fMRI experiment (40 object concepts) was derived from the visual similarity 559 judgments obtained from our online rating task. Specifically, similarity ratings for each pair of 560 object concepts were averaged across participants, normalized, and expressed within a 40x40 561 RDM (1averaged normalized rating). Thus, the value in a given cell of this RDM reflects the 562 visual similarity of the object concepts at that intersection. This behavior-based visual RDM is 563 our visual dissimilarity model. 564

Behavior-Based Conceptual RDM 565
A behavior-based model that captured conceptual dissimilarities between all pairs of object 566 concepts included in the fMRI experiment was derived from data obtained in our online feature-567 generation task. In order to ensure that the semantic relationships captured by our conceptual 568 similarity model were not influenced by verbal descriptions of visual attributes, we 569 systematically removed features that characterized either visual form or color (e.g., "is round" or 570 "is red"). Using these criteria a total of 58 features (8% of the total number of features provided) 571 were removed. We next quantified conceptual similarity using a concept-feature matrix in which 572 rows corresponded to object concepts (i.e., 40 rows) and columns to conceptual features (i.e., 573 723 features -58 visual features = 665 columns) ( Figure 1B, center). Specifically, we computed 574 the cosine angle between each row; cosine similarity reflects the conceptual distances between 575 object concepts such that high cosine similarities between items denote short conceptual 576 distance. The conceptual dissimilarities between all pairs of object concepts were expressed as a 577 40 x 40 RDM. The value within each cell of the conceptual model RDM was calculated as 1 -578 the cosine similarity value between the corresponding object concepts. This behavior-based 579 conceptual RDM is our conceptual dissimilarity model. 580

Behavior-Based RSA: Comparison of Behavior-Based RDMs 582
We next quantified similarity between our behavior-based visual RDM and behavior-based 583 conceptual RDM using Kendall's tau-a as the relatedness measure. This ranked correlation 584 coefficient is the most appropriate inferential statistic to use when comparing sparse RDMs that 585 predict many tied ranks (i.e., both models predict complete dissimilarity between many object 586 pairs; Nili et al., 2014). Inferential analysis of model similarity was performed using a stimulus-587 label randomization test (10,000 iterations) that simulated the null hypothesis of unrelated RDMs 588 (i.e., zero correlation) based on the obtained variance. Significance was assessed through 589 comparison of the obtained Kendall's tau-a coefficient to the equivalent distribution of ranked 590 null values. As noted in the Results section, this analysis revealed that our behavior-based visual 591 and conceptual RDMs were not significantly correlated (Kendall's tau-a = .01, p = .09). 592 Moreover, inclusion of the 58 features that described color and visual form in the behavior-based 593 conceptual RDM did not significantly alter its relationship with the visual behavior-based visual 594 RDM (Kendall's tau-a = .01, p = .09). 595 596

Experimental Procedures: fMRI Feature Verification Task 597
During scanning, participants completed a feature verification task that required a yes/no 598 judgment indicating whether a given feature was applicable to a specific object concept on a 599 trial-by-trial basis. We systematically varied the feature verification probes in a manner that 600 established a visual feature verification task context and conceptual feature verification task 601 context. Verification probes comprising the visual task context were selected to encourage 602 processing of the visual semantic features that characterize each object concept (i.e., shape, 603 color, and surface detail). To this end, eight specific probes were used: shape [(angular, 604 rounded), (elongated, symmetrical)], color (light, dark), and surface (smooth, rough). Notably, 605 all features are associated with two opposing probes (e.g., angular and rounded; natural and 606 manufactured) to ensure that participants made an equal number of "yes" and "no" responses. 607 Verification probes comprising the conceptual feature verification task context were selected to 608 encourage processing of the abstract conceptual features that characterize each object concept 609 (i.e., animacy, origin, function, and affective associations). To this end, eight specific verification 610 probes were used: (living, non-living), (manufactured, natural), (tool, non-tool), (pleasant, 611 unpleasant). 612 Procedures 613 The primary experimental task was evenly divided over eight runs of functional data acquisition. 614 Each run lasted 7m 56s and was evenly divided into two blocks, each of which corresponded to 615 either a visual verification task context or a conceptual feature verification task context. The 616 order of task blocks was counter-balanced across participants. Each block was associated with a 617 different feature verification probe, with the first and second block in each run separated by 12s 618 of rest. Blocks began with an 8s presentation of a feature verification probe that was to be 619 referenced for all intra-block trials. With this design, each object concept was repeated 16 times: 620 eight repetitions across the visual feature verification task context and eight repetitions across the 621 conceptual feature verification task context. Behavioral responses were recorded using an MR-622 compatible keypad. 623 Stimuli were centrally presented for 2s and each trial was separated by a jittered period of 624 baseline fixation that ranged 2-6s. Trial order and jitter interval were optimized for each run 625 Following completion of the main experimental task, each participant completed an independent 631 functional localizer scan that was subsequently used to identify LOC. Participants viewed 632 objects, scrambled objects, words, scrambled words, faces, and scenes in separate 24s blocks (12 functional volumes). Within each block, 32 images were presented for 400ms each with a 350ms 634 ISI. There were four groups of six blocks, with each group separated by a 12s fixation period, 635 and each block corresponding to a different stimulus category. Block order (i.e., stimulus 636 category) was counterbalanced across groups. All stimuli were presented in the context of a 1-637 back task to ensure that participants remained engaged throughout the entire scan. Presentation 638 of images within blocks was pseudo-random with 1-back repetition occurring 1-2 times per 639 block. Images were initially skull-stripped using a brain extraction tool (BET, Smith, 2002) to remove 674 non-brain tissue from the image. Data were then corrected for slice-acquisition time, high-pass 675 temporally filtered (using a 50s period cut-off for event-related runs, and a 128s period cut-off 676 for the blocked localizer run), and motion corrected (MCFLIRT, Jenkinson et al., 2002).

ROI-Based RSA: Comparisons of Behavior-Based RDMs with Brain-Based RDMs and Brain-692
Based RDMs with Brain-Based RDMs 693 We used linear correlations to quantify the participant-specific dissimilarities (1 -Pearson's r) 694 between all object-evoked multi-voxel activity patterns (n = 40) with each ROI (n = 4). 695 Dissimilarity measures were expressed in 40x40 RDMs for each run (n = 8) and verification task 696 context (n = 2), separately. Thus, for each ROI, each participant had eight RDMs that reflected 697 the (dis)similarity structure from the visual feature verification task context, and eight RDMs that 698 reflected the (dis)similarity structure from the conceptual verification task context. We then 699 calculated one mean RDM for each feature verification task context by averaging run-specific 700 RDMs across participants. Thus, one brain-based RDM was created for the visual task context 701 (i.e., brain-based visual task RDM) and one brain-based RDM was created for the conceptual 702 task context (i.e., brain-based conceptual task RDM). 703 We next examined how well each of the behavior-based RDMs fit each of the obtained brain-704 based RDMs for each ROI. Model fit was quantified as the ranked correlation coefficient 705 (Kendall's tau-a) between behavior-based RDMs and the brain-based RDMs. Significance 706 testing was performed using a stimulus-label randomization test (10,000 iterations per model) 707 Bonferroni corrected for multiple comparisons. were also quantified using the same approach. The correlation coefficients obtained between 715 behavior-based RDMs and brain-based RDMs were then Fisher-z transformed and mapped to the 716 voxel at the centre of each searchlight to create a whole-brain similarity map. Participant-specific 717 similarity maps were then normalized to a standard MNI template using FNIRT (Greve and 718 Fischl, 2009). To assess the statistical significance of searchlight maps across participants, all 719 maps were corrected for multiple comparisons without choosing an arbitrary uncorrected 720 threshold using threshold-free cluster enhancement (TFCE) with a corrected statistical threshold 721 of p < 0.05 on the cluster level (Smith and Nichols, 2009). A Monte Carlo simulation permuting 722 condition labels was used to estimate a null TFCE distribution. First, 100 null searchlight maps 723 were generated for each participant by randomly permuting condition labels within each obtained searchlight RDM. Next, 10,000 null TFCE maps were constructed by randomly sampling from 725 these null data sets in order to estimate a null TFCE distribution (Stelzer et al., 2013      Reflects dissimilarity between object-specific neural activity patterns in the conceptual feature verification task context.

Behavior-Based Conceptual RDM
Reflects dissimilarity between object-specific neural activity patterns in the visual feature verification task context.

Overlap of Similarity Maps in Visual and Conceptual Task Contexts
Visual and conceptual similarity code