Line-Drawn Scenes Provide Sufficient Information for Discrimination of Threat and Mere Negativity

Previous work using color photographic scenes has shown that human observers are keenly sensitive to different types of threatening and negative stimuli and reliably classify them by the presence, and spatial and temporal directions of threat. To test whether such distinctions can be extracted from impoverished visual information, we used 500 line drawings made by hand-tracing the original set of photographic scenes. Sixty participants rated the scenes on spatial and temporal dimensions of threat. Based on these ratings, trend analysis revealed five scene categories that were comparable to those identified for the matching color photographic scenes. Another 61 participants were randomly assigned to rate the valence or arousal evoked by the line drawings. The line drawings perceived to be the most negative were also perceived to be the most arousing, replicating the finding for color photographic scenes. We demonstrate here that humans are very sensitive to the spatial and temporal directions of threat even when they must extract this information from simple line drawings, and rate the line drawings very similarly to matched color photographs. The set of 500 hand-traced line-drawing scenes has been made freely available to the research community: http://www.kveragalab.org/threat.html.

In Group 1, participant-group correlations for two participants were negative (r = -.286 and r = -.054; the rest of the r's above .816). Removing these two participants from Group 1 increased inter-rater reliability from .972 to .986. In Group 2, three participants with poor correlations were identified (r = .255, r = .280 and r = .321; the rest of the r's above .702).
Removing these three participants from Group 2 increased inter-rater reliability from r = .969 to r = .983. Similarly, in Group 3, another three participants were identified (r = .149, r = .259, r = .273; the rest of the r's above .700). After removing these 3 participants, the inter-rater reliability of Group 3 also increased from .950 to .969. It is important to note that two participants with significantly faster responses also had poor participant-group correlations, offering further confirmation of our decision to exclude their data from further analysis.
After removing data of the eight participants described above, the number of ratings for each image provided by participants in Group 1 ranged from 13 to 18, with a mean of 17.33 ratings (SD = .88) per image; in Group 2 it ranged from 11 to 17, with a mean of 16.32 ratings (SD = .97) per image; and in Group 3 it ranged from 12 to 17, with a mean of 16.41 ratings (SD = .89) per image.

Face validity
In order to evaluate the face validity of ratings provided by the three groups we further investigated images with the highest and lowest ratings and those with the largest and smallest standard deviation, indicating low and high levels of agreement, respectively (Supplemental In addition, we plotted mean ratings of each group by percent of responses grouped by five image categories (Deadly Threat, Direct Threat, Indirect Threat, Threat Aftermath, and Low Threat). Percent of responses was calculated separately for each image category by dividing frequency of mean ratings by the number of stimuli in each image category. Supplemental Figure   1b shows that, as expected, participants in Group 1 rated Direct Threat and Deadly Threat images the highest, participants in Group 2 rated Indirect Threat and Deadly Threat images the highest, participants in Group 3 rated Threat Aftermath images the highest, while all three groups of participants rated the Low Threat images the lowest.
Overall, these analyses suggest that each group was keenly sensitive to their respective task and able to extract the relevant information from the line drawings.

Univariate distributions
The distribution of the image-wise means and standard deviations is shown in Supplemental Figure 2 (panels a and b respectively). The ratings provided by all three groups showed good usage of the entire range of the scale (ratings ranged from 1.06 to 5.83 for Group 1; from 1 to 5.88 for Group 2; and from 1.06 to 5.71 for Group 3). The overall mean rating for Group 1 was 3.25, and the overall median standard deviation was .80. Overall, the distribution of the mean ratings from Group 1 was slightly bimodal with Low Threat and Threat Aftermath images rated lower as compared to Direct Threat and Deadly Threat images, which were rated higher. Low Threat images also showed highest agreement as compared to other image categories. The overall mean rating for Group 2 was 3.65, and the median standard deviation was .73. The distribution of Group 2 ratings looked somewhat negatively skewed, with Low Threat and Threat Aftermath images receiving lower ratings as compared to higher ratings for Indirect Threat and Deadly Threat images. Overall, participants in Group 2 showed higher agreement as compared to Group 1. Finally, the mean rating for Group 3 was 3.43, with the median standard deviation of .85, with fairly normal distribution with slight negative skew. Once again, Low Threat images had the lowest ratings and Threat Aftermath images received the highest ratings.
Participants in Group 3 also showed the highest agreement for Low Threat images.

Relationship between means and standard deviations
We also investigated the relationship between image-wise means and image-wise This impression was further confirmed by the fact that linear regression provided the best fit to the data, with a relatively weak relationship between the means and standard deviations, R 2 = .172. While the Low Threat images were easiest to assess in terms of the probability of past harm having already occurred (since no harm was depicted in these images), the rest of the images presented more room for interpretation, thus leading to less agreement. Supplemental Figure 2. Univariate distribution of image-wise mean ratings for each group (a) and image-wise standard deviations (SD) for each group (b). (c) Relationship between imagewise means and image-wise standard deviations (SD), with the best-fitting quadratic regression line for Groups 1 and 2, and the best-fitting linear regression line for Group 3.