People's reports of unexpected events for everyday scenarios: Over 1000 textual responses, human-labelled for valence/sentiment, controllability and topic category

With this article, we present a repository containing datasets, analysis code, and some outputs related to a paper in press at Cognition. The data were collected as part of a pre-test, pilot test, and main study all designed in SurveyGizmo and participants recruited via Prolific.co (combined N=303). Datasets consist of raw and annotated data, where participant responses are free-text entries about what unexpected events might occur after a series of events, presented them with based on everyday scenarios. The code consists of all computational additions to the data, and analysis carried out for the results presented in the article. This data is released for the purpose of transparency and to allow for reproducability of the work. This human-labelled data should also be of use to machine learning researchers researching text analytics, natural language processing and sources of common-sense knowledge.


Value of the Data
• This data release supports the replication of the experiments (data collection and/or analyses) reported in Quinn & Keane (in press). • The sharing of this data supports further insights into our understanding of unexpected events and event cognition in general. • The data could be used for text analytics or machine learning applications dealing with expectations about event sequences or sentiments about events

Data Description
README.md -a markdown file that introduces the repository, the paper it represents, the data within, as well as how to run the code. 1_pilot_test/ 0_to_label/ These files have been created from the annotated files in 0_data/2_pilot_test_data/1_annotated_data/ for human labelling. There is one .csv file and one .xlsx file per material. 1_MQ_labels/ These files have been manually labelled by rater MQ. Labelling was completed on 27th of July. There is one .csv file and one .xlsx file per material.
2_main_study/ 0_to_label/ These files have been created from the annotated files in 0_data/3_main_study_data for human labelling. There is one .csv file per material. 1_MQ_labels/ These files have been manually labelled by rater MQ. Copy of "2_pipeline/2_main_study/0_to_label" files for the rater. Responses randomized. A new column "random" was manually created and used to sort the spreadsheet. The order was the same for both raters. ID and unnecessary vars hidden. Category, valence, control, and goals label headings added. Labelling was completed on 27th of July. There is one .csv file per material. 2_CF_labels/ These files have been manually labelled by rater CF. Copy of "2_pipeline/2_main_study/0_to_label" files for the rater. Responses randomized. A new column "random" was manually created and used to sort the spreadsheet. IV. 3_output • control_report/ -The statistics and figures in the related paper were partly saved and loaded into Latex using a package called Kallysto. The following folders contain the data that was saved using this package.

Experimental Design, Materials and Methods
The data and analysis code presented in this paper is related to a Cognition manuscript [1] , and is split largely into three parts: the pre-test, pilot test, and main study. The following sections describe the experimental design, materials, and methodology.
Materials : The pre-test was conducted to determine a set of materials for use in the main study. For the pre-test 4 versions of 20 materials (80 individual materials) were created. The 20 materials can be seen in the files in 0_data/0_material_sets/pre_test_material_sets/. Each of the twenty materials had 4 versions: Positively valenced and uncontrollable (means absent), positively valenced and controllable (means present), negatively valenced and uncontrollable (means absent), and negatively valenced and controllable (means present). Material versions were carefully matched on the objects introduced in the scenarios so that, to the best of our ability, the only things that differed between variants of the same material were the variables of interest, valence and controllability.
Experimental Design : The pre-test followed a Latin Square Design: 20 Materials x 2 Control (Present/Absent) x 2 Valence (Positive/Negative) x 2 Question Types (Control/Valence). The 20 materials were divided into four subsets of 5 materials each, deliberately chosen to divide materials with similar themes such as shopping or travelling equally into each subset.
The material subsets were then assigned to four different condition combinations by Control (Means Present, Absent) and by Valence (Positive/Negative).
The following are the Graeco-Latin squares used to counterbalance material subsets assignment to the four condition-combinations in the pre-test and main study. This design has been shown to remove both remote and immediate sequence effects where both condition order and material assignment should be counterbalanced [2] .
Each row of each square corresponds to a condition set a participant could be assigned to. The letter refers to the condition combination seen in Table 2 , and the number refers to the material subset seen in Table 1 .
Method : Participants (N = 64) were randomly assigned to the counterbalanced subsets. Materials were randomly presented within their respective blocks. After each material, participants   A1  C3  B2  D4  D2  B1  C4  A3  B3  A4  D1  C2  C1  D3  A2  B4  C2  D1  A4  B3  B4  A2  D3  C1  D4  B2  C3  A1  A3  C4  B1  D2   Table 4 Material subsets for pilot and main study. were asked to rate the controllability and valence of the scenario they just read on 7-point Likert-type scales. The Controllability and Valence questions were counterbalanced such that half of the participants in each Graeco-Latin square condition saw the controllability question first, and the other half saw the valence question first.
Materials : The 8 materials (chosen from the pre-test) are divided into four subsets of 2 materials each deliberately chosen to divide materials with similar themes such as shopping or travelling equally into each subset ( Tables 3 and 4 ).
Experimental design : The material subsets were then assigned to four different condition combinations by Control (Means Present, Absent) and by Valence (Positive/Negative). Four material sets were created.
Method : Each material set was presented to a separate group of 5 participants. After each material, participants were asked to answer the question, "Something unexpected occurred. What do you think happened?" in a free-response text box. No participant saw more than one version of a given material. Materials were presented in a random order.
The main study followed a similar design to the pilot test, using the same materials and design. A total of 219 participants were collected. Data from the main study were analyzed for the effects of material valence and material controllability on response valence and response controllability.

Ethics Statements
Data collection from human subjects for all experiments listed was conducted with the approval of University College Dublin's ethics review board [LS-E-18-115-Keane-Exemption]. All participants completed informed consent before participating in the studies and were allowed to discontinue participation at any time.