Data on multi-actor parameter design tasks by engineering students with variable problem size, coupling, and team size

The experiment studies the effect of technical and social sources of complexity on effort required to complete abstracted design tasks. Parameter design tasks define a set of input design parameters and functional requirements modeled with a linear coupling matrix. Selecting design variables to meet all functional requirements within error limits completes a task. Technical complexity arises from the number and degree of coupling between design parameters. Social complexity arises from the number of designers involved in a task. The experiment includes 10 sessions with between 19 and 24 rounds of randomly generated parameter design tasks each having between two and six design variables and one, two, or three designers. Designers completed individual tasks in parallel during rounds. This article contains raw and post-processed data from 374 completed tasks ranging in effort from a few seconds for simple tasks to more than 15 min for complex ones.


a b s t r a c t
The experiment studies the effect of technical and social sources of complexity on effort required to complete abstracted design tasks. Parameter design tasks define a set of input design parameters and functional requirements modeled with a linear coupling matrix. Selecting design variables to meet all functional requirements within error limits completes a task. Technical complexity arises from the number and degree of coupling between design parameters. Social complexity arises from the number of designers involved in a task. The experiment includes 10 sessions with between 19 and 24 rounds of randomly generated parameter design tasks each having between two and six design variables and one, two, or three designers. Designers completed individual tasks in parallel during rounds. This article contains raw and postprocessed data from 374 completed tasks ranging in effort from a few seconds for simple tasks to more than 15 min for complex ones.
& This dataset provides a baseline of individual and small group performance on randomly generated fully decoupled and fully coupled tasks for cross-validation of future studies using parameter design tasks.
Researchers may use this dataset to investigate temporal activities and search strategies within individual tasks to train computational design agents or explain the effectiveness of individual designers or design teams.

Data
This dataset is included in supplementary data in multiple file formats. The session configuration data (raw/config.json) in JSON format describes details for each session. Key-value pairs in Table 1 include output file names, error tolerance configured in the software application, and any rounds omitted from analysis due to technical problems.
The designer data (raw/designers.csv) in comma-delimited CSV format describes demographic information collected from participants. Columns in Table 2 include gender and age.
The experiment configuration data (raw/session#/experiment#.json) in JSON format describes the conditions for experimental rounds. Key-value pairs in Table 3 describe each round with a name, a coupling matrix, a target vector, indices to assign inputs and outputs to designers, and labels for inputs and outputs. Note that rounds with multiple individual tasks completed in parallel are modeled as composite tasks without interaction between designers. The experimental log data (raw/session#/session#.log) in comma-delimited text format describes actions taking place during a session. Columns in Table 4 include the timestamp, action type, and action data. Most actions reflect updates to design parameters; however, the log file records the complete state of the inputs and outputs for all tasks at each time.
The post-processor script (post/processor.py) written in the Python language includes a main function to execute the script. The post-processor reads raw data and configuration files to build an in-memory model of a session with the following class hierarchy: PostProcessor composes Session composes Round composes Task composes Action. Each class provides helper functions for postprocessing functions.
The post-processor outputs task summary data (post/tasks.csv) in CSV format where each row describes results of a task during a design experiment. Columns in Table 5 include the task session, round, participating designers, task size, degree of coupling, team size, error threshold, and time to complete the task.

Participants and conditions
The study was conducted using an IRB approved protocol. It includes 30 graduate students in technical design-related fields such as aerospace and systems engineering. Participants were assigned to 90-min sessions in cohorts of three based on mutual availability. Sessions were conducted in university classrooms with the three designers and research administrator seated around a single table with computer displays positioned to restrict screen sharing.
The study includes 19 (for sessions 1-3) or 24 (for sessions 4-10) rounds of tasks in partially randomized order to avoid coupled tasks with four variables in first ten rounds. Tasks vary the number of designers, number of design variables, and degree of coupling following the experimental conditions in Table 6. Tasks with one designer are conducted in parallel for all three designers in the same round. A complete description of the experimental design and methods is available in [1] and additional analysis using post-processed data is available in [2].

Design task
Design tasks are based on previous work defining a class of randomly generated parameter design tasks for experimental study [3]. This formulation relates a set of design parameters to a set of functional requirements by a square coupling matrix. A task is complete when design parameters achieve a set of target functional requirements within a specified error threshold. Coupled tasks Table 5 Post-processed task file CSV columns.

Design interface
Participants interact with a graphical user interface illustrated in Fig. 1 to change design parameters and check error in satisfying functional requirements. Vertical sliders represent design parameters where the thumb displays the current value between upper and lower bounds. Both the mouse (click-and-drag) and keyboard can control design parameters where up and down arrows change by 0.01 units and page-up and page-down keys change by 0.1 units. Horizontal sliders represent functional requirements highlighting the acceptable region for solutions in green where the thumb displays the current value.
A distributed software layer based on the IEEE standard 1516 architecture for simulation interoperability communicates design changes between designer interfaces. A research administrator controls overall task progression from a separate console, which also logs actions. Any change to a design parameter immediately induces an update to show new functional requirements. A chime plays when all tasks are complete in a round, signaling the participants found an acceptable solution.