Machines as teammates: A research agenda on AI in team collaboration

A B S T


Introduction
Imagine the following scenario: A typhoon has knocked out the infrastructure of a small nation. Hundreds of thousands of people in hard-to-reach places lack food, water, power, and medical care. The situation is complex -solutions that address one challenge create new ones. To find a workable solution, your emergency response team must balance hundreds of physical, social, political, emotional, and ethical considerations. It is mind-boggling to keep track of all the competing concerns. One teammate, though, seems to have a special talent for assessing the many implications of a proposed course of action. She remembers the legal limitations of the governor's emergency powers, locations of key emergency supplies, and every step of the various emergency procedures for hospitals, schools, and zoos. Her insightful suggestions contribute to the team's success in saving thousands of lives. But she is not human; she is a machine.
This scenario sketches a complex situation in which humans and a machine teammate need to quickly analyze a situation, communicate and cooperate with each other, coordinate emergency response efforts, and find reasonable solutions for emerging problems. In this context, collaboration between humans and the machine teammate plays a critical role for implementing effective emergency response efforts that can save thousands of lives. Although this scenario still remains hypothetical, recent progress in artificial intelligence (AI) capabilities suggests that collaboration technologies may soon be more than just tools to enhance team performance; machines may become teammates [1]. For machines to be effective teammates, they will need to be more capable than today's chatbots, social robots, or digital assistants that support team collaboration. They will need to engage in at least some of the steps in a complex problem solving process, i.e., defining a problem, identifying root causes, proposing and evaluating solutions, choosing among options, making plans, taking actions, learning from past interactions, and participating in after-action reviews. Such machine partners would have the potential to considerably enhance team collaboration. But what might the implications be for human team members, collaborative work practices, and outcomes, for organizations, and for society? AI research has not yet produced technology capable of critical thinking and problem solving on par with human abilities, but progress is being made toward those goals [2]. AI might add value to teams and organizations that may be leaps ahead from current technological team support [3]. In contrast to that, AI might also result in the elimination of jobs or may be used to endanger the safety of humans [4,5]. Numerous discussions evolve around the question whether AI will be a benefit or harm to society in the future [6]. For example, will machine teammates augment human intelligence? Will machine teammates make humans dumb? Will humans get jealous when machines join their team? Will humans feel strengthened with a machine teammate at their side?
The relevance of these questions grows as recent advances in AI suggest this may soon be a possibility. Early research is already under way to explore phenomena surrounding the use of AI in collaboration (e.g., [7,8]), but it is not yet possible to offer definitive answers to any of these questions; we do not know what we do not know. It is, though, possible and useful to organize a research agenda of exploratory research questions to foster interrelated programs of research on the philosophical and pragmatic implications of machines as teammates. Such a research agenda will help us to understand (1) what aspects and concepts to consider in the design of machine teammates in a collaborative environment, and (2) what phenomena of interest really matter for the development of theoretical predictions. We focus on how collaboration researchers could proceed to address this research gap and therefore narrow down the research question of this paper to: How can collaboration researchers study the design of machine teammates and the effects of machine teammates on work practices and outcomes in team collaboration?
The purpose of this paper is to provide a research agenda that collaboration researchers can use to investigate the anticipated effects of designed machine teammates based on the qualified opinions of collaboration researchers. To that end, we initially conducted a survey among 65 collaboration researchers and collected 819 research questions they deemed important. We then performed qualitative content analysis to induce meaningful themes of design considerations and latent theoretical predictions from these research questions. We present the results of this analysis in the form of a research agenda. The research agenda differs between the three design areas machine artifact, collaboration, and institution that deal with various design choices of AI in team collaboration. In addition, the research agenda outlines 17 ambivalent effects, dualities, that the surveyed collaboration researchers anticipate when machines are added to team collaboration as teammates. This research agenda is useful to future research for organizing the design choices of collaborating machine teammates, discovering and describing the phenomena of dualities, developing and testing theoretical models to explain and predict variations in these dualities, and ultimately to understand the many implications of AI in machine teammates. Such research is critical to ensure that machine teammates are designed to sustainably augment human collaboration with beneficial outcomes for individuals, organizations, and societies.

Collaboration technologies
History shows that humans can achieve great things when they collaborate in teams [1]. Yet, teams are not always effective. Some of the major challenges to successful collaboration include poorly designed tasks, ineffective collaborative work practices, and inadequate information systems that are unable to facilitate teamwork [9].
Our understanding of the role of technology progressed swiftly with the intensive research on collaboration technology in general and Group Support Systems (GSS) in particular. Much of the early research was based on the understanding that GSS design features and a few relevant situational variables facilitate team processes and outcomes [10,11]. DeSanctis and Gallupe [10] proposed a multidimensional taxonomy of systems as an organizing research framework to study the effects of GSS. At its core, the organizing framework differed between three levels of GSS systems [10]. Level 1 systems support communication in the team with GSS features such as anonymity or parallelism. Level 2 systems support information processing with GSS features such as voting or categorizing. Level 3 systems support task performance with GSS features that automatically guide the behavior of humans, such as imposing communication patterns onto the group, asking clarification questions, giving recommendations, or providing feedback [10,12] The framework considered initially three critical situational variables as influencing factors: group size, member proximity, and task type [10]. As research progressed, additional factors were identified, such as virtuality (face-to-face vs. blended vs. online team) [13], synchronicity (synchronous vs. asynchronous interaction), or group facilitation [12,14,15]. But still, findings on the effects of GSS were inconsistent. In response to that, a new model was developed based on a meta-review that suggested that GSS performance (e.g., numbers of ideas generated, satisfaction, decision quality, and time) was affected by the fit between the GSS features and the task as well as with appropriation support in the form of training, software restrictiveness, and facilitation [16]. Even though research could demonstrate the potential positive effects of GSS on team performance when considering fit and relevant situational factors, practice showed to be reluctant in adopting and sustaining GSS infrastructures (R. O [17].). As it turned out, the expert facilitator, who provided direct interventions into the team process to encourage faithful appropriation, was the key bottleneck to the diffusion of GSS [16,17].
The "missing-expert-facilitator" challenge has been the focus of collaboration engineering (CE) research [18]. CE aims at packaging facilitation skills and GSS expertise in such a way that reusable and predictable collaboration work practices can be designed and executed for recurring, critical work situations [18]. To enable such reusable and predictable work practices, CE developed the concept of thinkLets, which are scripted facilitation techniques that trigger predictable effects and group dynamics among team members that work toward a common goal (R. O. [17]). Practitioners can be easily trained in these recurring work practices without becoming expert facilitators [17]. A main difference to previous GSS research is that CE research builds on the philosophy that design decisions have to be made on multiple levels spanning people, process, information, technology, and leadership [19]. Briggs et al. [20] translated this philosophy into the six-layer model of collaboration (SLMC). It functions as an organizing scheme for the concepts and methods of collaboration science that build the basis for the required design choices that have to be made. These layers comprise (1) collaboration goals, (2) group products, (3) group activities, (4) group procedures, (5) collaboration tools, and (5) collaborative behaviors. Similar to other layered models, layers in SLMC are interfaced with the ones that are above and/or below. Each layer attempts to make transparent the available design choices one has for the design of collaborative work practices based on relevant literature synthesized from different research streams [20]. This should help collaboration engineers, who design repeatable work practices, to make the necessary design decisions layer by layer to reduce cognitive load and increase performance [21].
The progress on the interplay between facilitation, collaboration technologies, and other influencing factors provides relevant insight into the effects of technology on team outcomes, such as improved knowledge sharing, task performance, satisfaction with process and outcomes, or shared understanding [22]. Despite these gained insights, effective IT-supported team collaboration remains a challenge because of multiple reasons. Collaboration engineers are expensive and rare [21], which leaves practitioners that are usually domain experts but not professional facilitators, with the challenge to plan their meetings themselves and an increased potential to fail [18]. Additionally, the organizational context in which collaboration takes place changes tremendously in the time of digital transformation. Many organizations have adopted Open Innovation as a problem solving model to outsource their idea generation, convergence, and/or evaluation processes to the crowd [23,24]. Facilitating a crowd may differ considerably from teams because (1) individual crowd members are unlikely to interact with each other, (2) they may be anonymous to the sponsoring organization, and (3) crowd tasks are usually of short duration. Moreover, temporary impromptu and action teams, which refer to groups that form unexpectedly [25], are increasingly characteristic for novel collaboration settings. They differ from traditional teams as they may not follow predesigned command structures, may not have a central authority, or may form only for a short duration. Finally, collaboration practice and research are about to face off with yet another disruptive force: the machine teammate entering AI into team collaboration that has the potential to alter and advance our understanding of collaboration as once GSS and CE did. The machine teammate is an autonomous, pro-active, and sophisticated technology that draws inferences from information, derives new insights from information, learns from past experiences, finds and provides relevant information to test assumptions, helps evaluate the consequences of potential solutions, debates the validity of proposed positions offering evidence and arguments, proposes solutions and provides predictions to unstructured problems, plus participates in cognitive decision making with human actors. Such a machine teammate may be an important technology to deal with in current designs and investigations of team collaboration. But what do we know today about intelligent machines in team collaboration?

AI joins the team
AI refers to the capability of a machine or computer to imitate intelligent human behavior or thought [26]. How this machine should behave or think, though, is disputed: should an AI be completely rational or incorporate social, emotional, or ethical considerations? Affective computing is a subdomain of AI, which investigates how AI learns to incorporate and understand emotional signals from humans, such as happiness, anger, or deception [27]. A rational AI, by contrast, would always base its decision-making on optimizing its objectives, rather than incorporating social or emotional factors.
AI has become more ubiquitous because of the increased accessibility of hardware and software that run large dense neural network training algorithms (also called Deep Learning), which mimic the neural architecture of the brain. These algorithms can be trained on unstructured data such as images, audio, or text, and have revolutionized the degree to which machines can learn to reason, classify, and understand. Currently, these algorithms are specific to narrow task domains, such as speech recognition, image classification, human emotion, and characteristic recognition. For example, the humanoid robot NAO can adjust its behavior based on the identified gender of its interaction partner [28].
Human-AI interaction requires more than just smart algorithms. It requires actual coordination of complex activities such as communication, joint action, and human-aware execution [8,29] to successfully complete a task, with potentially shifting goals, in varying environmental conditions mired in uncertainty.
With such rapid improvements to AI, ethical and moral challenges posed by AI are receiving greater attention as well. Answers to questions such as "what moral code should an AI follow?" [30] and "what unintended consequences could result from the technology that threaten human autonomy?" [26] are being examined. The optimal conditions for humans and truly intelligent AI to coexist and work together have not yet been adequately analyzed. For example, when expert polygraph examiners collaborated with an AI to detect deception, the human examiners did not improve their deception detection accuracy [31]. Instead of helping, the AI threatened the self-efficacy of the human experts by challenging their decisions, and as a result, the correct AI recommendations were disregarded. Similarly, the humanoid robot NAO has been found to influence acquiescence in children such that the children confirmed with the opinion of the robots instead of their own judgment [32].
These limited examples allow us to draw some inferences regarding the future of collaboration with machine teammates. As with the illustrations, mixed results can be expected with regard to the effects of machine teammates due to the diverse collaborative environments that AI will be used in. It is possible that machine teammates will be designed with different collaboration capabilities. Additionally, teams may develop different norms regarding the use of a machine as teammate or organizations might rely on different regulations for machine teammates. Hence, different implementations of a machine teammate in a team and an organization will most likely result in different effects. With this in mind, it appears meaningful to formulate a research agenda to structure future research efforts in our quest to generate cumulative knowledge on AI in team collaboration.

Method
We conducted a survey with 65 collaboration researchers to collect research questions on machine teammates. We used these research questions to develop a research agenda on the design and effects of AI machine teammates in team collaboration.

Survey design
The survey consisted of three parts. The first part aimed at getting participants into a creative thinking mode to envision a future where machines will be our teammates. We offered participants a fictional scenario, which aimed at describing a machine teammate in action: A Category 5 Hurricane is sweeping over Florida. Jim, the severe weather technician, Mike -his boss, and Kate the AI Weather expert check the latest damage report of sensitive infrastructure to hospitals, main streets, and bridges. Jim is worried that the widening cracks in the concrete columns of the highway bridge, as reported from the sensor devices, may collapse. He wants to send one of the repair ants -smart ant-like robots -that can navigate in hurricane 5 winds and are equipped with a variety of tools. But, Kate is not convinced and explains: "The Bayside medical clinic has 30 critical care patients. The clinic's power generator is down and the storm surge is expected to hit the clinic in 20 min. There is a 93% greater likelihood of loss of life if repair ants do not reach the facility in time. So, the repair-ant is needed first at the clinic." Jim looks at Mike, "What do you think?" he asks. Mike looks thoughtful, "I had a repair-ant scheduled for maintenance tonight. It might just have enough 3D printing material left to produce gum for the most important cracks in the bridge" he says. "We might just be able to pull both the repairs off." To foster shared understanding, we defined machines as teammates (MaT) as "those technologies that draw inferences from information, derive new insights from information, find and provide relevant information to test assumptions, debate the validity of propositions offering evidence and arguments, propose solutions to unstructured problems, and participate in cognitive decision making processes with human actors".
In the second part, we asked participants "What research questions (RQs) will the collaboration community have to answer to move from our current state-of-the-art to the future we envision with machines as teammates?" First, participants engaged in a free brainstorming [33] activity where they provided as many research questions as they could think of. When they moved to the next page, participants engaged in a brainstorming activity with prompts. We adapted the brainstorming technique LeafHopper [34] using the following categories as prompts: affective, cognitive, communication, economic, ethical, organizational, physical, political, societal, technical, and other. An example prompt was "What technical research questions must we answer to have machines as teammates?" We selected the categories to cover a broad range of aspects of the socio-technical system of a machine teammate to stimulate researchers' creative thinking. The variety of categories should ensure that researchers with diverse backgrounds, yet a shared interest in collaboration research, could contribute to the brainstorming task.
In the third part, we collected demographic information from participants (career level, expertise, gender, and country) and solicited additional qualitative feedback. Participants could also opt-in with their e-mail addresses to receive results from this study.

Sample
The survey was sent to collaboration researchers around the world. We had three subsamples: first, we invited authors of the HICSS 2018 conference through its mailing list. Second, we invited 96 collaboration researchers that we deemed to be domain experts in their areas of HCI, CSCW, or IS research. Third, also the authors could provide questions, as they are representative of the domain CE. The survey was accessible from February 28th to March 19th 2018. We received 65 responses (8 by co-authors, 42 by domain experts, and 15 by HICSS authors) that were later qualitatively analyzed within the authoring team. Respondents submitted a total of 819 ideas for research questions. The idea frequency table (Table 1) shows the number of received contributions per category and per participant group. In the first step (FreeBrainstorm), we received 270 contributions. In the second step (LeafHopper), we received 549 additional contributions.

Analysis procedure
We received a rich set of responses (N = 819). As expected, some of these ideas were redundant. Some ideas were on different levels of abstraction. Moreover, many ideas were not stated as open-ended or closed-ended questions but rather as statements and/or opinions. Therefore, we developed a multistep analysis procedure, which was in essence an iterative approach of qualitative content analysis consisting of content structuring and inductive theme analysis [35].
In step 1, three of the authors organized a subset of one hundred ideas into inductively derived categories to lower information overload. The preliminary categories were: machine artifact design, individual, social, organization, and society. Two of the co-authors and four additional graduate and PhD students used these categories and organized all remaining ideas using the collaboration system Think Tank. Then, all co-authors met virtually to discuss and explain the meaning of category labels. Subsequently, subteams of at least two co-authors were assigned to each category to evaluate the ideas in a category and determine whether they were a good fit for the category. If an idea was found to be a poor fit, that idea was moved into the category that was deemed to be most appropriate.
In step 2, each subteam categorized the ideas from their category pool into common themes. Themes were for example "appearance" in the category "machine artifact design," "trust" in the category "group," or "cost and benefit" in the category "organization." The subteams also resolved differences in abstraction for their themes and selected the research questions for their category that were considered as representative for the themes. To further reduce information overload, the subteams removed redundant or merged highly similar ideas.
In step 3, the authors recognized a duality aspect inherent to many of the themes, e.g., benefit vs. threat, good vs. bad, and chance vs. risk. A duality refers to "an instance of opposition or contrast between two concepts or two aspects of something." [36]. The coding continued with the analysis lens of dualities. Dualities were deduced from associated research questions that signaled ambivalence with respect to the direction with which MaTs affected theoretical concepts. Then, the authors selected the theoretical concepts that previous research had satisfactorily operationalized and that could be used in future empirical collaboration research to investigate the effects of MaTs. The following provides an example of the coding (Table 3): Each duality expresses a paradoxical effect that arises from machines entering as partners into human team collaboration. The paradoxical effect could exist 1) within a theoretical concept with different manifestations (concept dichotomy) or 2) between two concepts (association dichotomy). An example of a concept dichotomy in human-machine collaboration is that a human could accept the technology (i.e., machine teammate) or reject it. In that sense, the theoretical concept is "technology acceptance" and the dualism exists in the notion that technology is "accepted" or "rejected." An example of  an association dichotomy in human-machine collaboration is a machine teammate that might receive acknowledgement for a job well done, which could lead to higher team expectations. In this case, "work acknowledgement" and "team expectation" represent associated theoretical concepts. The dichotomy describes that the associated concept changes as the base concept changes. Overall, the coding resulted in 17 identified dualities.
Only the categories machine artifact design, group, organization, and society remained with their themes. These themes did not address dichotomies but raised aspects of design for human-machine collaboration, e.g., the theme "sensing capability" within the category "machine artifact design." We merged the categories "organization" and "society" into "institution." Three categories (machine artifact design, collaboration, and institution) remained, which we refer to as design areas.

Design areas for AI human-machine collaboration
The first part of the results addresses the design areas for AI human-machine collaboration. The analysis revealed three design areas, which are machine artifact design, collaboration design, and institution design. Each of these design areas shortly describes design challenges and provides exemplary research questions. Core topics from the original research questions are used to argue for the themes. In that sense, a core topic can be identified with an ID such as 236_3. The first three numbers refer to a randomly assigned user ID, and the last number is a running count of the user contribution. In this case, the original voice refers to user with ID 236 and his/her third submitted contribution. All collected contributions are provided in the appendix.

Machine artifact design
This design area is concerned with the diverse possibilities that exist to design a machine teammate. It consists of seven identified themes that, in turn, connect similar or closely related design choices of a machine teammate. Although the overall design will affect and will be affected by team collaboration, these consequences are not in the focus of the design areas. The potential consequences will be presented in Section 5.
Appearance. This theme addresses the question how a machine teammate should look like (178_3). Design choices need to be made as to whether the machine teammate should have a gender and which (231_7), whether it should appear as a cartoon, avatar, or human-like (231_9, 256_1), whether it should have a personality (231_12), or whether humans should communicate via text or speech (168_1). These contributions are summarized in the following research question: • How should a machine as teammate look like?

Sensing & awareness.
This theme highlights what kind of sensory information, e.g., camera, heat, movement, heart rate (179_5), smell, or touch (272_3) a machine teammate should be equipped with. Moreover, research questions in this theme highlight to what extent machine teammates could infer emotions (221_6), interpret body language (221_2), and understand intention from text and interactions (220_2). We summarize this theme in the research question: • How can machines as teammates sense their environment to become aware of their surroundings?
Learning and knowledge processing. This theme concerns how machine teammates should learn and share their learning with their human teammates (178_4). Besides building and maintaining a knowledge base (179_6), learning also addresses how machines can read body language (221_5), differentiate between serious requests and social chatter (220_3), set and attain goals (265_6), or have moral principles (235_5). Machine teammates could possess tremendous recording capacities (289_3) to remember the history of their interactions with different human teammates (235_4), and improve upon their experiences (268_4). It might also become important that they can forget (331_7). The corresponding research questions are: • How can a machine as teammate select and acquire data that it can process?
• How can a machine as teammate learn to process and forget information?
• How can machines as teammates learn and how can they share their learning with their collaboration partners?
Conversation. One central capability of a machine teammate could be the ability to interact and socialize with their peers (256_2, 215_3, 168_3). This could concern the ability of turn-taking (179_7), understanding irony (220_4) or jargons (189_3), being polite (168_2, 185_9), or politically correct in their interactions (167_1). The summarizing research question is: • How can we design the verbal and nonverbal communication from the machine, so that it fits the collaborative situation?
Architecture. This theme highlights the key architectural components for a functioning machine teammate (256_3). This might concern the questions on what kind of devices (231_13), e.g., distributed ondevice deep learning architecture (189_4), the machine teammate will run, if it will be miniaturized (189_4), or if it will have an emergency "off/on" button (327_7). Moreover, the production and use of a machine teammate might create considerable demand for energy (220_6), which needs to be considered in its architecture. This is captured in the following research questions: • What are key components of a machine as a teammate and how do they relate to one another?
• How can we design energy efficient machine teammates? Visibility and reliability. To determine flawed behavior of a machine teammate, (203_3), designers could make deep-learning algorithms understandable for humans (237_2) so that they can explain their recommendations (256_4) and can be reviewed by humans at various stages (237_1). To ensure the reliability of a machine teammate, designers might need to find ways to determine when behaviors of the machine actor become flawed or when the machine actor develops undesirable intents (303_3). Alongside this, designers could also consider the need to transfer the machine teammate's "personality" in case it breaks down (220_8). The derived research questions are: • How can machines as teammates explain their actions? • How can we build systems that are sufficiently reliable and make transparent how reliable they are for each suggestion they make?
• How do we deal with breakdowns?

Collaboration design
This design area is concerned with the design of the team, task, and collaboration process. Hence, the focus shifts from the machine teammate to a team collaboration setting with human actors.
Team design. Future human-machine teams could be designed based on the core competencies brought in by humans and the core capabilities of machine teammates (181_7). Machine teammates might not only actively participate in problem solving (220_9), but eventually also adopt the role of a leader (264_8). Moreover, design choices might need to consider the size of the team (168_4, 231_16) and if the team is collocated or virtual (262_3). These research questions summarize this aspect: • What is a good division of labor between machine teammates and human teammates?
• What is the ideal team size for machines as teammates for a specific task?
Task design. Human-machine teams could be designed based on the types of tasks that are most suitable for such mixed teams (168_5). Machines might possess general collaboration capabilities to actively engage in collaboration or capabilities for very specialized tasks (220_10). Some collaboration tasks might be more likely to become automated (181_5) while some tasks might be limited to humans only (256_6). Such aspects are reflected in these research questions: • What are the criteria to decide whether a task can be executed by a machine, human, or through human-machine collaboration?
• How can we identify applications and problems that can benefit from the integration of human and machine knowledge?
• How can we decide between general purpose machine actors that can do anything and highly specialized machine actors built for a specialized role or task?
Work practice design. Machine teammates could be trained for specific collaboration processes, such as coordination, knowledge sharing, or evaluation (167_3), which might spark changes in creativity, groupthink, or problem solving (225_3). The mode of communication (voice or text) might influence the effectiveness of these collaboration processes greatly (231_20). When collaboration technology changes its role from tool to partner (171_3), it might become necessary to find new approaches to model and engineer the new collaboration and decisionmaking processes (171_2, 175_3). This is captured in the following research questions: • How can we engage machine teammates in collaboration processes? • How can we systematically design machines as teammates in a human-centric way?
• How ready are our tools and techniques for engineering collaborative processes for modeling future collaborative processes?

Institution design
This design area addresses questions related to the design of structures and rules for organizations and society.
Responsibility and liability. Machine teammates might perform actions (261_4) or make decisions (244_2) that cause problems (319_19). Organizations as well as federal governments might need to clarify, if the machine, the designer, or the human teammates are responsible and liable (261_5, 171_4). The rights and obligations of machine teammates and other stakeholders need to be clarified (178_13). Therefore, design choices relate to the definition of policies, regulations, and laws for machine teammates (327_10). These questions summarize this aspect: • Who is accountable for the decisions of machines? • What governance approaches are needed to set up a machine-ascollaborator work context?
• What rights and obligations do machine teammates have?
Education and training. When machine teammates join the team, humans will most likely need to adapt and change. Organizations could facilitate this change by training people in the required collaboration competences for collaborating with machines (178_10). On the societal level, we might see changes to education programs so that students become savvy in developing and working productively with machine teammates (175_5) and validating them (236_6). The associated research questions are: • How can we change our education programs to develop student competencies for working with machine teammates?
• How should people be trained to collaborate with machine teammates? Fig. 1 summarizes the three design areas: machine artifact design, collaboration design, and institution design, and lists the major design choices for each area.

Dualities in effects
The second part of the results addresses the dualities in the form of concept dichotomies or association dichotomies that could arise from AI human-machine collaboration. A concept dichotomy refers to the paradoxical effect that designed AI team collaboration has on a theoretical concept. An association dichotomy refers to the paradoxical association between two theoretical concepts in designed AI team collaboration.

Concept dichotomies
We found several potentially conflicting consequences for the use of machine teammates. Machine teammates might change the affect, knowledge, technology acceptance, trust, and group dynamics among teammates. Machine teammates might also change human health or job availability in organizations or within the society. We refer to these kinds of dual effect phenomena as conceptual dichotomies, which are described in more depth in the following.

Affect positive/negative
This dichotomy describes the positive and negative emotions that humans might feel when machine teammates join the team. In case machine teammates can understand and react to human emotions (179_10), they could build emotional bonds with humans (233_13) and show empathy or provide emotional support (264_14). Yet, there might be cases where humans feel inferior, feel a lack of belonging, or feel they lose status (220_34, 231_32). This might negatively affect their self-esteem (189_13), induce emotional stress (178_28), and increase anger and frustration (178_27).
• How do we deal with anger and frustration against machines as teammates?
• Under which conditions will people enjoy working with a machine teammate?

Team knowledge augmented / depleted
One of the intended effects of AI collaboration is to relieve human teammates from some of the mundane tasks that a machine can do better (e.g., calculations, information retrieval, and pattern recognition). Machine teammates will need to explain and visualize their suggestions (224_5) to augment human intelligence (227_5) and support the team in coming up with conclusions (207_19). Machines might even be able to fill structural holes (319_26). At the same time, there exists a risk that certain competences vanish (167_10, 189_14) or that humans become dependent on machines (227_5, 227_6). For example, with interfaces becoming voice enabled, we might see decreases in the human ability to read (225_11).
• How can artificial intelligence be used to support decision-making without depleting human knowledge?
• To what extent does (emotional) intelligence increase or decrease when machines join collaborative work?
• Under which conditions can and should machines augment humans' cognition?

Technology accepted/rejected
We currently lack an understanding of the conditions under which humans accept machines as teammates, for example, whether they are more likely to accept a humorous or a serious machine teammate (167_11) or a machine teammate that supports coordination tasks or creative tasks (171_11). At the same time, we might see that humans reject technology because they do not take the machine teammate seriously (302_5), they do not want to obey to a machine that assigns tasks (347_9), or have technophobia in general (268_10). Additionally, a person's cultural disposition might affect to what extent they accept or reject technology (175_18).
• To what extent will human collaborators accept the input from machine collaborators?
• To what extent do different styles of verbal and nonverbal communication affect the acceptance of the machine collaborator?
• Which machine-generated recommendations and solutions will individuals accept when they are the ones to carry out the work?

Trust built/lost
Trust could concern trust in the machine teammate (178_29), trust in its recommendations (256_20), or trust in its underlying algorithms (274_12). A machine teammate could change how we build trust with other humans (319_28) when we start to trust a machine recommendation more than a human recommendation (175_19). We might lose trust in the machine teammate when it contradicts a human (312_37) or when a human experiences certain emotions (312_38). We might lose trust in a machine's recommendations when the associated decision is particularly difficult (e.g., life or death) (312_39).
• How much should we trust the machine teammate's insights and recommendations?
• How does contradicting the human affect the human's trust in the machine?

Group dynamics positive/negative
When machines join the team, they might be trained to identify certain group dynamics (167_12). They could help to foster team cohesion (347_11) but create negative group dynamics such as conflicts (178_30, 207_20).
• How do machine teammates influence group conflict? • What group dynamics should the machines be able to assess to foster improved team performance?

Health enabler / risk
Machine teammates could contribute to the safety of humans, particularly in collaborative industrial teams (262_11) where they can use their physical strength (274_13) to protect humans. Equipped with sensors (274_13) and several safeguards (319_29), they could additionally foster the well-being and fitness of humans (189_15). At the same time, machine teammates could be a risk for humans as they might threaten the psychological health of humans (167_13) or leave dedicated areas (220_42) where they might harm humans.
• How can impact the psychological health of human co-workers? • How can we insure the safety of humans in collaborative industrial teams with robots?

Jobs created / cut
When a machine becomes capable of performing certain tasks, organizations might require a smaller human labor force (175_21). This might be particularly true for highly repetitive tasks that require low skilled workers (267_15, 272_13). At the same time, new jobs might be created or humans could focus on certain more complex tasks in existing jobs (319_31). These jobs might be highly creative (267_14), require logic and rational thinking (272_13), or specialized skills (267_15).
• How can we deal with the reduced availability of low-skill jobs for humans that will result from increasingly capable machines?
• Do machines as teammates replace jobs or repurpose them?

Association dichotomies
The use of machine teammates should empower teams to achieve superior collaboration results. Machine teammates could become creative, efficient reasoners. They can also be human-like and adaptive. In addition, teams with a machine teammate might benefit from improved decision making, quicker task accomplishment, increased acknowledgement for their work, could receive more responsibility, and could have more transparent team processes. Organizations might benefit from machine teammates because they drive new value creation. Yet, once this is improved, new state of a theoretical concept is achieved, and a dark side of human-machine collaboration might emerge that is detrimental to another, associated theoretical concept.

Higher quality of decision making -reduced capability to criticize
Machines might be able to solve the problem of poor decision making in collaboration environments characterized by information overload. A machine teammate could improve information processing by mitigating negative cognitive biases (167_5) or effectively identifying reliable, accurate information (215_4). When contributions of a machine teammate are constantly useful and decisions are, in fact, improved (266_9), we might face a new problem, where humans become dependent on the automated machine algorithms and become passive information seekers (225_6).
• How can machine teammates be used to overcome human cognitive biases in decision making?
• How can a machine teammate determine how reliable, accurate, or truthful the information source is?
• How should humans interact with automated procedures without losing the ability to analyze and criticize?

Increased pace of work -increased cognitive overload
Machine teammates might increase the pace of collaborative efforts (167_6). They could be always "on" (221_9) and perform tasks while human teammates return to their private life. They might also be fast (256_12) because of their computation advantage over humans in certain tasks (235_8). Although it might be beneficial for a team to accelerate certain work tasks, e.g., information seeking, this could also spark an unintended challenge. Machine teammates might explain their reasoning insufficiently (178_20) for a human to understand, which might lead to misunderstandings between humans and machines (233_7) and increased demand on cognitive effort to sort out misunderstanding. Humans might need to rest while performing effortful tasks (269_15) and might need to adapt quickly to new tasks (236_9). This could be overwhelming for individuals as machine teammates are unable to deal with humans' limited cognitive capacity (178_21).
• If MaT increase the pace of collaborative efforts, what positive or negative effects might such increased pace entail?
• How can we ensure transparency and speed of machines' decision preparation processes to match human decision makers' cognitive capacity?

Increased creativity -lack of serendipity
Machines might autonomously generate creative solutions (224_3). To do this, they need to gather insights that can be justified with data (201_1) or help highlighting disagreement among participants (215_5). Yet, many algorithms gain "insights" by assessing closeness and similarity of events, people, etc. This might create the problem of reinforcing existing views (225_7) decreasing the out-of-the-box thinking.
• As the relationship between machines and humans becomes more intertwined, how do we ensure that humans' creativity does not become constrained?
• How should knowledge creation be dynamically shared between machines and humans?

More efficient reasoning -fewer human-driven decisions
A machine teammate might be able to draw inferences, give insights, and provide relevant information (256_14). If this is the case, they might become a more reliable source of information than experts or other people (221_11). They might become an integral part of our decision-making processes (244_4). When their proposal might be judged better than another human's (289_10) because of, for example, calculated confidence intervals (312_24), their recommendations might become highly persuasive for humans. Humans might rely on machine teammates to such an extent that deskilling may set in, resulting in fewer human-driven decisions. Eventually, a machine teammate could often have the final say (312_23).
• How does a machine teammate determine if the information and insights he/she offers is relevant to the ongoing discussion with other teammates?
• What factors influence humans so that they rely on machine recommendations over time?

More work acknowledgement -higher expectations
We usually recognize and acknowledge good work completed by humans. However, also machines might provide important (intellectual) contributions to the team (201_3), which, according to this logic, would get recognized and rewarded (207_16). If so, employers might expect more from teams with a machine teammate and increase their workload (231_22). At the same time, machine inputs might be misappropriated, if proper credit is not given (302_4).
• How should machines be rewarded with their contribution to the projects?
• Will employers expect more from employees who are part of teams with machine teammates?

More anthropomorphism -more manipulation
When we collaborate with machine agents, e.g., in the form of avatars or robots, we tend to associate human-like characteristics to these nonhuman entities (called "anthropomorphizing"). This way, humans might start to like and accept the machine counterpart (231_23). Yet, other humans might exploit this kind of trust and manipulate or trick (231_24) other humans. Humans might manipulate others with the help of machine teammates (233_10) to strengthen their own position in a team (175_10). Hence, it might become important for machine teammates to have "certain characteristics that make them distinguishable as machines" (168_6). This might lower the likelihood that a machine "disguises" (171_7) itself as a human collaborator.
• How should human-like machine teammates appear or what characteristics should they have to be useful and likeable partners?
• Should machine collaborators be clearly identifiable as being machines or is it better to "disguise" them as being human collaborators?

More responsibility -loss of control
If machines are more helpful, process more information, and have better answers than humans (221_13, 256_15), employers might consider assigning machine teammates more authority (221_13) and responsibilities (237_7). This might create problems with control. If employers consider replacing a human teammate with a machine teammate due to good performance (256_15), humans might fear that machines take over (171_8). If people take the back seat and let machines perform tasks that until recently only humans were able to do, human teammates may feel inferior (267_4), have only nominal control (189_7), and may feel that an informal transfer of power and leadership may set in (189_8, 272_6).
• Should a machine get more authority if has better answers than humans, or if it can process more information?
• How can machines help individuals to have more power or influence in a team process?

More visibility -loss of privacy
To achieve effective collaboration and personalization (220_20), algorithms of machine teammates need to become transparent and controllable (225_8, 227_4). Data collected might comprise data from built-in cameras (185_15), about human teammates (225_8), but also confidential project information (207_17). With this increase in visibility, problems of privacy might emerge (175_11). Teammates might feel monitored and surveyed (175_12, 220_21) increasing the need for safe guards (175_11) and rules of confidentiality (207_17).
• How can we ensure data collected about a person and the inferences made based on them are transparent and controllable by the person?
• What safe guards need to be in place when organizations use machines that access confidential information?

Higher adaptiveness -more misbehavior
Machine teammates might require highly adaptive personalities to fit the individual preferences of their teammates (167_8) or a specific situation (235_13). Adaptiveness might refer to the emotional expressions (236_10), personality (171_9), use of communication channels (207_18), or bending the rules from time to time (220_28). When their learning algorithms are highly adaptive, machine teammates might also learn bad behavior from their human counterparts. They might express aggressive behavior (220_24), have prejudices (220_25), send nasty messages (231_28), or become biased (215_11). • How can we allow machine teammates to learn from their perceptions without the fear that they learn bad behavior?
• How can machines build up something like a moral conscience? • How can we teach machine teammates to "bend" the rules from time to time, without the fear that they will use it against us?

Higher value creation -extreme power shifts
Machines as teammates might affect humans beyond team boundaries. Machines might create organizational value, because they could improve an organization's productivity (226_11), could be commercialized (175_15), or rented (185_17). Some costs might occur, such as investment costs to acquire/build the technology (262_9), paying taxes (220_30), or retraining workers (262_9). However, it could be that these costs are considerably lower than the labor costs of the human workforce. This could trigger substantial power shifts among societies, organizations, and humans. Machine teammates could cause power differentials as they might improve the national strength (289_15) or help create more monetary or cognitive resources (225_10). Those who have machines (178_25) may become more powerful while those without a claim to ownership may lose power and prosperity.
• Should organizations develop machines in house or will we have COTS AI?
• How much does it cost to hire/build machine teammates vs. human teammates for the same task?
• How do societies react to the shifts in power between those who have machines as teammates and those who haven't? Fig. 2 provides an overview of the 17 dichotomies presented above.

Novelty of the research agenda
The goal of this paper was to develop a research agenda that supports collaboration researchers investigating socio-technical systems where machine teammates collaborate with human teammates to achieve a common goal. Based on a survey of 65 collaboration researchers, we discovered three design areas that guide attention toward the conditions under which the designed AI team collaboration affects either the positive or negative side of 17 dualities. We combine the three design areas and the 17 dualities in a MaT research agenda, which is depicted in Fig. 3.
Already during the last "AI hype" in the second half of the 1980s, researchers speculated that AI may significantly support group collaboration. We can now update their speculations with far advanced knowledge on AI and on collaboration [37]. We propose that AI will not (just) be the functionality of a tool but rather a machine teammate characterized by a high level of autonomy, based on superior knowledge processing capabilities, sensing capabilities, and natural language interaction with humans. This raises a whole new set of design issues ranging from HCI (MaT appearance and sensing/awareness), classical AI (learning and knowledge processing, visibility and reliability, and architecture), and computer linguistics (conversation). In doing so, we reconnect collaboration research to modern computer science and debates in other areas of modern IS research.
We anticipate that the decisions made in the three design areas with their 12 themes will define the composition of the machine teammate and its environment. The three areas, machine artifact, collaboration, and institution, complement each other. Design choices in one of these areas will influence design choices in the other two areas. The research agenda encourages to consider variations in AI-based human-machine collaboration depending on the design choices one makes with respect to the machine artifact, the collaboration, and the institutional environment in which the collaboration should take place.
The MaT research agenda also strives to catch and structure the most relevant consequences of designed AI team collaboration. It was striking to see so many research questions that linked to positive and negative anticipated consequences. This ambivalence in predicted effects is in line with the argument that AI is a dual-use technology; it can be used for both beneficial and harmful purposes [2]. The MaT research agenda incorporates this ambivalence in its dualities, which are organized into concept dichotomies and association dichotomies. Hence, the research agenda emphasizes the interdependence between design choices and consequences, which are a key to unravel the ambiguous theoretical predictions inherent to the dualities. It has long been established that system designs affect team collaboration for better or worse [10]. Progress in GSS and CE added knowledge of how nontechnical variables, such as facilitation, need to be designed and put into practice for improved team collaboration [18]. The identified MaT dualities differ as they add variables, e.g., negative affect and team knowledge depletion. They highlight potential effects that collaboration researchers have not necessarily focused on; they emphasize the dark side of AI team collaboration. Furthermore, dualities such as "jobs created/lost" or "higher value creation -extreme power shifts" represent consequences outside the team context and refer to organizational and societal concerns. In this sense, the research agenda differs from previous emphases as it stresses the need to build and test AI in team collaboration for beneficial consequences, not just for teams but also for organizations and societies.

Research implications
The outlined dualities and design areas could help collaboration researchers from different domains, such as information systems, human-computer interaction, or organizational psychology to design research investigations into MaT in the following three ways: First, the dualities could provide anchor points for exploratory research within organizations that already assimilated machine teammates into their organizational processes. For example, investigating the dualities through multiple case study research could help shed light on the relevance of these ambivalent effects in practice and the conditions under which they emerge. This empirical evidence is essential for understanding which of the dualities matter under what conditions and in what professional environments. Additionally, such insights allow future research to focus on the most relevant problems of AI in team collaboration.
Second, researchers could use the design areas to typify the machine  teammate and its environment, develop prototypes, and test them in the lab. Such as the common description of laboratory experiments with explanations on treatments, dependent variables, subjects, etc., researchers could use the design areas of the MaT research agenda and the themes organized in these to add a more structured description of the machine artifact, the collaboration in which the machine teammate is employed, and its institutional environment. This would make the design choices of the machine teammate in its collaborative environment transparent and facilitate replication of studies. Eventually, design principles could be deduced to guide the implementation of machine teammates that are beneficial for humans, organizations, and society.
Third, knowing about the effects and design choices allows future research to falsify collaboration-related theories and their boundary conditions. They could inspire collaboration researchers to develop and expand theory-based research models. For example, future research could investigate the concept dichotomy "team knowledge augmented/ depleted" using the theoretical lens of transactive memory systems [38] and examine how machine teammates can engage in team information and knowledge processing for improved collaboration outcomes [39]. Future research could also investigate the association dichotomy of "more responsibility -loss of control" using the theoretical lens of control theory [40,41] to test control modes and perceptions of human teammates when machine teammates take over certain tasks [42]. Researchers might develop new theories to explain new phenomena that might arise with machine teammates and identify new boundary conditions. Hence, the MaT research agenda could be a first step toward a more systematic identification of whitespaces in existing collaboration theories.

Practical implications
The findings of this study could already be useful for managers that intend to adopt or have already adopted virtual assistants, conversational agents, or other AI collaboration technology into their workplaces. In these situations, managers could consider themselves as organizational designers who could influence, for example, the composition of teams, the distribution of tasks, or the extent of inclusion in collaborative work practices. Both types of dualities enable managers to become vigilant what effects the introduction of highly capable AI might entail in human-machine work environments.
Also, designers could benefit from the use of the MaT research agenda as it outlines several design factors that can be connected to one or more dualities. For example, when a designer intends to create a trustworthy machine teammate (see, trust built/lost), the research agenda also draws the attention to the design areas of collaboration and institution that might be relevant. The different aspects of the design areas, e.g., visibility and reliability in machine artifact design, could serve as further guidance to perform more comprehensive evaluation studies that focus on the effects on the human workforce.

Limitations and future work
This exploratory study has several limitations that should be considered. First, the study discovered three design areas, i.e., machine artifact design, collaboration design, and institution design, and identified dualities as consequences of the design choices made in these areas. However, the resulting research agenda cannot be considered "complete". Additional research questions could be formulated for each of the parts of the agenda. This is inherent in the fact that the research questions and associated research agenda are based on the collective input from a selection of the collaboration research community. In this sense, the research agenda is the beginning, not the end. It is meant to inspire and inform future studies, not limit this area of study. We trust that future research will further extend the research agenda.
Second, the research questions and statements were sourced from collaboration researchers and not practitioners. This was intentional because a machine teammate, as envisioned in this study, has not yet been sufficiently studied in the field. Hence, the contributions can be considered as qualified opinions from a group of informants that are trained to be open-minded, neutral, and knowledgeable about the domain of interest here. Our results, however, might be biased toward what researchers find relevant to study and do not necessarily fully capture professionals' interests. Therefore, future research could acquire evidence for the (non) existence of dualities from organizations that are early adopters of predecessors of machine teammates, e.g., a chatbot or a digital assistant.
Third, the indicated relationships between association dichotomies are partly based on interpretations from content analysis and were not necessarily stated as such in any single research question. The construction of these associations was frequently built based on multiple research questions and statements that addressed these concepts and sometimes also on different levels of abstraction. Moreover, it is not our intention to suggest any kind of causality between the theoretical concepts as we do not yet possess sufficient understanding to argue for the directions of effects. For example, our association dichotomy "more visibility -loss of privacy" could also be argued that more need for privacy might lead to less visibility. Future research should, therefore, explore to what extent the suggested association dichotomies are well correlated and can explain the changes in collaboration practices and outcomes when a machine teammate is present.
This section summarizes the received research questions and their association to either design areas or dualities. A total of 215 contributions were categorized as comments, too general, or out-of-scope contributions and are not listed here.

Design areas
Machine design 236_3_Provide machines with strategies for understanding metaphors and contextual sentences 235_1_How do human teammates behave socially toward their machine teammates in different team constellations, e.g., with or without other human teammates?
215_2_Is it useful enough and compact enough that a person will want to take it with them all the time.
appearance 178_3_What should maintenance of machines as teammates look like?
231_7_Does the apparent gender of the teammate matter? Do other physical characteristics matter?
231_8_Does it matter if the teammate is unseen, i.e., just a voice or just text? 231_9_If seen, does it matter whether the teammate is a cartoon/ avatar or looks like a real person 231_10_Should the teammate have a physical (as opposed to virtual) form, i.e., is a physical robot.
231_11_Should a physical robot look like a real person? 256_1_How should the machine teammate look like? Should he/she be human-like or just an invisible computer system? 168_1_How to shape and utilize interfaces between machines and humans (e.g., text-based, speech, or nonverbalized)?
231_12_Are there individual differences in how people respond to a teammate? Does it vary by age, gender, personality, cognitive ability, and familiarity with computers?
235_2_Which communication mode (speech based, chat based etc.) is suitable for which kind of interaction?
185_6_Comfort: e.g., fluffy texture of teammate? What are implications here? 175_2_How do we visualize or embody machines as teammates? 225_2_What is the effect of different human-machine interfaces (touch, visual, audio, brain-computer, …) on the effectiveness of the whole human-machine system? 335_1_How should the interfaces look like through which we communicate with machines? 347_3_How will individuals react to machine if those display, or not, emotions.
264_5_Which types of interfaces are human workers most comfortable with? (e.g., regular computer terminal interface or humanoidlooking robot) 264_6_Do different cultures prefer different interfaces? 269_5_Do they have mimics and a face? 274_5_Can machines be sexually abused? Is it better to give machines an asexual appearance? 275_3_How do we design the interface of intelligent cognitive assistants to make the collaboration between humans and machines more enjoyable and effective? 289_2_Should machines as teammates have an eternal body?
Sensing and awareness 179_5_What sensors should the machine use (just plain camera, Heat, Movement, or heart rate…)?
220_2_Machines need to understand when people are talking to them independently from a certain keyword (derived from context) 221_2_How can machines infer emotion from humans? 235_3_Should machines act emotional, empathic,… and how can we implement this? 269_6_Should machines be emotional at all? 189_2_Enable machines to represent and process human emotions and states-of-mind 221_3_See first question, machines need to infer intentions from emotion, body language, etc. they should probably also be able to communicate emotions 319_8_Should we design teammate machines to be empathetic? 221_4_How can machines interpret messages from humans to understand intentions?
272_3_There is a lot of technical growth in this area, currently visual (audio/image/text) and speech are the main inputs to machines, how about smell, touch, and intuition?
274_6_Nonverbal communication-body contact, how close should they come, do humans like distance or being touched by the machines? 322_1_As agents can imitate and read human emotions "even micro expressions," how will this alter our relationship with our autonomous agents?
274_7_How good is speech processing, so that humans aren't reminded every time a machine does not understand, that it is not human?
302_1_Human abilities such as talking or humor may make communication with a machine entity more familiar.
312_10_What is the role of machine agent "personality" in collaboration?
Learning and knowledge processing 178_4_How should machines as teammates learn and how should they share their learning with their collaboration partners?
185_7_What type of memory is required for immediate interaction and what type of memory to learn from? 268_1_When a human makes decision, the decision is based on several knowledge areas and disciplines with complex relationships. How a machine can be programmed to contain knowledge of different disciplines? What disciplines should be included?
266_5_Where are we going to get the data? Or how are we going to mine the data?
269_7_Can they learn while we communicate? 268_2_Can machine's behavior and attitude be affected by the human collaborator (as seen in human teams)? If so, how it should be incorporated in the design process?
329_2_How to draw inferences from information? 319_9_Design -Supervised or unsupervised AI? 312_11_Are there scenarios where a less conversationally capable machine teammate produces better outcomes than a more capable one?
179_6_How do we build up and maintain the knowledge base of the machine teammates? How can the system learn?
207_6_Should the machines be designed with the capacities such as human brain or unlimited resources? 235_4_How can machine teammates remember the history of their interaction with different team mates and distinguish different team members?
268_3_Machines need to make subjective decisions based on their experience like human teammates. How this experience is gained by a machine as opposed to the experience and knowledge a human teammate gains through years? 289_3_Should machines as teammates have eternal recording power? 289_5_Can machines as teammates properly understand problems to be solved and share goals with humans? 269_8_Do they understand many languages? 319_12_Language -what kinds of language should teammates machine use?
221_5_How can machines read body language from humans? 220_3_Machines need to differentiate between serious requests to action and social chatter. 302_1_Information must be structured and made available in a machine-readable format, i.e., if we use emails for collaborative teamwork, the machine agent must be equipped with proper tools to understand such communication and be able to participate.
Conversation 168_2_What code of conduct is required for machines collaborating in teams with humans? Do machines need to be polite?
185_9_What influences cultural aspects in developing teammates, such as what is perceived as "polite"?
189_3_How to enable machine to be part of development and use of private languages, e.g., office or family jargons 167_1_Should machines as teammates be politically correct in their interaction with and about other co-workers? 220_4_Machines need to understand irony 256_2_How should the machine teammate interact with his/her peers?
215_3_Clearly your machine would have to be able to explain to me anything in your questions: What is RQs? (for example) One issue is designing the ability to socialize with the participants as well as transmitting social aspects such as the commitment level to a concept.
179_7_How can the machine learn how to interact with others (e.g., turn-taking? expected length of response? …) 168_3_When more than one machine collaborates together in a mixed team of machines and humans, should the machines be able to communicate among each other in a way that is incomprehensible by humans (machine language)?
274_8_Understanding humor, sarcasm, the context is still too difficult 312_12_Should interactions with machine teammates reinforce human-to-human communication norms (politeness, etc.)? 319_13_Tone -should teammate machines have a sense of humor? 233_3_Machines might not be intelligent enough to understand physical languages and other subtle expressions, how to promote collaboration in this case? 312_13_How does the machine teammates conversational capability influence perception and behavior?
265_2_Natural language processing and interaction with humans in the sense of interfaces (not interaction as in collaboration…) Architecture 256_3_What are the key architectural components of a machine that can function as a "teammate?" 231_13_What kinds of devices will the teammate best run on? 189_4_Miniaturization and distribution of machine learning models, e.g., as part of distributed on-device deep-learning architectures.
207_8_Should the machines have emergency button? 171_1_Are there are technical limitations that must be coded into the machines to avoid harmful outcomes? 327_7_How do humans control the "off" switch? 220_5_Will we built environments for robots or will we build robots for existing environments (wheels vs. legs)?
220_6_How can we produce enough clean energy and batteries for all the machines without destroying our environment? 220_7_How can we further reduce power consumption for processing power while at the same time increasing processing power? 231_14_How to structure organizations, and how to best deploy the machines. Will they be personal assistants for each person, or will they be shared?
264_7_Who is going to configure the machines, manage them etc.? 268_4_What design elements should be considered to create socially flexible machines as teammates?
Transparency & reliability 237_1_How can we create algorithmic processing so it can be reviewed by humans at various stages 256_4_Can the machine teammate explain his/her recommendations? How could he/she do that? 237_2_How do we make deep-learning algorithms understandable to humans?
269_9_Do we still understand the algorithms they are based on? 322_2_Autonomous agent "Responsibility and Trust": The machine learning models that underpin autonomous agents need to be as transparent as possible. Autonomous agents need to be able to explain their actions or behavior that we can understand.
203_3_Who is in charge of acquisition, programming, testing, and determining when behaviors are flaws or just favor one stakeholder over another? 220_8_How can we transfer "personality" of one robot into a successor, if the original one is broken? 346_9_How will machines as teammates weigh reliability vs. validity issues? 346_10_How will machines teammates' evaluation of reliability and validity trade-offs in studies affect the quality of human/machine research collaboration?
268_5_Machines work perfectly until they break down. A minor technical issue may cause a significant social and collaboration issue. How can we keep these issues at the minimum level?
272_4_What are the techniques to fall back, in case AI is shutdown unexpectedly?
319_14_Data -what kinds of data should we use for training and how often should we update to ensure accuracy? 322_3_Autonomous agent "reliability" and "predictability": As Autonomous agents are designed-based deterministic and nondeterministic decision-making algorithms, new system verification methods should be researched/studies. Because these agents operate in partially unknown adversarial environments and acting upon ambiguous information, new verification techniques are required to confirm that a system does not have undesirable behaviors or intents.
322_4_Cyber Autonomy: As Autonomous agents are designed based on adaptive-and reinforcement-learning algorithms, new cyber security methods should be researched/studies to avoid traceless cyber attacks Design methodology 178_5_How can we design machines as teammates in a humancentric way? 175_3_How can we model decision making or collaboration processes where some part is performed by machines? To what extent is this different from people's activities?
171_2_How ready are our tools for engineering collaborative processes (e.g., facilitation process model) for modeling future collaborative processes?
181_2_What is needed to model human-machine collaboration in a structured and understandable way?
171_3_Do you need to adapt current collaboration engineering basics (e.g., the COPDA) to account for the changing role of technology (from tool to partner)?
181_3_How does a workflow of human-machine collaboration look like?
312_14_What processes do people use to evaluate machines as teammates?

Collaboration design
231_15_Are there cultural/national differences in response to teammate?
203_4_What is work? What is the difference between play and work? For what purposes will teams be drawn upon? What roles will humans have available to play in contrast to roles that machines will play? To what extent will physical and cyber realms remain distinct (they already are starting to blur at the edges)?
261_1_Under what conditions could we collaborate with teammates, determinants, and enablers and inhibitors?
Team design 168_4_What is the ideal team size for machines as teammates? 231_16_When introducing a teammate, does team size (number of humans) matter? 319_15_How many teammate machines should an organization have?
312_15_What is the optimal ratio of human to machines in a team? 231_17_How do teams of humans compare in performance with mixed teams (or dyads)? 231_18_Is there any advantage in having more than one teammate? 178_6_What types of relationships do humans build with machines as teammates?
207_9_What role the machines should be in the relationship with "its teammates?" 262_3_What challenges are there in using this technology in virtual teams? in collocated teams? 220_9_How can machines not only passively answer questions but actively participate in a collaborative problem solving process? 331_9_Who/what will lead the meetings? 335_3_How should human-machine teams be composed to reach optimal outcomes? 347_4_Management of teams comprises of people and machines 264_8_Who's the boss? The human (emotions) or the machine (data)?
265_7_Hierarchy between humans and machines 265_8_Is there a hierarchy between human and machine teammates -so, do I need to negotiate with the machine, or can I tell it that it needs to know that "A is B?" 267_1_What impact will machines as teammates have on an aging workforce and are there differences in outcomes between digital natives and digital immigrants?
269_10_Does it make a team more efficient to have bots as team mates?
302_2_How does the presence of machine teammates influence the conversation in terms of a subjectivity-objectivity spectrum? (i.e., if we test two groups: one with only humans and the other one mixed, would we find that human-only teams tend to discuss subjective qualities of teamwork more than the other group? how does this influence endresults of the team work?) 312_16_When should a machine teammate proactively provide information, compared to responding only to human queries (e.g., Microsoft Clippy)?
264_9_Diverse teams and cultural implications -the machines have to be suitable for workers with different cultural backgrounds 181_4_Can competence profiles build the basis for a matchmaking mechanism that helps to create groups of humans and machines to solve specific tasks? 312_17_How well do people handle being outnumbered by machines in a team?
Task design 168_5_What kinds of tasks are most suitable for human-only, machine-only, or mixed teams? 236_4_How to identify applications and problems that can benefit from the integration of human and machine knowledge? 230_2_How to identify applications and problems that can benefit from the integration of human and machine knowledge?
181_5_What are task that have the potential to become automated/ executed by a machine and what are task that need to be executed by humans?
207_10_Can the machines participate in any/just only parts/procedures it likes in the projects?
256_5_What kind of questions can we ask the machine teammate? 181_6_What are the foundations and demands to decide whether a task can be executed by a machine, human or human-machine collaboration?
256_6_What are the limitations of the machine teammate? 256_7_What are the things the machine teammate can bring to the table?
256_8_What are the problems we typically ask a teammate in group discussion or project? How many of those we can ask a machine teammate?
261_2_For which tasks machines could help? 167_2_In which domains will we see machines as teammates? 178_7_What collaboration infrastructure do we need to include machines as teammates into virtual/physical collaboration?
269_11_What the most useful areas to use machines as team mates?
269_12_Are there areas where we should not use them? 319_16_Duties -what kinds of duties should machines have? 319_17_What areas do we want teammate machines to influence and how should we develop them to do so?
181_7_What are the core skills competences of AI? What are the core competences and skills of humans? 220_10_How can we find a good compromise between general purpose A.I that can do anything and highly specialized machines built for a single purpose?
221_6_What stakes can or should they represent? 221_7_What stakes can a machine represent in a decision-making process?
236_5_How to organize decision-making procedures in presence of conflicting requirements?
227_2_How will the orchestration work in the way addressed in the RQ above, and how will people/machines/hybrid teams work together?
Work practice design 256_9_What are the organizational processes the machine teammates can be a part of? 233_4_We should also consider various degree of collaboration. 231_19_How does introducing a teammate affect team process? 178_8_What effects do machines as teammates have on collaboration processes?
167_3_For which communicative group processes (e.g., coordination, knowledge sharing, and evaluation) should machines be trained? 225_3_What is the effect on different group processes (creativity, group think, and problem solving) when intelligent agents are involved?
231_20_Does the mode of communication (spoken, written) affect how the real person relates to and works with the teammate?
233_5_In the case of joint projects, how to ensure smooth communication?
227_3_How can we orchestrate human robot teams to enhance cooperation?
261_3_How to coordinate tasks with machines? 185_10_Differences when working in organizational environment: hierarchy implemented in communication and behavior (e.g., CEO vs. employee) 268_6_Do machines learn the team process at the same pace as human teammates or they are already pre-programmed to know the team rules and norms?

Institution design
Education and training 175_4_What needs to change in our education programs to have students become savvy in working with AI machines?
175_5_What needs to change in our education programs to train students to develop productive AI?
225_4_How to make learning in educational institutions and beyond design oriented so that education becomes an enabler and shaper of digital transformation rather than a follower?
225_5_What are the skills and competencies that need to be acquired in the educational system, and how to best ensure these skills are acquired?
236_6_Evolving educational programs to include skills for designing, interacting with and validating intelligent artificial agents.
256_10_How to train people to collaborate with the machine teammates?
178_9_What collaboration capabilities do humans need to build to engage with machines as teammates?
178_10_How can we help humans to build such collaboration capabilities?
178_11_What second-and third-level learning effects can we expect from humans and from machines as teammates and how should we best address them?
178_12_What organizational capabilities do we need to build to include machines as teammates?
347_5_Training to working with these machines 267_2_How can we best implement training (acceptance of training and transfer of training) for working with non-human teammates?
275_4_Which skill set do human employees have to develop to maintain their employability?
319_18_Training -how should we train and how often? 322_5_Long-term training and education system redesign when agents can conduct mass decision making tasks in our society 262_4_What legislation can be passed to provide retraining of people who have been replaced by the technology?
265_9_What "educational" methods from the educational sciences need to be transferred to machine learning to teach robots what kinds of goals and approaches to living are "valid," valuable, and will make them appreciated (is that even something that robots would aspire to?) 327_8_How do children's abilities fully develop in a world where the cognitive/affective load for learning is born by the machine?
Responsibility and liability 233_6_Will machines be responsible for what he did/what he said? 261_4_To which extent machines are responsible for their acts, risks, and ethics? 261_5_Responsability in case of problems -who is responsible: the machine or its designer? 262_5_Who has responsibility for systems/robots relying on AI. For example, if there is an accident involving a driverless car, is the manufacturer of the car to blame or the system designer for the system running the car? etc.
244_2_who is responsible for the decisions of machines? 171_4_Will a human collaborator always be in charge and "responsible" for the result of the collaboration process or can this "responsibility" be transferred to an AI collaborator?
178_13_What rights and obligations do machines as teammates have?
331_10_Who should be punished for a wrong decision? 335_4_Who is held responsible for decisions made in human-machine collaboration?
268_7_Machines are not responsible for their deeds. How can be one responsible for a mistake made by a machine that happen to have significant negative social impact? 269_14_Who is responsible for the input or decisions? 319_19_Responsibility -who's to blame when things go awry? 327_9_What is our responsibility toward vulnerable populations (children, the aged…)?
289_6_How should we deal with machines as teammates if rational but ethically inappropriate proposal? 266_8_How can we make sure that our new teammates see us as people rather than any kind of variable in their optimization problem?
220_11_Do we need social conventions for how to deal with robots (don't treat it bad, don't be mean to a robot)?
178_14_What effects do machines as teammates have on the professional norms and values of its collaboration partners? 221_8_What stakes should/could a machine represent ethically? 312_18_Trolley problem -how closely should the AI's ethical decision making mimic human evaluation?
237_3_What governance approaches are needed to set up a machineas-collaborator work context? 269_13_Do we need new laws in some areas? 185_11_Authentication with teammates: who and how can interact in what role with the teammates?
236_7_Which is the original root of legal/ethical liability of an agent?
235_6_Who is legally responsible for a machine teammate actions? 175_6_What are the legal consequences of using machines during decision-making tasks that may end up having negative outcomes, e.g., damages?
189_5_Without transfer of liability, how can we avoid the human being liable without holding effective power?
289_7_As a result of machines as teammates acting autonomously, what kind of law should judge them in case of harming humans? 302_3_Machines must legally bear the same responsibilities as humans, so that no one can use them to conduct illegal business practices.
327_10_We have systems and laws to regulate all forms of human activities (the environment, marriage, and family, education…). Do we need new systems for organizing this collaborative relationship? 203_5_Why would any behaviors by machines be permissible (or not) that are acceptable for humans? Given the likely speed of evolution of machine capabilities, how can ethical and legal constraints be administered in a timely manner? 312_19_What should be the legal ramifications for machine teammates in a military context?

Association dichotomies
Higher quality of decision-making -reduced capability to criticize 215_4_Can the machine determine how reliable, accurate, or truthful the information from a source is.
220_12_How can machines judge the reliability of information to use only valid ones as a basis for reasoning? 220_13_How can machines explicitly ignore facts they've learned to answer "what if" questions for different scenarios? 215_6_Can the machine determine what is useful to a given individual?
262_6_How can we use technology to improve cognitive processing? 225_6_How can we educate people to be critical information seekers and users?
236_8_There is here a big challenge IMHO: how to interact with automated procedures without losing criticism 178_15_How do we cope with errors of machines as teammates? 256_11_If the decisions based on machine teammates' recommendations turn out to be wrong, who to blame? 179_8_How do we handle wrong answers? 207_11_How should the machines act when "its teammates" are wrong?
262_7_To what extent should we be concerned about monitoring/ controlling machines that use machine leaning to provide AI answers to work problems?
178_16_What effects do machines as teammates have on human decision making? 331_11_What is the difference between collaboration (with machines) and the concept of human augmentation? 347_6_Understanding how machines "think" may influence how their decisions about their own decision making and cognition 266_9_What if my team members are constantly improving in predicting choice (e.g., bargaining outcomes, risky choice, behavior in conflicts, etc.) 272_5_As almost all AI depends on available data, we need to always question the validity of data in this day and age. For instance, if say, data for housing model is based on say size of the lot. But in reality, we need other input such as no. of bedrooms, location from highway etc. If these new parameters are not available. Then one main question is can we generate new data with needed parameters for AI to learn 289_8_Can humans rectify the mistakes pointed out by machines as teammates without hesitation?
289_9_Is the proposal from machines as teammates useful as much as humans or more? 312_20_How can machine teammates be used to overcome human cognitive biases in decision making?
315_6_What are the impacts on our brain and cognition? 319_20_Are teammate machines likely to follow bounded rationality or be utility maximizers?
167_4_How will the accuracy of individual decision-making processes change?
167_5_How can machines as teammates recognize cognitive biases in collaborative decision-making and take countermeasures?
178_17_How can machines as teammates compensate for human cognitive limitations and biases?
178_18_What new decision heuristics emerge with machines as teammates?
178_19_How can we prevent/compensate for human cognitive biases?
331_12_What kind of cognitive biases will emerge in mixed groups? 346_11_How will machines as teammates recognize political bias in research?
346_12_How will machines as teammates respond to political bias in research?

Increased pace of work -increased cognitive load
167_6_Will machines as teammates increase the pace of collaborative efforts and with which positive or negative effects? 178_20_How do and should humans react in cases in which there is insufficient time for machines as teammates to fully explain their contributions?
220_14_How can we find a good compromise between long running calculation processes on a large body of data vs. a quick answer that is good enough?
231_21_There are timing issues. How much lag can there be between asking the machine a question and getting an answer? 221_9_How can people have a work-life balance when collaborating with machines that are always "on?" 185_12_RQ: Where to find the turn-off mode, how is standby defined and how to awake from standby?
236_9_Training people in adapting fast to new assignments and tasks 220_15_How can machines represent human context to quickly get "in sync" with a human collaborator? 220_16_How can we represent human cognitive context in a machine so that the machine can think along with the human?
175_7_How can we ensure that machines as teammates provide information in a way that can be correctly and timely processed? 233_7_How to deal with misunderstanding between team members and machines?
235_7_How can machines be improved in terms of better understanding meaning and context of human language?
220_17_How can we ensure that machines have understood questions and calls to action correctly?
207_12_What could happen if miscommunication between human and the machines?
233_8_There might be misunderstandings between machines and human beings. How to address this challenge? 215_7_It has to recognize for every individual it is dealing with what are their limitations in knowledge or analysis for any topic that is being 220_18_How can machines emulate the human short-, medium-, and long-term memory to efficiently work with humans that have this structure without suffering from the disadvantages?
178_21_How can machines as teammates argue for their contributions considering the limited cognitive capacities of humans?
189_6_Study machines role in permanent impairments to inter-personal sharing of emotion.
346_13_How will workload be organized with machines as teammates?
346_14_What issues might arise when organizing workload with machines as teammates?
266_10_How can AI technology applied in our teammates overcome and exploit human limits?
267_3_I believe the biggest technical RQ would be to address the fidelity (i.e., how human-like the machine is) of machine teammate. It is possible humans will not accept or trust a machine coworker that is of low fidelity (e.g., a Second Life avatar) 269_15_Should machines stop working/talking with you because you need to rest? 312_21_How can machine teammates best work with the limitations of human psychology? 215_8_Can the "machine" recognize the limitations a person has in the learning process?
171_5_Should machines mimic the time it takes for human to process information and to derive conclusions? 256_12_How fast should the machine teammate respond to make his/her interactions being on par with other human teammates? 235_8_How can we "slow down" and reduce the complexity of machines computational power to the capacity of their human peers? 235_9_How can we ensure transparency and speed of, e.g., machines' decision preparation processes to match human decision makers cognitive capacity? 237_4_How do we make algorithms more understandable to humans, as they are used? 346_15_How will cognitive abilities be balanced during the research design process with machines as teammates?
233_9_How to ensure timely communication? What if machines misunderstand the instructions from human? 235_10_How do we organize the handover of work between machines and humans so that each of them can understand and handle the input by the other? 354_3_Mapping AI decision making to our cognitive understanding of the reason for the decision Increased creativity -lack of serendipity 215_5_Be useful to some of the possible participants. Exposing disagreement is a very basic functions that has to happen to generate creativity 225_7_Many algorithms in use in the web today (information recommendation, filtering, ranking etc.) are based on social closeness and similarity (supposedly leading to filter bubbles). How can we create alternative algorithms that spark creativity in people rather than reinforcing existing views? 201_1_Can you machines have insights. Most great ideas come from insights that are then justified with data 237_5_As the relationship between machines and humans becomes more intertwined, how do we ensure that humans' creativity does not become constrained?
237_6_How should knowledge creation be dynamically shared between machines and humans?
244_3_Not only generate suggestions but also be able to explain why suggestions have been generated 175_8_Can we make machine behave creatively? 207_13_How should the machines and human go in the same original direction of solving a particular problem? 347_7_How can machine recognize good solutions for open and complex problems 331_2_Capacity to generate win-win solutions More efficient reasoning -fewer human-driven decisions 264_10_Who is the "manager" and who is the "worker?" Should machines make data-driven decisions or should it still be done by humans?
312_22_How often does a machine have to be right before we decide to eliminate the human from the decision completely?
207_14_Should the machines be treated like a human when come to its opinion? 289_10_Can machines as teammates persuade others when their proposal is judged better than the proposal by others?
221_10_If a machine is a team mate, should it be/represent a stakeholder in a decision making process?
185_13_Teammates as "buddies": how to what extent can and/or should a machine "behave" to get its current problem solved 331_13_How to develop the mechanism of persuasion with mixed groups? 331_14_Will machines as teammates be able to compromise? 269_16_Can they decide something or just recommend? 312_23_Should the machine ever have the final say in a decision? 312_24_How can a machine teammate accurately assess its confidence in a decision or recommendation? 312_25_What factors influence decisions to rely on machine recommendations over time?
312_26_When, if ever, is it acceptable for the machine to override a human's decision?
244_4_Will machines be allowed to make decisions? 215_9_Can it act for me in a given situation?! 312_27_Should the machine have the final say in a life-or-death decision? 256_13_How do the "knowledge" the machine teammate has should look like so that he/she can draw inferences, insights, and relevant information from? 256_14_How do the machine teammate determine if the information and insights he/she offers is relevant to the ongoing discussion with other teammates? 221_11_Who determines the ethics/values of machines, as they will become a more reliable source of information than people/experts to make decisions? 207_15_How should the machines classify the related information from unrelated information regarding a problem?
354_4_What level of decision making is humans comfortable with a machine making?
274_9_But, e.g., should machines decide on assisted suicide when they diagnose humans as terminally ill? At least their recommendation will have great impact on such decisions on turning off life-sustaining machines.
312_28_When, if ever, is it acceptable for a machine teammate to make a final life-or-death decision (e.g., medical or military context)?
221_12_If we have machines as ultimate reasoners, is there still room for negotiation on what people want? 312_29_How can information best be presented when it contradicts the assessment of the human?
231_22_Will the machine produce cognitive overload? Will it increase workloads. Will employers expect more of employees who have the machines? 215_10_It has to be a servant for everyone in the group that is using it as a teammate! to deal with a common concern.
201_2_How do you recognize the intellectual contribution of a machine?
201_3_Recognizing the contribution of the machine. 207_16_Should the machines be rewarded with its contribution to the projects?
167_7_How will perceived acknowledgement from human-human and machine-human for work in a team with machines change? 331_15_How to divide rewards/punishments between humans and technology?
302_4_Machine input could be misappropriated or abused if proper credit is not given.
268_8_Are machines and human actors going to be compared and evaluated in the same way?
274_10_Can they be friends, sport partners, or partners?
More anthropomorphism -more manipulation 235_11_How human-like should machine teammates appear or what are the characteristics they should have implemented to be useful and likeable partners? 231_23_Will humans treat teammates like a human friend? 175_9_To what extent will human team mates anthropomorphize machines? What are the consequences of this?
171_6_How "human" do we want machines to become? 231_24_A human tricks another human by pretending the machine is human.
175_10_How can machines be (mis)used to strengthen individuals' positions in a team? How can machines help individuals to have more power or influence in a team process? 171_7_Should machine collaborators be "flagged" as being machines or is it better to "disguise" them as being human collaborators?
168_6_Should machines (try to) act as human-like as possible when collaborating with humans or should they have certain characteristics that makes them distinguishable as machines by humans immediately? 220_19_Should robots be allowed to look like humans or must it be visible at first glance whether a human looking thing is a robot?
185_14_What are social boundaries for a teammate? 233_10_Someone might control machines, to let it perform what human beings want them to do. How to avoid this phenomenon from happening?
269_17_How human-like should they be? 269_18_What if other people do not recognize that they are bots and not human? 267_4_How does having machines as teammates lower organizational politics in the workplace. Is there a lower need for political skill (power, influence, and persuasion) in an environment where machines can't be manipulated/biased in the same way that humans can?
More responsibility -loss of control 171_8_Will humans at some point fear that machines could take over?
189_7_People "taking the back seat" as supervisors of fast and complex machine activity (like in autonomous vehicles) can translate into unenforceable nominal control 189_8_Diverse proficiency levels in person-machine interaction may lead to informal transfer of power and leadership while organizational roles stay nominally unchanged. This may impair assessment of intellectual capital and perception of the organization's capability 231_25_Assuming the human is in charge of the teammate, what sorts of leadership approaches will the human use 231_26_How much control should the human have over the teammate?
178_22_How can we address issues such as dominance by machines as teammates?
261_6_Human control in case of the machine become more intelligent than humans 272_6_First question to ask is, in the power hierarchy where are AI teammates being deployed?
268_9_Human teammates may feel sometimes that the machines are superior, as they never forget what they have heard or learned in the past and they process the information fast and accurate.
221_13_If the machine has better answers than humans, if it can process more information, should it get more authority?
256_15_Will the boss replace human teammates with machine teammates if the machine teammates are more helpful than the human teammates?
178_23_To what extent and how are machines as teammates responsible and accountable for their contributions or for the consequences of decisions taken upon their input?
189_9_Novel organizational relations are needed to make such leadership visible, e.g., via new non-IT roles related to machine coordination 178_24_How can and should responsibility or accountability be shared with machines as teammates?
237_7_How should responsibility and control be assigned? 272_7_Are AIs allowed to grow in their position? 272_8_To what level of influence to the top of ladder are allowed to AI?
289_11_Are machines as teammates allowed to command humans? 289_12_Can humans recognize machines as teammates as leaders? 319_21_Should we allow teammate machines to have power and control in an organization? 319_22_Can we have a machine that is a teammate and a leader? 327_11_How do humans remain in control? 327_12_Where is the locus of control in the relationship between machine and human?
185_2_Differences when working in organizational environment: hierarchy implemented in communication and behavior (e.g., CEO vs. employee) More transparency -loss of privacy 225_8_How can we ensure data collected about a person and the inferences taken on it is transparent, scrutable, and controllable by the person?
225_9_How can we make the decision-making process for decisions taken by humans in collaboration with machines transparent, inspectable, and scrutable? 227_4_Communication between humans and robots and transparency of what the machine does 220_20_How can we ensure all the data collected for personalization of the human machine communication is not abused?
179_9_How can we make the (unnatural) mixture of high competence and low competence of machines as teammates transparent to the users?
185_15_Teammates: usage of visual data from built-in cameras (also ethical aspect) 244_5_How to generate certainty percentages for AI-suggestions/ evaluations?
185_16_Data privacy: what data have to be recorded and where is it located?
207_17_What is the rule for the confidentiality of the projects/information should be applied to the machines?
175_11_What safe guards need to be in place when organizations use machines that become privy to confidential information? 231_27_Privacy issues in terms of what people tell the machine and in terms of what the machine does for the individual.
175_12_Do teammates feel that machines also monitor them apart from participating in decision making and problem solving processes? 220_21_Potential of surveillance 189_10_Currently, machines are ethically neutral, with the possible exception of built-in obfuscations for privacy preservation.
264_11_If machines are to work as "real" teammates, they should store some info on their human counterparts. What kind of data can/ should be saved/logged and for how long? When to use this data?
264_12_Human teammates relate to each other by talking about their experiences. They often store pieces of information about others and use these pieces later on in similar contexts. How can machines do it similarly without making humans feel uncomfortable as if they are being watched/recorded all the time? 264_13_Which data will be saved/logged? Who can access these data?
265_10_What do I/my team/my organization want machines to know about myself/my team/my organization?
267_5_Cybersecurity will become even more important as there is a move to having machines as teammates. How do organizations best prepare for possible data breaches, as the machine employees can be hacked with negative consequences for the organization and human workers.
267_6_Machine coworkers could have nearly unlimited storage space to record/monitor the workplace environment. This could cause privacy/ethics issues for human workers.
269_19_What kind of data should they process? 289_13_Can we use personal conversation records for learning machines as teammates? If we can use them, who is responsible for leakage of privacy?
237_8_What are the types of representation gaps that occur between machines and humans? How much visibility into the machine knowledge do humans need Higher adaptiveness -more misbehavior 220_22_How can we allow robots to learn from their perceptions without the fear that they learn bad behavior?
189_11_As we empower machines for autonomous decision making, do we need to build them in such a way that ethical behavior is encouraged-preserved? (Asimov's Robotics Laws) 220_23_How can machines build up something like a moral conscience?
220_24_Can we built machines with compassion but without aggression?
220_25_Machines learning about prejudice from humans and making them explicit 215_11_The same technology could be turned into a bias agent to sell things. Can the system be influenced by the users individually to be able to treat everyone the way they wanted with respect to the needs as perceived by the human the machine is dealing with.
231_28_The human asks the teammate to do something that would be unethical for a human to do, e.g., send nasty messages and trick someone.
220_26_How can we allow robots to learn from the internet without the fear to build a super A.I. that outsmarts humans? 167_8_Should machines as teammates have adaptive personalities to fit individual preferences? 235_12_How can machines be improved to better adapt to their human counterpart and the specific situation? 236_10_The emotional expressions provided by machines should be adaptive, for real 171_9_Should machines mimic a certain identity or stay neutral or even adapt their identity based on the identities of the human collaborators?
220_27_Machines need to understand that people change their mind from time to time 220_28_How can we teach robots to "bend" the rules from time to time, without the fear that they will use it against us?
207_18_How should the machines adjust its communicational level/ channel to match with "its teammates?" 220_29_Will machines build emotions on their own without having them designed for this ability just because of their similarity to human brains?
235_13_Which displayed personality traits and emotional expressions foster desired behavior of the human team mates?
269_20_What if they are treated bad? 269_21_Can they have a sense for the upcoming of emotions and adapt their communication strategy?
289_14_Do humans decline or be corrupted due to machines as teammates?
312_30_Should machine agents' personalities be changed depending on the situation, or who they are interacting with? 319_23_How can we teach machines to become good teammates? 319_24_What incorporates a bad teammate machine? What should we do to avoid building those?
Higher value creation -extreme power shifts 175_13_How do we demonstrate the value of machines as teammates?
266_11_How will machines as teammates affect the firm's productivity?
175_14_How can we quantify the value that machines as teammates bring?
215_12_There has to be value for the individual humans to make use of this machine intelligence 265_11_Will machines be our slaves who work for us and we enjoy free time (think: old Greece)?
175_15_Should organizations develop machines in house or will we have COTS AI?
185_17_Business models for "booking teammates" 256_16_How much does it cost to hire/build machine teammates vs. human teammates for the same task? 233_11_How to control the cost of collaborating with machines? 220_30_If machines are doing "real labor," are they getting paid so that there can be taxes on the wages? 168_7_How can work done by machines be taxed? 274_11_How much tax should employers of machines pay for using them? Unlike humans, they work for "free" at the moment.
262_8_What is the cost of retraining workers who have been displaced by the technology?
175_16_What is the ROI on machines as teammates? 262_9_Clearly the cost/benefit analyses need to be performed for the implementation of these new technologies. However, the costs of acquiring and implementing robots seem to be dropping radically and the benefits seem to be increasing. What about the costs to society in terms of people who are being replaced by machines.
189_12_Machines as workmates (as opposed to machines as tools) have new cost profiles that require novel micro and macroeconomic studies 220_31_What happens to traditional economical systems if labor can nearly 100% be replaced with money? 327_13_Will collaborative machines be a scaffold or a crutch? 234_3_How do we finance the state (welfare) in a context with fewer and fewer paid workers? 235_14_Should we pursue any technologically possible automation potentials, even at the expense of human teammates quality of work, life, or employability? 225_10_How can AI contribute to a more inclusive society, i.e., not benefit the ones that already have a lot of power, or the ones that have more resources (monetary, social, and cognitive) than others?
220_32_How can we ensure that robots will not stretch the discrepancy between a few super rich people and a large body of poor humans to a new all-time high?
220_33_How can we make sure that those in need benefit the most from robots and not mostly the rich?
231_29_As we become more automated, we will need new economic models because not everyone will be able to have a decent job. If we don't change models, eventually a handful of people will own everything, as the rest will be starving on the street.
289_15_How does machines as teammates affect national strength? 327_14_Where does the power reside? 178_25_How do societies react to the shifts in power between those who have machines as teammates and those who haven't?
265_12_Who will own the machines and make money from itdystopia: rich people own machines, poor people have no work and no money 315_7_How to avoid the increase of the digital divide? 322_6_Fair trade policy as a result of access to autonomy and data 275_5_How do we establish governance structures to control machine to machine interaction?
231_30_Will people have to pay for the teammate? Will everyone have their own (like a cell phone) that can be used in a variety of settings 203_6_Who pays for the machines? Does the one paying for the machine determine its behaviors? Where benefits are created, how are they divided between human and machine teammates? 178_26_What value is created by machines as teammates and how is that value distributed among the collaboration partners?
267_7_How does additive manufacturing (e.g., 3D printing, etc.) through machines as workers and teammates impact organizational and national (GDP) outcomes?
315_8_What are the power imbalances/balances promoted by this approach to computing? 329_4_How these machines choose when there are conflicts between the benefits of the people they work for and the benefits of the people who build them?

Concept dichotomies
Affect positive/negative 185_18_Negative emotions and its priority of emotions (ranking) when dealing with human, e.g., frustration vs. ignorance 189_13_Humans "taking the back" seat may affect their self-esteem and perception of standing. Psychology studies are needed.
178_27_How do we deal with anger and frustration against machines as teammates?
312_31_How can we overcome an instinct to perceive machine teammates as job competition?
233_12_Collaborating with machines will cause a lack of belongingness. How to address this challenge? 220_34_Do we need courses for self-esteem so that humans don't get feelings of inferiority? 231_31_People's loss of self-esteem, status, and meaning as they get displaced.
267_8_What possible negative health and psychological well-being outcomes may occur as a result to the teammate? An example would be "does the worker have lower psychological well-being for fear of job security (machines will replace their job too eventually)?" How will these possible lower well-being outcomes affect the workforce?
178_28_How do humans deal with emotional stress due to machines as teammates?
231_32_How does introducing a teammate affect stress, i.e., will it be a stressor or will it buffer stress? 231_33_How much will people enjoy working with a teammate? 256_17_Should the machine teammate be empathetic to other teammates? Does he/she need to tell jokes?
312_32_What emotional factors influence continued use of machine teammates?
167_9_Will machines with humor improve team performance? 179_10_How can machine understand our emotions and appropriately react to them in a team setting?
272_9_How human teammates' emotional attachment to AIs influences their relations to other humans?
233_13_It's hard to build emotional bond with machines; however, emotional bond is important for team collaboration. How to address this challenge?
264_14_How do machine teammates influence the job-satisfaction of humans? Can machines also provide empathy and emotional support at the workplace? 269_22_How should they react on emotions? 185_19_RQ. Reactions to sentiments, emotions within a collaborative setting 256_18_If being a good teammate requires a person to be sensitive to the feelings of his/her peers, how could the machine teammate detect those feelings and respond in a sensible way?
168_8_How to motivate human teammates if the machine teammate always knows the answer? 264_15_Can these machines provide the emotional support and empathy that can be compared to human teammates?
264_16_Can machines provide the emotional support as teammates? 220_35_Are machines with emotions dangerous for humans or is it necessary to build machines with emotions for good cooperation between humans and machines? 266_12_How can we tackle the issue of perceived fairness in the tech-human interactions?
312_33_What emotional factors influence liking in machine teammates?
312_34_How can emotional factors be manipulated to increase trust, liking, and use? 312_35_Should emotional factors be manipulated to increase trust, liking, and use (ethical question)?
319_25_Should we design teammate machines to display any sort of emotions when we know that they are not capable of doing so? How are humans likely to respond to that?
185_2_Teammates: understanding emotion and sentiments Team knowledge augmented/depleted 167_10_Will intelligence and emotional intelligence decrease when machines join collaborative work?
225_11_What is the effect on reading and literacy, if more and more interfaces turn to be voice enabled?
230_3_Can artificial intelligence support decision making without depleting human knowledge? 236_13_Can artificial intelligence support decision making without depleting human knowledge?
189_14_Understand better the permanent loss of cognitive abilities of humans. Machines as co-processors for the brain. The case of "Nintendo pilots" 267_9_Researchers have to consider the impact machines as teammates can have on transfer of training and lack of reliance on memory in general. Human workers/teammates may be apt to not rely upon their memory if there is a machine there with all the knowledge (similar to people not remembering phone numbers anymore because they are in the contacts list in a smartphone). How can we curb mental laziness in workers who depend on their machines as teammates?
227_5_How can we create human robot teams in a way that fosters human capabilities rather than making humans dependent on robots?
319_26_Can teammate machines fill structural holes inside a company and between organizations?
227_6_It will be about using human skills and complementing them rather than making humans slaves of machines.
244_6_How to explain/visualize the reasons for AI-decisions/suggestions to humans?
265_13_Explaining reasons for actions 207_19_Do the information/answers from the machines play a significant role in the team's conclusion?
220_36_How can a machine explain its inferencing process, so that humans can retrace the logic behind?
185_20_Does mental model theory apply also to machines? (shortterm memory impact) 267_10_What is the perceived technological sufficiency of the machine/AI/VR teammate and how does it impact the human's ability to perform to the best of their ability with a machine as coworker? For example, if there is an avatar of a coworker, but it is not of high fidelity, will the human's ability to perform their job suffer as a result?
Technology accepted/rejected 347_8_Acceptance by those impacted by the solution 167_11_Will machines with humor be more accepted by coworkers? 262_10_How should robots be designed to look more acceptable to their teammates? 256_19_How to convince people to accept and respect the machine teammates?
264_17_Is the human workforce going to accept machines as teammates?
269_23_How should we design bots that they will get accepted as teammates?
264_18_Is human labor force ready to accept machines as teammates?
319_27_Acceptance -do employees actually work or want to have machines as teammates?
269_24_Will people accept bots as team mates? 171_10_Will human collaborators accept the input from machine collaborators?
171_11_Are there certain tasks where input from machines is more likely to be accepted than other tasks (e.g., support in facilitation versus generation of ideas)?
175_17_How can we increase acceptance of recommendations from machines vs. people? 354_5_Acceptance of AI recommendations and the need to avoid discriminatory practices by AI.
354_6_We have in society some people who have techno-phobia and will resist the use of technology even when doing so prevents us from reducing human suffering. How much human suffering must we allow to accommodate superstition and unreasonable concern about machines?
347_9_How would individual feel about solutions generated by machines, acceptance by individuals that have to carry out the work and 302_5_Is machine input taken as seriously as that of humans in collaborative work? (i.e., is it easier to dismiss machines when you don't like what they say?) 231_34_What are the DVs we care about? I can think of acceptance of the technology, preference for using it vs. a real person, resistance to using it, trust, how much people use it, performance with vs. without it, satisfaction with it, emotional response, sabotage of the teammate.
268_10_What design elements should be incorporated in the development of collaboration technology to adjust the needs of people with communication technology phobia?
267_11_There are individual differences in the acceptance of technology, which appears to be correlated with acceptance of miscommunication with technology (e.g., Siri misinterpreting a command/ question) -some people ask the question with more enunciation and some just stop communicating with the technology and give up. This makes it important to consider both the technical aspect (better voice recognition, etc…), while also considering how to minimize or improve lack of acceptance of communication with technology.
175_18_Will machines as teammates be regarded differently in different cultures? Which cultures will be more likely to accept machines as teammates? Which cultures will be more likely to benefit from machines as teammates?
264_19_Do some religions have issues with robots? 231_35_Will humans treat the teammate as if it is a real person. Will it talk nicely, chat with it, yell at it?
Trust created/lost 178_29_How can we facilitate trust in machines as teammates? 256_20_How much should we trust the machine teammate's insights and recommendations? 235_15_How can we design machine teammates to be trustworthy and transparent in their behavior?
267_12_With the move to machines as teammates, one of the major research questions that should be addressed is how does trust between humans and machines as coworkers/teammates develop over time. Is the trust link between humans/machines more tenuous or volatile than trust between humans and how do we address the effects of different individual differences (e.g., technology acceptance) on how trust is built and maintained?
274_12_Do humans trust the algorithm of these machines? 175_19_What are the differences in trust with respect to recommendations from machines vs. people? 220_37_How can we build trust between humans and machine teammates?
231_36_How do humans develop trust of the teammate over time? 233_14_How to ensure trust between team members and machines? 347_10_How do we develop trust and other emergent team properties 319_28_Trust -are employees likely to trust their machine teammates more than human? 312_36_What conversational design factors influence trust in a machine teammate?
264_20_To what extend should the machines be trusted? 264_21_Are human teammates willing to trust their machine partners?
312_37_How does contradicting the human affect the human's trust in the machine?
312_38_What emotional factors influence trust in machine teammates?
312_39_How does the weight of the decision (life or death, monetary value, job on the line) affect humans' willingness to trust a machine?
Group dynamics positive/negative 181_8_How does AI influence group dynamics? 167_12_What group dynamics will be important to be learnt and read by machines to foster improved team performance? 347_11_How would that influence team cohesion and emotional contagion 231_37_Does existence of a teammate in a group of humans affect inter-human conflict?
207_20_What the defend mechanism to prevent the conflict between human and the machines? 178_30_How can we facilitate conflict management with machines as teammates?
335_5_How can disagreements between humans and machines be discussed and solved?
Health enabler/safety risk 189_15_Study the human-machine relation in fitness and wellbeing.
220_38_Robots shall never harm humans 220_39_Robots need a lot of sensors to detect potential accidents early and prevent the worst 274_13_They are stronger, but must use their strength to protect humans and not to harm them 319_29_How to safeguard for the well-being of teammates machines?
262_11_How can we insure the safety of humans in collaborative industrial teams with robots?
272_10_There should be a board or consortium to ensure safety and secure usage of AI and research should also point in that need of it 220_40_Robots need a soft shell to protect humans from accidents with robots 269_25_Are they a threat for workers? 220_41_How can we prove that we have a water-proof network of laws to keep robots from becoming superhumans?
220_42_Are robots allowed to walk in public unrestrictedly such as humans?
168_9_What kinds of regulations must exist to ensure that machines won't harm humans?
233_15_When machines become strong, how to address the social security issues? 220_43_How can we minimize machines unnerving humans due to following the rules 100%.
167_13_How will machines as teammates impact the psychological health of human co-workers?
175_20_What ethical guidelines should be developed to sanction the use of machines as teammates? (Like Asimov's robot laws) 272_11_How safe are the AI teammates decisions?
Jobs created/lost 175_21_What are the consequences for the work force if AI starts taking over certain tasks? 231_38_Will companies use machines as an excuse to fire people? 234_4_Will machines make human labor obsolete? If so, what do we do with the now "superfluous" workforce?
231_39_What will people do to support themselves who are displaced from the job market?
221_14_If machines substitute a substantial part of our workforce, what should people do instead? 220_44_What happens to people losing jobs because machines are doing it cheaper? 302_6_The fact that many will lose their jobs over machines which do not require sleep, work-life balance, or a salary. Society must decide how to accommodate those humans who will inevitably be left out. Will there be universal income so that we can all benefit from machine workers? Will there be re-training/re-education programs for those left out, so that they can secure a job in other fields? Will there be generous severance packages that allow a person to regroup? 261_7_loss of jobs 312_40_How can we deal with the reduction in low-skill jobs that will result from increasingly capable machines? 267_13_How do we as a society deal with the prospect of higher unemployment at a global level if unskilled workers are no longer necessary?
233_16_If machines become team member, how to address the unemployment issues?
262_12_How will a massive replacement of laborers with machines affect our society? 331_16_What to do with the unemployed 236_11_How to rethink social assistance in a jobless economy? 289_16_How does machines as teammates affect human employment? 319_30_How do machines as teammates change our understanding of labor markets?
274_14_Will there be enough work for all humans? Will humans feel useless and unneeded? What can humans still contribute?
272_12_How are the various job markets would get affected? 354_7_See the discussion about universal wages as a result of job loss.
236_12_Are we entering in a jobless society? 231_40_Taking people's jobs. 267_14_Older and less-skilled workers may be replaced more frequently than younger/more skilled workers (e.g., creative creators). Is this fair to the worker? 319_31_Do machines as teammates replace jobs or repurpose them? 272_13_Though AI can potentially remove repetitive jobs, but still there should be a human supervision over AI to prevent unexpected outcomes that defies logic and rational thinking 266_13_What will be the impact of our new teammates on the job market?
235_16_How will the organization of paid work need to adapt to the shift in tasks, e.g., in terms of work hours or job profiles and qualification? 267_15_Similar to the ethical RQ above, the workforce is going to change a lot. It already is. But it will create a need for those who do specialized jobs that do not have a lot of monotony (those easily replaced by 189_16_Labour costs have a lower bound due to the Malthusian constraint of workers supporting their families. Minimum salary laws recognize this constraint.
274_15_What kind of research field will vanish? Maybe they can collect data themselves and do data analytics projects and publish them on MISQ, ISR… then a variety of research fields might not be needed to be done by humans anymore! 312_1_How can we deal with the reduction in low-skill jobs that will result from increasingly capable machines? Eva Bittner is Assistant Professor and Head of the Research Group on Information Systems, especially socio-technical systems design at the University of Hamburg. She holds a doctorate degree from the University of Kassel. Her research interests include collaboration engineering and human-machine-collaboration, knowledge work and management, innovation management and co-creation/social innovation. In particular, she focuses on designing collaborative processes and practices for knowledge-intensive activities using the power of modern ICT. She is the author of numerous national and international publications, which have appeared e.g. in the Journal of Management Information Systems and the Business and Information Systems Engineering Journal.
Dr. Briggs studies the cognitive foundations of collaboration and applies his findings to the design and deployment of collaboration systems. He is Co-Founder of the field of Collaboration Engineering and co-inventor of the ThinkLets design pattern language for collaborative work practices. He has made theoretical contributions on group productivity, ideation, creativity, consensus, change, technology transition, and satisfaction. He is currently working to reduce military decision cycles with collaboration systems and to reduce the dropout rate among K12 learners-at-risk with collaborative learning techniques, with a focus on technical and cognitive challenges for transferring engineered work practices to non-experts without training on either the techniques or technologies.
Triparna de Vreede is a faculty member in the Muma College of Business where she teaches multidisciplinary graduate and undergraduate courses. She is also the Academic Director of the Masters in Management program. She is an interdisciplinary researcher who specializes in innovative research and pursues grants from federal and corporate sources. Her expertise lies in advanced collaboration, psychological factors influencing AI-human interaction, crowdsourcing, and engagement. Her research, practice, and teaching involve business topics such as collaboration, creativity, international culture, people analytics, and information systems.