The distant Horizon: Investigating the relationship between social sciences academic research and game development

Research in the social sciences devotes a great amount of attention to investigating the impact of video games on the individual and on society. However, results generated by this research often fail to inform game development. The present study investigated the outreach of research conducted by the academic community by interviewing 30 game developers and 14 researchers, highlighting critical aspects in the relationship between game research and game industry. Specifically, we found that the difference in priorities, speed cycles, and dissemination practices between these two contexts hinder communication. Subsequently, we carried out a focus group for a set of developers and researchers (N = 6) with the aim of eliciting recommendation for improving communication between academics and developers. Among the recommendations to emerge were calls to di-versify dissemination channels, promote joint conferences and develop research-production partnerships. It was felt such measures could strengthen the influence of research results outside the academic community.


Introduction
Video games are becoming more and more widespread, and their presence in our daily life is growing.According to the Q4 2017 GameTrack survey, 1 periodically run in four European Countries, the percentage of 6-64 year-olds who play games ranges from 44% (Spain) to 65% (France).The survey also reports a growing spread of gaming among 35-44 year-olds (from 36% in 2010 to 46% in 2016) and among 45-64 year-olds (from 21% to 27%).Similarly, the Pew Research Center's American Trends Panel 2 reports that about 43% of adult Americans play video games "sometimes" or "often".The phenomenon, therefore, interests a large proportion of the Western population alone, with a potentially tremendous impact on its culture.
In light of this pervasiveness, it's no wonder, that social sciences research into video games and gaming has devoted considerable attention to investigating their influence on both the individual and society at large.Major research threads being explored include the possible psychological effects of games [1], their potential for education, training and behavioural change [2,3], and their cultural and ethical implications (e.g.[4]).Given the high level of interest, the EU has funded numerous games-related research initiatives over recent years, from highly-focused domain-specific projects to broad-scale networks of excellence devoted to (serious) games per se.The overarching objective of these efforts has been to explore effective ways of channelling the proven motivating power of games to trigger 'purposeful' benefits, with particular emphasis on learning and behaviour change [5].
In 2016, the European Commission also funded a project, called Gaming Horizons (www.gaminghorizons.eu), to investigate video games and society from a more social sciences perspective, and to propose alternative framings for the role games might assume in society.The project considered digital games and digital gaming in a broad sense, covering not just entertainment-oriented games of various kinds and genres (including those produced with artistic aspirations), but also so-called serious games (those designed and/or adopted for purposeful ends) and gamification (i.e. the use of game design elements in a non-gaming contexts; [6]).In its investigations, Gaming Horizons explicitly pursued a participatory approach directly involving a broad range of stakeholder groups like game developers, policy makers, researchers, educators, players, and parents of players.It was also mandated to implement Responsible Research 3 principles.This paper describes some of the core results and outcomes the project generated.

Setting the scene
The present study examines the relationship between social sciences academic research, digital gaming and game development.In order to explain why we decided to focus on this topic, we will begin by illustrating how, in the initial phases of the Gaming Horizons project, the considerable gap between these two sectors became apparent, the extent of which we did not anticipate before project investigations got underway [7].
The issue became immediately evident when analysing the results that emerged from the very first major project undertaking, namely an umbrella review of social sciences research on games and gamification.One of the key aims of this literature review was to identify whether academic authors, when discussing their research results, (a) make specific recommendations about what steps could or should be taken to address any (problematic) issues they identified, and (b) what kind of recommendations they make and to whom.We found that where they did so, their recommendations were disproportionately directed towards (other) academic researchers, with few directed towards other stakeholder groups such as developers or policy makers.Specifically, out of 81 recommendations identified in the papers we selected for the review, 38 (46%) were directed to researchers, and only 11 (12%) to game developers, despite the fact that the studies dealt (in more or less detail) with issues related to the concerns of those who design and produce digital games, whether of the purposeful or entertainmentoriented variety.This was especially true for games-related articles derived from the psychological literature, in which author recommendations are mostly suggestions for future research avenues; here, recommendations directed to stakeholders other than researchers are almost totally absent.
The landscape was slightly different when considering published research exploring the educational perspective of games and gaming.In this case, authors directed recommendations to developers, policy makers, and educators.These mostly regarded the need to develop more effective games for educational use (e.g.[8]) and to better align existing ones with learning goals and students' needs [9,10].Additionally, some papers focusing on the question of inclusion provided some specific recommendations for developers (e.g., [11]).
Nevertheless, the overall picture gained from the review of research literature, at least that published in peer-reviewed journal articles, is that the games-related academic discourse is mostly carried out internally, and possibly self-referential; those producing this (substantial) body of knowledge seem fairly unconcerned about reaching practitioners outside the academic community.This apparent lack of impact is further compounded by the fact that even those research questions attracting considerable academic attention, such as whether or not frequent gaming generates cognitive benefits, are still largely unresolved and necessitate more evidence [12].Conclusions drawn from existing studies are usually rather tentative, offering very few definitive answers to research question.As a consequence, few guidelines and recommendations are issued to external practitioners.
At the same time, it should be acknowledged that caution is an important principle of scientific endeavour, and one not to be totally discouraged.That said, it is understandable how it could be off-putting when viewed through the lenses of game developers, who would find definite research results particularly informative when making practical decisions, especially when they are seeking hard scientific evidence to back (sometimes critical and/or contentious) game design decisions.Indeed, the lack of design and/or policy-relevant recommendations combined with tentativeness in drawing conclusions often leads to considerable difficulty in identifying research findings with practical applicability.
The risk of social sciences research on games becoming isolated from game industry [13] is troubling, since it would limit the beneficial impact that (potentially significant) scientific findings might have on those outside academic circles, and especially on those very individuals who devise and develop such important and pervasive cultural artifacts as digital games [14].Such a gap could widen into a chasm if researchers fail to get a firmer grasp of the dynamics that drive game development in its various guises, especially regarding the constraints and opportunities games offer [15,16] and the issues that could arise from specific development strategies (e.g.[17]).
This paper reports on findings from the Gaming Horizons project that regard the relationship between social sciences academic research and game development; it discusses the most critical aspects of this relationship, the factors that seem to inhibit beneficial cross-communication flows, and how relevant stakeholders believe such issues could be tackled to strengthen mutual ties.

Method
The full results of the Gaming Horizons project are openly available in the main project deliverables [14,18,19].
In this contribution we will focus exclusively on two project activities that directly involved both researchers and game developers.The first was a series of interviews in which critical aspects of the relationship between social sciences game research 4 and game development were identified and explored.The second regarded a focus group specifically devoted to producing recommendations to improve communication and collaboration between these two sectors.All the informants involved in the study signed written consensus to the use of anonymised data for research purposes.The ethical aspects of the project were evaluated and approved by the European Commission.

Interview method
The Gaming Hrizons interviews focused on exploring the opinions and experiences of several stakeholders involved with videogames and gamification [14].In total, we carried out 73 one-on-one interviews, involving educators with experience with Game-Based Learning (N = 12), policy makers (N = 4), players (N = 13), social sciences game researchers (N = 14), and game developers (N = 30).In the present paper, we will only focus on the latter two groups.
Participants were recruited using purposeful sampling [20], taking care to include participants who were knowledgeable about the topic while also seeking broad representation in terms of gender, ethnicity, age, range of experience with games, and stakeholder role.Twenty of the 30 developers we interviewed were recruited during two non-academic professional conferences: the 2017 Game Developers' Conference 5 in San Francisco, and Game Happens 2017 2 in Genoa, Italy.The remaining stakeholders were mostly recruited by drawing upon the professional networks of project consortium members.Of the 30 developers interviewed, 12 are women and 18 men.All developer interviewees were from Europe or North America.Seven of the 30 interviewees were actively teaching game development at the time of the interviews.Of the 14 researchers interviewed, 6 are women and 8 men.All researchers interviewed lived in Europe, aside from an interviewee from Australia.All interviews were conducted in English.
The interviews were semi-structured and covered a wide range of topics related to video games and gamification, from the "purposeful" harnessing of digital games to their alleged link to aggressive behaviour.Both researchers and developers were explicitly asked about their opinion on the relationship between social sciences game research and game development, including the game industry.Game developers were also asked whether their development practice was influenced to any degree by academic social sciences research, and if they perceived themselves as being informed about the main findings reported in the social sciences academic literature.
While the interviews had no pre-determined length, in practice most lasted from 45′ to 90′.For analysis purposes, the project adopted the Framework Method, an approachparticularly suited to applied social and policy research [21,22] which can be considered an a type of thematic analysis [23].The method was applied using a mostly deductive approach, by devising a codebook with about 90 codes identified from the Gaming Horizons literature review and other activities previously performed in the Gaming Horizons project.For example, research lines identified during the literature review were included as separate codes.The codebook was validated by six coders, who iteratively coded two 'trial' interviews until adequate coder agreement was achieved.Following this, the codebook was used for the full set of interviews, with few amendments for adding new codes that were identified inductively.The interviews were transcribed and coded in nVivo v.11 [24]; the anonymized transcriptions are available as an Open Dataset [25].Henceforth, all interview excerpts will be referenced using the filename used in the dataset repository, which includes the stakeholder type and a progressive number (e.g."Developer 11").

Focus group method
One of the last phases of Gaming Horizons entailed a series of indepth focus groups.These were devoted to specific topics that had emerged during previous project phases as being contentious and/or of particular significance to the Gaming Horizons mandate.As discussed in the Results section, one such issue was the relationship between social sciences research and game development.Therefore, we carried out a focus group specifically devoted to investigating this relationship and to exploring avenues for reducing the apparent disconnect between academia and the game industry.The focus group involved social sciences game researchers (N = 2), game developers (N = 2), and stakeholders belonging to both categories (N = 2).Participants were recruited via direct contact and calls on social media and institutional websites.At the end of the focus group, participants were asked to negotiate and formulate explicit recommendations to improve the research/development relationship.For more on the Gaming Horizons focus groups, see [26].
The whole discussion, which lasted 78 min, was audio-recorded and is available upon request.Post-transcription analysis was carried out using the Framework Method inductively, looking for both implicit and explicit recommendations issued by participants.

Developers' attitude towards social sciences research
The picture painted by developers in the interviews was very varied: seven out of thirty were intimately familiar with academic research, and worked as academics for part of their career, as researchers or professors.
I have an academic background, that's for certain, I wouldn't really still call myself an academic, because I really didn't like inhabiting the world of academia.I found it really restrictive.Developer 9, lines 400-402 I ended up mostly focusing on design but from a somewhat academic angle.Developer 23, lines 184-185 Another participant did not consider himself as part of academia, but reported being greatly influenced by social sciences research: As someone who joined the game industry at a time when there really wasn't a games academy and who has keenly followed the release of each new textbook or monograph on video games and other kinds of non-digital games and their design, and as someone who has eagerly read many, many academic papers studying game development, I can't really begin to summarize the impact that this work has had on my game design practice.Developer 17, lines 293-298 However, the rest of the developers we interviewed reported feeling somewhat distant from academic research, both as a career choice and as a source of knowledge: To be honest with you, I just haven't seen that many great studies about computer games over the past few years.Developer 2, lines 318-319 I certainly think that there's huge scepticism and huge mistrust between games and any form of institution, including the arts, including the public funded sector and including academia.Developer 9, lines 418-420 I have this bubble of how I got into the games industry and how I taught myself about making games.And then over here is academic games research.I've read a few books here and there from smart games academic folks, but it's always just been on this periphery.Developer 25, lines 195-198

Barriers: language
When prompted to explain why they felt social sciences academic research was distant from their game development practice (or even seen as hostile), participants mentioned several factors.One of those, reported by two developers, is the issue of language accessibility, as researchers' jargon can be difficult to parse for non-academics: It took me a good half a day, half a day I could've spent doing anything else, going through this academic journals trying to figure out what the heck they were saying.I think that's also, part of a large reason you don't see more academic papers or academia talked about […] how it's assisted the game development, because a lot of academia is inaccessible to those who are outside of it.Developer 6, lines 376-381 I think the problem comes when you don't realize that that specialist knowledge and language in some situations is alienating.Developer 9, lines 106-107

Barriers: low market applicability
At a more fundamental level, two participants also reported that the kind of questions investigated by social sciences research are rarely useful from a commercial standpoint -which is often a foremost priority in game development.
Generally, at least towards the start of a project, there's a big research phase […] But it's never been -historically never been that much directly useful, academic literature relating to games.Developer 3, lines 264-268 Pitching the very first lesbian route [in a branching narrative] of our company's history, I needed the research to prove people would buy this, people would support it, we would see an increase in sales, […] but there wasn't any research that I could find for games that showed this.Developer 30, lines 275-278 On the other hand, an interviewee familiar with social sciences video game research mentioned a specific example of a study which had market implications and that did exert an influence on the game industry: Hegemony of Play [27], as a paper, proposed that there are large market segments going unserved by current game design […] this has led to a reciprocal impact on the market with the creation of games like the 2012 Game of the Year, Journey.Developer 17, lines 313-318

Barriers: different dissemination channels
In general, the interviewed developers did report being interested in social sciences games research and innovation (see Section 3.1.7),but the sources they turned to for keeping up to date with these developments tended to be non-academic, industry-oriented websites.
I will frequently go to Gamasutra to see if there's anything posted there, or the GDC Vaults Developer 30, lines 202-203 Unless it's articles that come up on places like Gamasutra, where I do read keenly, I must admit I don't read academic papers on games.Developer 29, lines 268-269.
Most developers also expressed strong enthusiasm for attending game developers' conferences, which were characterized as essential for exchanging practical research information.
There's a difference between sharing your talent and sharing your time and your talent, and I think that's what GDC is about, like people are sharing their time and their talent together and that's very powerful.Developer 8, lines 266-268 I guess sort of like learning from our peers is the most useful thing, in the same way we all learn from each other in the studio.That's what makes the biggest difference to us in terms of where influences come from, aside from the games we play and love.Developer 29, lines 263-266 Even those developers (8) who did report keeping up to date with academic research mostly accessed it through conference archives: I use other resources, like academia.orgor the archives of conferences, like DiGRA and FDG, in order to seek out subject matter.Developer 17, This positive opinion of conferences was to be expected, since most of the developer interviewees were recruited during non-academic conferences.However, there was also a critical voice among interviewees: The standard format for conferences, where one person stands at one end of a big room and just yells a bunch of information out, is not the most conducive format to learning.And I think that's something that academia and conferences in general aren't quite catching onto although they're slowly getting there.You know, I think they're important.I think developers do love sharing information.It's just this question of whether we need to all be sitting in a giant room being very uncomfortable and sweaty, listening to someone talk about something; or whether there's better ways that we can be sharing information.Developer 2, lines 344-352.

Barrier: different speed cycles
An interesting point on the general preference for trade conferences over academic papers was offered by an interviewee when commenting on the pace of change of game industry: I've been channelling [my academic interests] into events, both by speaking and curating talks and games.It's definitely a lot in there and has more freedom because you don't have to take 100 years of academic research baggage with you all the time.It's there somewhere and people will call bullshit if you don't source and reference things on your talk and your statements, but I feel like we can move faster at conferences than you can with writing, writing and reading papers and books.Developer 23, lines 242-248 Indeed, the slow pace of research was seen by two developers as clashing with the needs of industry, and this chould be considered a third factor (along with language accessibility and low market applicability of social sciences research) for the relatively low communication between academics and game developers our investigation highlighted: There is generally not a great history between academia and games in my opinion.I think there are numerous reasons for that, one of them being that academia in general moves quite slowly, changes very slowly.Games on the other hand change very quickly.Developer 2, lines 327-330.I think I submitted a paper once, but as I started making more games, I did get sort of less -both time and interest in actually writing papers because it took me a lot of time and I didn't get as much enjoyment from it as actually making games.Developer 23, lines 193-197

Positive aspects: accessibility of research output
The general decoupling of social sciences research from market dynamics, on which interviewees generally agreed upon (see Section 3.1.3),was however sometimes perceived as having positive aspects.For example, academia's open policy of results sharing was favourably compared to the more closed industrial research.
In my field, games user research, there is some publishing, but I think partly just because of the nature of academia versus industry, it's mostly people in academia doing the publishing rather than people working in industry.Developer 28, lines 271-274 I'm glad that [academic research] exists and I'm glad that we have a society in which people can pursue that goal with no specific kind of commercial value.Developer 29, lines 280-284 When I'm trying to prove a point, when I'm trying to say, hey, we should definitely include this, it's great that there are places that have started more and more to have that research available.Developer 30, lines 198-200

Positive aspects: topics of interest
Among the 8 participants that reported having been influenced by social sciences research in their development practice, user psychology studies seem to have exerted the most influence, being directly referenced by three interviewees: I'm seeing a lot more developers look to academic research for answers to some of these questions because they are questions about psychology.Developer 2, lines 301-303 Bartle types [28]  On the other hand, one of the interviewees who expressed this wish did so with a significant degree of ambivalence: As a game designer, I sometimes don't want to know -I -I don't want to have all these research papers and be like, 'Oh, no, but if I do this one thing then maybe that makes players, I don't know, feelfeel bad about themselves two hours later.' […] I definitely think there should be more research into it.But it's a very creative area and […] you often don't design it with all of these research papers in mind […] I'm afraid it would be too restricted if you've got these sort of these are the things that work, that we know work and these are the things that we know that don't work, and then you have to sort of adjust what you want to make to that.Developer 15, lines 236-262

Researchers' point of view: barriers
Some of the findings that emerged from the developer interviews were also confirmed by the interviews with researchers.Specifically, two researchers reported feeling that sectorial language is impeding communication with the game industry, and even between different academic fields.This is quite complex, actually, to even have a shared vocabulary.I can see it in my department.So  Researchers also agreed that as social sciences academic research is not directly motivated by market concerns, this makes it less appealing for industry partners.Interestingly, though, one researcher pointed out that the lack of economic drive in the field of serious games actually held serious games back from a technological standpoint, since the lack of investment in educational games on the part of industry widened the gap in quality between entertainment and serious games.
There wasn't the ability to invest in [serious] games, so they didn't progress as fast as the mainstreams games industry and the two industries really separated and you had, the mainstream video games didn't contain learning content that were doing amazing things and then this edutainment industry that was, felt very backwards.Researcher 7, Regarding the different speed cycles of industry and academia, no researcher mentioned the notion.However, one interviewee lamented that social sciences academic research had less influence on the game industry than might be the case, and saw the cause of this in the academic community itself -both because it shirks responsibility to exert this influence, and because it is partly out of touch with the less welltread areas of the games landscape: I think that us academics, we very easily complain about how bad video game developers sell their products and yet we don't do enough in actually fostering a different type of video game culture.So I think on our side and our responsibility, we should do a bigger effort to try to write scholarship, to try to teach more those things that are in the periphery and to challenge the default video games.Researcher 12, lines 140-144 I think one of the things that we do in academia is define the canon, whether we like it or not; what we teach becomes canonical, at least for the students we have in front of us, right?So I would say that we would need to embrace the fact that we are canon-makers […] and then establish a canon in universities where we are a little bit aware that the video game medium is broader than whatever is published by -whatever is published on consoles.Researcher 12, lines 235-241 Relatedly, another researcher asserted that social sciences academic research tends to consider games as a whole, without taking into account the differences between games, genres, and playing contexts.This can contribute to the feeling that social sciences research is out of touch with the reality of games as media.
I think one of the problems with games as a researcher, is that we have this silly word 'game' and it just sort of encompasses everything […] if you have to actually make parallels between Uncharted 3 and Candy Crush, it is a completely different kind of experience we call those games, but actually the interactions are completely different and the experience is completely different, and where you might play that is completely different.Researcher 7, lines 148-155 Lastly, and most interestingly, when explicitly asked about the impact of games research outside the boundaries of the academic community, researchers of educational video games solely talked about reaching teachers and schools.While it's true that, after academia, they saw their chief dissemination targets as being practitioners within the educational sphere rather than professionals in different areas of gaming or related media fields.
Most research, it's not arriving at the end-user, at the teacher, and the teachers are not often engaged in looking at what are the results of research before deciding to engage in this kind of novelties.Researcher 8, lines 340-342 In terms of the research, who have I been talking and disseminating to?I've been going to like the Geography Teachers' Conference and the Geography Teacher Education Conference and sharing research there.Researcher 10, lines 449-452.

Barriers: language
All participants in the focus group confirmed the existence of communication difficulties between social sciences researchers and game developers.
The game sector is one in which the gap […] is I believe even more wide, compared to more common applications.[…] But I believe a bigger effort could be exerted.

43′54″-44′21″
Their own experiences highlight how misunderstandings, and even outright clashes, can be common when these professionals work together on a project.
[In a project I worked on] there was, indeed, a great distance between us developers and the theorists […] it's been difficult to interface with each other, there were a lot of fights, a lot of misunderstandings.

18′34″-19′02″
On the other hand, a developer in the focus group who had experience of joint collaborations with researchers clearly valued the scientific output produced, expressing surprise at the quality and level of detail.This is the interesting thing.We developed the prototype, a complex game […] and it was given to a research institution, like you, that did a field test analysis for two months and gave back to us developers an extremely detailed hundred-pages pdf report, with analyses, […] in this second phase, interfacing with the researchers was very… we developers, said, 'great', congratulations for the work they did, and the quality of the report they gave us back, too, was very precise and specific.

19′13″-20′13″
When discussing the distance between research and development, however, a participant reported that even inside the game development sphere there can sometimes be relatively little communication between different compartments.This observation is similar to one expressed in the researchers' interviews that reported how research presents the very same problem.
If I had to summarize my experience, or even what I see around, I see there are many separate compartments, even inside development, not to say in relation with pure researchers.I often read papers and said 'yeah, well, who knows how you could implement that'.45′45″-46′07″

Barriers: low market applicability
Experiences of joint collaborations between research and game industry reported by participants were exclusively related to serious games, and had either started out as pure research projects, or, alternatively, as European/regional projects explicitly conceived to develop a serious game through cross-collaborations between academic and industrial entities.
It's a research project, so it was started by researcher, especially me, so it was interesting for us to study research topics related to computer graphics and interaction.Then [name], won the research fellowship grant and he is a game enthusiast, and he put in his own input [as designer].32′37″-33′06″ However, one researcher participant reported that, in one such collaboration, the research results were not implemented in the final product, as they were more 'theoretical' in nature and their application would have led to an unsustainable product in terms of performance.
What I indeed noticed is the gap between research and development, because when talking with this company, they said 'yeah, okay, you got nice results, but we'll probably apply those in ten, twenty years.[…] because indeed they are interesting results, but quickly finding a practical application for them is sometimes impossible.36′01″-36′34″ We [researchers] were more interested in developing and examining that aspect, so research about the individual, the semantic object inside the game, the automatic interaction inside the game.And it's an aspect that the ones who have to develop a game, and to create something efficient, both in terms of interaction -our interactions were rather slow, because behind them was a semantic engine for interpreting them […] for a real game […] it's not really sustainable.

37′55″-38′47″
This problem was mirrored by the experience of another participant who, as a developer, was not able to meet the needs of social sciences researchers.
There's a lot of distance with social sciences.Putting in [a game] an effective educational mechanic, or psychological dynamics […] it was difficult for me [as a developer] to understand how to offer a technological contribution that would meet the needs coming from [educational researchers].

41′47″-42′20″
Conversely, one aspect that contradicted the interview results is access to research output.Developers interviewed in the previous phases reported relative ease of access to social sciences papers (see Section 3.1.6)and difficulty in decoding them and understanding their jargon.Developers participating in the focus group, instead, reported problems accessing non-open access papers.
I would like to have access as… there's a lot of literature, like games journals […] But the majority of it […] you find a twenty-page paper for eighty euros.So in theory I should pay to access it.23′22″-23′42″ Interestingly, when presented with this assertion, the researcher participants explained how they themselves get around barriers to access, thereby sharing 'inside knowledge' in what they, too, perceived to be a common problem in academic literature.
Well, the alternative is writing to the researcher, usually they don't refuse.24′06″-24′10″ There's also the Russian website […] completely illegal, put up by a Russian researcher, as the story goes.When I can't access a paper because my institution won't pay for it […] on this website, I can access it.24′10″-24′38″

Recommendations
As to recommendations issued at the end of the focus group, the main one focused on including all relevant roles for game development in cross-disciplinary teams, including the role of a "game director" with the explicit role of facilitating interaction between team members and keeping an eye on the 'big picture'.
In most productions, even of serious games, you lack what in movies would be the director.So in a sense the game designer is writing the screenplay, and the production director takes care of the technical aspects, and maybe you find the one who did research, but in the end the one that says whether the game is well developed, in terms of inclusion, of UI, of narrative, is the director.[…] we may have people who are available, great technicians, excellent programmers, who have difficulty speaking with the researchers.Because at the upper levels there is no director who combines these parts.And there are so few of those, let's say, directors in the game sector.46′10″-47′48″ The many roles in game development have to work together, because when one competence is missing we understand, now, that the game would be mutilated.It would miss a part.61′49″-61′58″ Additionally, participants advocated higher budgets for serious games (in their own words: 'serious games, serious budget'), so as to better collect and integrate all competences in game development, especially when this is not exclusively geared towards economic return.
Lastly, they suggested being keenly aware, at all times, of the existence of gaps, distances and misunderstandings, not only between research and development, but also between different development sectors and different academic fields.
Even for a researcher in computer graphics, gaining an understanding of the pedagogical or psychological aspects of the game is not easy to do.And even us, we are not able to then translate suchlike into game mechanics.37′00″-37′15″

Discussion
The interviews and focus group described in the previous sections yielded valuable, but somewhat different, insights regarding the relationship between social sciences academic research and game development.
First of all, our researcher interviewees, when talking about nonacademic dissemination, mostly focused on reaching teachers and educators rather than developers, confirming that developers are often not considered the main targets of research results.
The interviews with developers, meanwhile, demonstrated that developers found it difficult to identify the practical applications of much social sciences research, especially from a market perspective; however, they did express interest in studies on user psychology and game narratives.
The developers also identified some factors that discouraged them from engaging with social sciences publications, the main one being sectorial-specific language: academic research papers were characterized by interviewees as being written in obscure, field-specific jargon.Researcher interviewees confirmed this notion, noting how it also hinders communication between different academic fields.
On the other hand, developers lauded the fact that academic research results and data are often made publicly available, compared to the much more closed nature of industrial research (which is often subject to commercial non-disclosure agreements).Curiously, though, one focus group participant stated the opposite, reporting difficulty in accessing research due to pricy journal subscription fees.As a result, it's not clear from our research whether academic research could be considered easily accessible from an economic standpoint, althoughclearly -the recent growth of open access scientific publication has the potential to greatly increase the impact of research outside the scientific community [30].
An additional factor cited by two developer interviewees as contributing to a disconnect between game research and game development is the different pace characterising these contexts.Game development tends to have a fast cycle, in which innovations are rapidly introduced and exploited before they lose their novelty value on the market.Research, instead, usually privileges the steady accumulation of knowledge and the careful drawing of conclusions.As a result, game research risks being out of touch with the current reality of the game market, investigating games, game types or game mechanics that have since fallen out of favour and thus become somewhat outdated.Additionally, the time lapse between a manuscript's completion and its ultimate publication can be considerable, probably exacerbating the untimeliness of research findings [31].Possibly for this reason, developers clearly expressed a preference for gaining fresh input by attending conferences rather than by reading journal publications.Our interviewees not only highlighted how conferences are useful for building connections and engaging with different stakeholders, but they also noted how these events tend to be more dynamic and on the cutting edge of current innovation.The situation is somewhat different for academics, as presenting their work on the fast-cycle conference circuit (rather than in academic journals) can induce them to release fairly rushed and sometimes half-baked research findings that, when published in proceedings, often yield generally lower citation rates [32].
Lastly, focus group participants related their own experiences mixing game research and serious games development, noting how difficult collaboration can be between different professional roles and advocating for putting someone expressly in charge of easing communication and evaluating the 'big picture'.It should be noted, however, that the relatively low number of participants involved in the focus group, compared to the number involved in the interviews, should be considered a limitation of the study and should be taken into account when considering, specifically, conclusions drawn from the focus group.

Conclusive recommendations
Our investigations pointed to a lack of concrete recommendations emerging from social science academic papers as a potentially critical aspect undermining the potential impact of much games research, especially in the game development sector.Accordingly, in this section we will strive to draw -from the wealth of research materials detailed above -concrete suggestions that may prove beneficial for improving the relationship between social sciences academic research and game development.At the same time, we are well aware that, due to the different priorities of these two sectors, there will always remain a certain intrinsic distance between them that can only be bridged in part.Recommendations for addressing other critical issues that emerged during the [PROJECT NAME] project are available in the project Manifesto [33].
Our first recommendation would be to capitalize on developers' preference for (dynamic, interactive) conferences and promote events that combine input and exchange from both researchers and developers.Such blended conferences could succeed in disseminating research results in a more timely fashion.They could also facilitate communication between usually separate communities both because of the general social environment, which tends to be friendly and relaxed, and because the presentation format may be less alienating for developers, in terms of language, compared to journal publications.Additionally, there are already a number of developers' conferences (some of which, like GDC, are highly popular), so the conference format is already familiar to game developers.However, to ensure that research presented in these events is high quality, these conferences should also be perceived by researchers as desirable outlets.To ensure this, joint conferences should strive to publish peer-reviewed proceedings indexed on the most widely used citation databases or offer pathways for publication of contributions in reputable journals.
Our second recommendation is to encourage further non-academic dissemination of social science research results through websites, blogs and social networks that are popular with the game development community (i.e.Gamasutra6 ).To properly incentivize non-academic dissemination, though, this effort should be recognized as valuable and valid in terms of a researcher's productivity and career.The present trend towards the use of quantitative indicators for the assessment of research productivity is certainly not helping progress in this direction, and a deeper reform of the research evaluation system would benefit both non-academic dissemination and participation in joint conferences.
Our third recommendation is for greater support of and participation in the Open Science movement, which would help to make research results increasingly accessible both inside and outside the academic community.
Our fourth recommendation, geared towards the game industry generally (both indie and commercial/AAA) as well as game development project coordinators, is to explicitly put someone in the role of facilitating collaboration between different professionals and fragmented communities, in order to help in recognising and taking on board pertinent research input.
Lastly, our fifth recommendation would be to increase support and funding opportunities for games projects that involve both developers and researchers, without dedicating this steam exclusively to the development of serious or 'purposeful' games (as has largely been the case until now).Better communication and closer interaction between the social science research and game development worlds would undoubtedly give a boost towards the production of towards more creative, ethically sensitive and culturally valuable products.As our focus group highlighted, when such project support targets both stakeholder groups, as is currently the case for much serious games development, collaboration actually does occur despite the communication difficulties faced.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
[29]bsolutely influence how I think about how the products and games that I work on will be experienced by different kinds of players.Developer 8, lines 170-172 I think that the paper Mechanics, Dynamics, Aesthetics by Robin Hunicke, Marc LeBlanc and Robert Zubek[29]had a major influence on me in giving me a model for game design […] The MDA model opens us up to consider […] emotional, intellectual and otherwise phenomenological impacts that a game could have on a person, thereby expanding the design possibility space for games.