Toward Next Generation Mixed Reality Games: AResearch Through Design Approach

Mixed reality (MR) games refer to games that integrate physical entities with digitally mediated contents. Currently, it entails relatively high development eﬀorts and technical threshold to incorporate multiple virtual and physical building blocks. Moreover, the absence of a coherent technology stack renders the design and development process even more challenging than conventional digital games. This research proposed an MR game technology stack that aﬀords a wide spectrum of virtual and physical modules with ﬂexibility to adapt to speciﬁc application domains and on-cite needs. We adopted a Research-through-Design (RtD) approach. We ﬁrst surveyed 24 state-of-the-art studies, and results were synergized into three diﬀerent spectra of technological aﬀordances, respectively activity range, user interface and feedback control. Based on the initial insights, we then went through two iterations and resulted in an MR game technology stack. A user co-design workshop was further conducted, where 15 participants with diﬀerent game making experiences were invited to try the proposed technology stack, and 4 conceptual game designs were generated after hands-on tutorial and group co-ideation sessions. We also collected participant feedback via user survey and semi-structured interview. The proposed technology stack received an overall positive feedback, particularly the innovative in-game interactions and the motivation for game creation it brought to the users.


Introduction
Mixed reality (MR) games integrate physical entities, e.g. the physical environment, real objects etc., with digitally mediated contents into immersive entertainment experience.According to the Reality-Virtuality Continuum proposed by Milgram and Kishino (1994), MR games encompass a wide spectrum of hybrid systems and applications, varying from pervasive augmented reality (AR) games, e.g.Pokemon Go, to fully-immersive virtual reality (VR) games with tangible objects involved, examples like the work by Cheok et al (2002) and Harley et al (2017).Sometimes, it also refers to Extended Reality (XR) games as a collective term for AR, VR and MR games.With its core value of bridging and blending the cyber and the physical world, MR game is considered to be the essence of future metaverse by some tech tycoons including Microsoft; and the key to approaching the metaverse is to "democratize the game building", according to Waters (2022).In the recent decade, the application of MR technology in serious games and gamification has been widely witnessed in both public and private sectors, including education (Zikas et al (2016)), culture heritage (Ioannides et al (2017)) and healthcare (Abdelkader et al (2011)).
Currently, it has to traverse multiple technology stacks, such as computer vision, projection mapping, indoor/outdoor positioning, Internet of Things (IoT)...to just name a few, for game developers to create a single MR game.Despite the high technical threshold and development cost, commercial game engines and proprietary solutions provide rather limited and rudimentary supports, and MR game creators oftentimes find themselves drowning in too many lower-layer details, making the design and development process far more difficult when compared with conventional digital games, according to Medford et al (2018).
Existing MR game research are found rely on mostly domain/use-specific solutions, thus preventing them from being generalized to a wider, general-purpose context, as shown in Subsection 2.1.We identified a rather diverse nature of current MR game research body regarding in-game interactions, content representations, the degrees of mobility and immersiveness etc., all the way to the mechanics how the virtual and the physical components are synergized with one another.These highly divergent perspectives hence rendered it even more challenging, if not infeasible at all, to settle at a unified, one-size-fits-all solution.
Previous studies (Tsai and Wang (1999); Krahn et al (2008); Wang et al (2011)) indicate that a modular technology stack with readily integrated virtual and physical components will greatly mitigate the development cost and the technical hurdles.Moreover, it increases the overall customizability and adaptability so as to fulfill various situational needs and domain-specific requirements, thus facilitating research community and game creators to fully exploit MR for innovative, full-fledged serious games and gamified experience.
In this research, we first surveyed current state of the art literature about MR game development tools and frameworks, from which we analyzed the presented technological affordances from multiple dimensions.We then proposed our own MR game technology stack as a result of an iterative, research-through-design approach.Finally, we conducted a user-centered co-design workshop, where end users were invited to try and evaluate the proposed technology stack, and a thorough result analysis was done based on the collected user feedback.The overall research was structured as Fig. 1.

State-of-the-Art Review
A literature query was performed on 9th April, 2023.Our review targets were academic publications, initially retrieved from IEEE Xplore and ACM Library.Scopus as well as snowballing were for extra query results that were not included by the previous two data sources.We specified all the query strings identically, in a way that both MR/XR game, and one or more of the following keywords: game engine, development framework or toolkit, programming interface, editing or authoring tool, must be contained in all metadata.Thus, we were able to narrow down our review targets from general MR game studies, to those who plausibly accentuated the technological perspective and hence had adequate details for us to analyze their technological affordances.
After first-round screening based on titles and abstracts, apparently irrelevant results were eliminated, and a rest of 44 publications remained for the second round of full-text screening.We further ruled out less relevant articles, for example, in some papers MR was just a terminology appeared concurrently with VR and AR, while the described systems actually fell out of the definition of MR.Also, some papers presenting general findings such as design guidelines and common technical trends in MR systems without adequate details were also excluded.As a result of the second round screening, we ended up with 24 articles that were finally accepted.
While we can go through more implementation aspects e.g.software components, authoring/editing tools, computational and communicational architectures etc., it will usually be less meaningful if we focus only on the individual technical primitives and their combinations.Rather, we attempted to extract a higher-level understanding by drawing on the resulting technological affordances, according to Gaver (1991) and Hutchby (2001), which are defined by what end users perceive a specific MR system to be like, regardless of its building technologies.As Fig. 2 illustrated, we identified a set of technological affordances from current MR game systems, which then were categorized into three different spectra, respectively: shown in Fig. 2, the left extremity locates at where players are confined to a stationary setting similar to playing conventional computer games, and as mobility extends, players' physical activities can take place in table-or room-sized space.
For example, Kajastila and Hämäläinen (2014) proposed an augmented climbing wall where climbers followed a projected climb route on a wall-sized surface.The activity range might fall right between the interval of the table-sized and roomsized space.The right extremity of this spectrum encompasses pervasive gaming settings where players are able to explore location-based MR contents in a large geographical scale.et al (2015); Kajastila and Hämäläinen (2014)).That being said, it is difficult to assert that NUI must be closer to reality extremity than TUI in some occasions, but both TUI and NUI are generally considered closer to the real-life interactions taking place in physical reality.Surely, there are also hybrid user interfaces, like the work by Smith and Graham (2010), in which tangible objects (e.g. a car toys) were combined with the touchscreen GUI on a tabletop computer.3. Feedback Control: User-perceived feedback control that a MR system establishes between the virtual and physical entities.We defined the left extremity as "physical entity sensing", referring to that the MR systems appear to at least possess certain mechanism to capture the status of physical entities.Next, "virtual entity actuation" describes MR systems' ability to drive virtual entities or trigger virtual events in reaction to the status and its changes of physical entities.Many MR systems in our literature review located in the first interval between "physical entity sensing" and "virtual entity actuation".For example, Behmel et al (2014)  In this case, the interaction between virtual assets and physical entities permits not only sensing physical status and reviewing static virtual contents, but also dynamic construction and editing of the game levels.As the degree of feedback control gets more intensive, it will reach the the right extremity of this spectrum, which we define as "virtual-physical synchronization".
Both physical entities and their virtual counterparts involved in an MR game system are able to be fully synchronized.Whichever side has changed its status, it will thus trigger the corresponding update of the other's status in a real-time and automatic manner, similar to what is known as the "digital twins".Most work in the last interval between "physical entity actuation" and "virtual-physical synchronization" were human-robot games, like the work by Pratticò et al (2019); Jing et al (2017); Lupetti et al (2015); Tan et al (2006), where a physical robot could react to the behavior of virtual assets, e.g. a projected virtual pingpong on the floor.
It is noteworthy that each of the aforementioned technological affordance is a continuous spectrum, and again we want to stress that there are also cases that possibly fall between the intervals.Interestingly, these spectra coincide with the Reality-Virtuality Continuum and all manifest a gradual transition from virtuality (left) to reality (right).These three spectra were commonly shared by the MR game systems we have reviewed, despite the technodiversity that the literature body demonstrated.By positioning MR game systems at specific intervals on the spectra, it allows designers and developers to form a more precise and clearer vision of the target user experience, before coming down to the actual implementation stage.Thus, we believe that it is of specific benefits to take into account these general technical affordances, when designing and developing no matter a single use case or more generalized development tools for MR games.
In the following subsection, we will present an MR game technology stack as a result of an iterative design process, then further reveal the rationale behind the design decisions to showcase how we reflected on the aforementioned affordances in our own practice.

Mixed Reality Game Technology Stack
Reflecting on our findings from literature review, we propose a modular technology stack for designing and developing MR games.We followed a research-through-design approach and went through an iterative design process.
In the first iteration of our prototype, we intended to provide players with an activity range as wide as possible, but without being exclusively confined to pervasive game scenarios.Previous research by Cheok et al (2004) and Magerkurth et al (2005) has intensively investigated the combined use of location-based gaming and augmented reality; successful commercial cases like Pokemon Go also came out in the market.However, virtual reality that can be applied to both outdoor and indoor scenarios remains still an underexploited area.To this end, we drew on the mobility of smartphones and cardboard VR goggles, and further integrated Real-Time Kinematic (RTK) positioning module to improve the precision of player location data from around 10 meters (referring to ordinary GPS positioning) to a decimeter level.Specifically, we utilized commercial-off-the-shelf (COTS) RTK rovers taking account of the overall availability and affordability (see Fig. 3a).More details can be found in our previous work by Xiao et al (2021).By utilizing the native SDK of cardboard goggles, natural user interface can partially realized such as gaze and head movement.However, we found that with the only magnetic button sitting on the side of cardboard goggles, it highly constrained the interactability, which turned out inadequate for gameplay in most occasions.Therefore, in our second iteration we incorporated RFID module to enable fullspectrum user interactions, by leveraging motion detection via RFID tags attached to body surface (NUI) and RFID-embedded physical objects (TUI) (see Fig. 3b).RFID was preferred because it is cableless, battery-free, low-cost and flexible to be blended with environments and physical entities, thus highly compatible to mobile game scenarios where the players with their VR goggles on.One of our previous studies by Xiao et al (2022b) has showcased how to dynamically change players' virtual coordination, i.e. teleport, and load virtual assets from a remote server by scanning RFID tags.These use cases demonstrated the technical feasibility to drive virtual assets using physical entities, which was identical to the second interval located on the feedback control spectrum.
The overarching technology stack we propose is shown in Fig. 4. Unity was adopted as the hub, by which we integrate all system components except the external RFID host server to form a holistic MR game development environment.By building our architecture upon an existing popular game engine, it is able to reuse rich built resources and reduce development cost; for experienced Unity users, it can further lower down the learning efforts and maintain a consistent usage habit.We integrate three major functional modules on the basis of Unity, respectively: 1) Cardboard VR module, where we relies on the Google Cardboard VR SDK for Unity to handle lower-layer motion detection, like head movement, orientation and gaze etc. 2) Outdoor Positioning module, which leverages the Google 3D Maps for Unity to establish a spatial mapping between the physical and virtual environment.To achieved an enhanced GPS positioning precision, we employs a SparkFun GPS-RTK2 rover with a Bluetooth module on board.The RTK node with its patch antenna is a palm-size device and communicates with the smartphone via Bluetooth.3) RFIDbased Interactive module, where the mobile VR application utilizes UDP messaging to keep listening to an external RFID host server.To maintain an overall consistency, the program on the RFID server side was developed by using C#, same as the scripting language in Unity.The server program runs on an independent laptop, which connects to a ThingMagic M5EC RFID reader via USB cable.Whenever the RFID reader detects a tag, the host service will look up the tag's EPC (Electronic Product Code, acting as a universal identifier of an RFID tag) in a local CSV(Comma-Separated Values) file and send the associated data to the mobile VR application in a standard JSON format.Thus, the CSV file functions as an authoring tool, allowing users to customize the data being associated to a specific RFID tag (see Figure 4 for an exemplary data structure contained by RFID).It turns each RFID tag into a data input device or event trigger, which conveys the user-defined data on its activation to the mobile VR application for further processing the data or manipulating the virtual assets etc.The easy configuration of RFID tags and their associated in-game behaviors without complicated coding is a significant feature when we designed the technology stack, as most designers and developers of MR serious games and gamified applications are domain experts and researchers, who may not necessarily possess relevant expertise and technical skills on game programming.
By deliberately separating the RFID host device from the mobile VR, our intention was to cover a wide range of different user requirements on mobility and interactability.As implied from the former subsection, it entails a technology stack to be flexible and extensible enough so as to supply different technological affordances to the developers for different application scenarios.Consider two contrasting use cases: an urban scavenge hunt game and a multiplayer motion-based exergame.The first game scenario relies intensively on the RTK positioning module to navigate through an outdoor environment and reveal location-based virtual contents; while the players may need to carry a mobile RFID reader to explore the surrounding environment and scan hidden tags for e.g.hints for resolving a puzzle, narratives for guiding the players to the next location etc.While the second one is more about co-located multi-player embodied play, and multiple RFID tags can be assigned to different players and attached to different body parts; by making clever use of the read range of one or more positionfixed readers, enriched embodied interaction and social experience can be expected.By decoupling the RFID host service from the mobile VR app, our proposed technology stack allows tailored deployment according to particular on-site needs, such as mobile RFID readers (with Android OS) using 5G network or fixed readers with USB or WiFi connections to PCs, with little or no re-adaption on the mobile VR side.

User Co-design Workshop
To directly observe and gain an in-depth understanding of end user feedback about the intended use and affordances of the proposed technology stack, we organized a co-design workshop.A total of 15 participants who were interested in game design and development were recruited via internal chat groups in first author's institution.Therefore, all the participants were university students and research assistants, with a majority coming from design background and a minor group from technical, explicitly, the computer science background.Among them, there were 7 females and 8 males, ranging from 19 to 25 years old with an average age of 21.In addition, we also collected information about the participants' previous game making experience.7 participants claimed no experience at all, while 8 of them claimed that they have game making experience less than three years.
The overall workshop was structured into three sequential parts, respectively: 1) hands-on tutorial, 2) group co-ideation, and 3) user surveys and interviews.We will give a more detailed account of each of the topics in the following three sub-subsections.To note that, except for specific devices and materials such as cardboard VR goggles, RFID readers and tags, we did not provide the workshop participants with ordinary hardware and software like PCs with preinstalled Unity and Visual Studio.Rather, we asked the participants to bring their own laptops and smartphones, so as to approach the in-the-wild condition as closely as possible.

Hands-On Tutorial
In the first session, the participants were given an overarching introduction of MR games, then offered a step-by-step tutorial to establish the MR game development environment (see Fig. 5a).As shown earlier, the proposed technology stack is a composite of three distinct functional modules, i.e. the mobile VR module, the RFID-based interactive module and the outdoor positioning module.To prevent the learning curve being too steep for beginners, we further broke down the task into subtasks, where each functional module was added, run and tested independently, following a similar style to what is known as incremental development, according to Larman and Basili (2003).Specifically, the participants first downloaded Google Cardboard SDK for Unity, proceeded with necessary configuration, built and ran a sample game scene on their own Android smartphones.After successfully testing and experiencing the cardboard VR, the participants were then instructed to import the RFID messenger and the JSON data parser from a prepared unity package.This enabled the participants to dynamically load virtual assets from a remote web server into the same sample scene by scanning RFID tags.To brief the process, the RFID host server and web asset bundles were shared among multiple participants, hence not all participants needed to run their own RFID host service or upload their own assets to the web server.Before moving to the next stage, the participants again built and tested the newly-integrated module on the mobile ends.Finally, the outdoor positioning module and its dependent Google 3D Maps for Unity packages were imported.Unfortunately, due to the time and spatial restriction of the workshop, we were not able to let the participants to go out and test this function in the outdoor environment.Instead, we demonstrated several videos of sample game prototypes using the outdoor VR component.
Moreover, due to the hardware/software diversity and thus the resulting technical glitches, there were predictably varying progresses among participants at the end of the tutorial session.Instead of asking each participants to complete all the tasks, the workshop organizer demonstrated the overall workflow, and the participants were encouraged to proceed as far as they could within the planned time slot.

Group Co-ideation
Based on the intake of the previous session, the participants were separated into 4 random groups and continued to a 45-minute co-ideation session.After sensitizing them with the proposed MR technology stack via hands-on practice, video and live demonstration, we expected the participants to envision what can be done with the technology stack and generate conceptual game designs via group discussions and brainstorming.As a result of the co-ideation process, each group went through a 10minute presentation about their ideas (see Fig. 5b), followed by a voting session for all participants and audience to select their favorite conceptual game.As illustrated in Fig. 6, the four presented game concepts include: players can raise an imaginary pet in the virtual world, such as slimes, unicorns and dragons.By feeding the pets with different data, e.g.location-specific data, mental and physical data from the owner etc., it determines the unique direction of evolution for each virtual pet.Each pet is bound to an RFID tag, which can possibly be embedded into a key holder, an amulet, a mobile pendant and alike attachable or wearable accessories; and only collocated players with VR goggles on around a same "pet spot", where an RFID reader has been installed, can see the others' virtual pets.In this way, players in the real world can connect with each other through virtual pets, creating a community and the sense of belongingness by exchanging the experience of cultivating their virtual pets.2. MR murder mystery game: Conventional murder mystery games, or more generally, script entertainment, rely heavily on script text reading, artificial stage settings, game masters (GMs) and/or non player characters (NPCs) to proceed the story and puzzle solving.The second group suggested an MR-enhanced murder mystery game, where the proposed technology stack can be used to improve the overall interactability and immersiveness.For example, GPS can be used for guiding players to a real flower shop or around a scenery spot; while RFID tags can be hidden in costumes, treasure chests, weapons and other physical objects, and assign different virtual attributes to these objects.Say a player is role playing a spy with a mission to sneak into a banquet.He or she may need to pick and wear the right combination of costumes and accessories either by acquiring enough "elegance" value for the doorman (who wears VR goggles and can see all guests' attributes) to allow him/her enter the front entrance, or the player can go another way around by wearing the costumes that are low key enough then sneaking into the banquet from the backyard or service entrance.Thus, it enriches the way how the players can proceed with in-game tasks and interact with the NPCs, and greatly extend the game context and playful experience in a real-world settings.3. AR tour guiding game: The third conceptual design anchored in serious games with specific purposes to guide tourists through an itinerary around a scenery spot or within a museum and provide augmented location-aware information.The thrid group proposed to leverage a multi-branched narrative, which is oftentimes seen in text-based adventure games (T-AVGs).Players' choices on entering different story lines will then structure their itinerary in a different and meaningful way, thus shaping a more personalized and related sightseeing experience.Distinguished from the previous two game design, where the RFID tags are attached to the players and one or more readers are installed at fixed positions, players in this game need to carry a mobile RFID reader with them, which may be hidden in a lantern for instance.According to the points of interests displayed in a mini map, players may move the lantern around an environment object or exhibit, and when the lantern get close to a hidden RFID tag, it will then light up and reveal a secret hints or a piece of key information for the players.In this case, a see-through AR mode will be used to provide players augmented visual presentations while maintaining their situation awareness (see Endsley ( 2021)) towards physical exhibits and surroundings at the same time.4. MR multiverse: The forth conceptual game design is similar to the third one in the sense of its emphasis on educational purpose, as well as that players can switch to several "parallel universes" by making different decisions in the game.The proposed technology stack is used to show players a virtual timeline of a real natural landscape, by traversing its past, presence and future.At certain point, players are given the chance to determine the direction of how future will develop.For example, if players decide to take care of a sapling in the virtual world, it will help increase the chance to avoid possible future disasters as results of global warming and climate changes.The more people make the sustainable choices, the more likely for the game to arrive at a promising future.Among all the above conceptual game designs, the MR murder mystery game was the top voted idea.For the winning team, each member was rewarded a steam gift card that worth 50RMB.

User survey and Interview
Following up the co-design workshop, we conducted questionnaire-based survey and semi-structured user interview with individual participant, and each session lasted for around 15-20 minutes.All 15 interviews were carried out either online or in person, and completed within less than 5 days after the workshop, so as to ensure that the participants' impression of the workshop content remained fresh and clear.All interviews were audio and video recorded.We conducted our user experiments in compliance with the Helsinki Declaration (2013).Prior to user experiment and each interview session, we informed the participant about his/her right to withdraw from the interview at anytime, the use of collected data, e.g.individual information, images and sound, as well as other relevant issues.The participant was asked to sign an informed consent form, if he/she agreed to grant his/her consent.
The participant was first directed to an online questionnaire page, which consisted of four questions, as shown in Table 1 below.We referred to some existing validated questionnaires, e.g.Validated UTAUT2 questionnaire for elderly (see the work by Siow ( 2016)) and so forth (Camilleri and Camilleri (2022); Indrawati and Putri (2018)), and then tailored our own questionnaire accordingly.The participants were asked to respond to the four questions using a 5-point Likert scale, with the lowest point 1 indicating the most negative feedback (very difficult/strongly disagree) and the highest point 5 indicating the most positive feedback (very easy/strongly agree).After the participant filling up the questionnaire, the semi-structured interview would start.Some prepared open-ended questions were listed in Table 2.All recorded interview audios were later transcribed into text files, and Thematic analysis was conducted to break down and comprehend the transcribed interview data.Two researchers have conducted the coding independently, and the discrepancy in the resulting codes were then addressed by discussions until the consensus of the research team was reached.

Questionnaire Survey Results
Based on collected questionnaire responses, we further summarized, analyzed and visualized the user survey results using bar charts (7).The results showed that despite plenty of technical glitches and first-time contact, the workshop participants perceived an average learning difficulty around 3.2 (standard deviation=0.98)about the introduced MR game technology stack (Q1-1).Moreover, regarding perceived innovativeness (Q1-2, potential contribution to innovative game development), perceived motivation (Q1-3, user motivation to game development), willingness of future use (Q1-4), the results indicated a considerably high average ratings of 4.4 (standard deviation=0.8),4.5 (standard deviation=0.62)and 4.5 (standard deviation=0.81)respectively.
Given a small sample size of 15 participants, it was difficult to assert any statistical significance.However, some results, e.g.high perceived innovativeness and motivation, relatively medium learnability, were further reinforced by the following semi-structured interview.In general, we believe that there manifested a common positive feedback about the perceived potentials of our MR game technology stack.

Semi-Structured User Interview Results
In this subsection, we will present the common themes that have emerged from the user interview, following the same sequence as the interview questions.For the first question, "which conceptual game design did the participant vote for as their favorite one", most participants' answers were the MR murder mystery game, even though we did not specifically restrict the participants to vote for the idea of their own team.Prior to the COVID pandemic, the amount of offline murder mystery shops in China reached 30,000, according to Xie (2021), and it has already become a popular entertainment and socializing form among the younger generation.Indeed, when asked about the reason why they voted for the MR murder mystery game, a significant part of the participants answered that they had experienced the commercial offline ones before.As one participant P3 said during the interview: "Conventional murder mystery game has limitations like restricted immersion and limited player communication, mostly relying on oral communication.I believe that's where MR technology can come in and improve." Interestingly, multiple participants (P3, P6, P7) unanimously expressed the idea that MR murder mystery games, which do not fit in the traditional definition of serious games though, fall into a specific genre of games-with-a-purpose to "provide added values beyond just digital entertainment that contribute to offline real economy, like local tourism, retail and cultural events" (P6).A great potential and innovation space awaits for further exploration, where the technology is expected to bring in not only a stronger sense of immersiveness and authenticity, but also greater opportunities to generate real social impact.
When asked "what kind of games that would be suitable to leverage the proposed technology stack", the participants' opinions branched into two divergent tracks.The first one, partially aligning with the observation drawn from the first question, includes offline murder mystery games, LARPs, scavenger hunt and other service and event games that entail an intense incorporation with specific physical environment, stages, costumes and items etc. Distinguished from product games, it appears more frequently on the event organizer and service provider side the duty of preparing game settings and particular gaming equipment, e.g.MR headsets, RFID tags and readers.Thus, it permits new technology-enabled gaming experience with accessibility towards a wider audience, without imposing extra device requirements on the users.One such application scenario mentioned by both P7 and P14 was for the museum and exhibition visitors, closely related to the third AR tour guiding game concept presented earlier.P14 claimed herself to be a history museum mania, "but oftentimes I found those exhibitions lacking the handlers for long-term memory.It's easy to lose track with the exhibits because of information overload."To this end, the proposed technology stack is believed to be able to magnify the efficiency and effectiveness of digital storytelling and narrative based serious gaming, according to Abrahamson (1998).
In contrast, the other possible game scenarios suggested by the participants focused on a more casual or even daily gaming context, e.g.location-based mini games that players can carefreely initiate and terminate a game session during their commute time.This sort of mini games can possibly be embedded into the existing social media applications, e.g.WeChat, Tiktok, Facebook etc., and do not require players to download any game software or applications, or go to specific game venues.One such example brought up by P5, which she thought could be combined with MR, was a mobile pervasive chasing game making use of WeChat's location sharing function.Similarly, P9 also mentioned another WeChat applet, YangLeGeYang, a match-three mini game which got explosively popular over the Chinese social network in 2022.He commented that: "This kind of light-weighted applet games with simple gameplay, once spread through the social media, may have the chance to grow into a phenomenally popular one.Then people may be attracted to buy a cheap cardboard VR headset and put their phones into it, just to try and play the game." Two bottlenecks can be implied from the above comment, which prevent current MR/XR games from further popularization: technology availability and lacking of killer applications.We have witnessed earlier in conventional console and PC games market how a phenomenal hit title could spurred purchases and massive-scale upgrade in consumer hardware, mainly game consoles and graphics cards.While MR/XR games may yet stay out of the mainstream options of gamers, low-cost entry-level devices for easy tryout may help lower down the barriers for technology availability aside from a killer application; and the effects of "viral marketing" via social media may further accelerate this process.These findings are also in line with our earlier research by Xiao et al (2022c) about user engagement in technical systems, however, the same study also pointed out that a more profound ecosystem and game culture need to be cultivated for maintaining long-term user engagement, when the initial technical novelty wears out.
As for whether there was any pre-/post-workshop differences or not, 14 out of 15 participants gave positive feedback.For non-experienced participants, the most reported change was the transition in their understanding of game design and development.According to P2, "To my understanding, game making was once simply equal to coding and software programming...For the first time, it made me aware that how much physical components can actually lend to game creation.(For example?)Say I am making a rogue game, and I can tweak the parameters of each virtual item at each level in a very detailed and complicated way, but still it's totally different...It can't compare to the real feeling that physical objects brings about." The same insight was shared and reinforced by the feedback from the experienced participants.When compared the proposed technology stack with their previous tools, e.g.Unity, Processing etc., a significant portion of participants considered that the introduction of RFID and geographical positioning add to overall interactability as well as an enriched spatial experience, making them start to "think outside the screen".Or as P11 put it, "My research topic is about VR interactive narratives.Many current VR interactive films still heavily rely on or just simply transplant the conventional screen-based interaction into VR, like popping up a menu and asking the audience to pause and make a choice at certain point.VR turned out to be no more than just a gimmick-like thing.The use of RFID makes it possible to rely on more natural and meaningful use interactions, instead of asking users to learn and follow an interaction guide or explicitly perform something." There surely were some negative feedback about the use of the programming toolkit, which also directs us to our last question: What further improvement would you like to propose for the current technology stack?
We have received some specific feedback regarding the development tools of the proposed technology stack, explicitly, the comprehensibility of programming interface (7).For example, as bare RFID tags are utilized as part of the programming interface, the participants complained that it was difficult to distinguish between multiple tags; also, there was no visible indicator in current interface for showing the read range of the RFID reader, resulting in repetitive tests and extra efforts for users to try out and adjust by themselves.Therefore, a more distinguishable RFID interface was considered as a must (3).In the same vein, users now have to directly modify the source code so as to adapt to their own local settings and environment, for instance, users may need to specify the IP address and port number of their own RFID host server by changing the corresponding code.A better encapsulated interface instead of exposing lower-layer technical details will facilitate especially non-tech-savvy beginners (2).In addition, tutorials and community support were specifically valued by some of the experienced participants.(3).

Discussion
As the principal results of the user evaluation, we have recognized a few commonly shared opinions among the participants.By synergizing them with our own interpretations, which were rooted in our previous practice and empirical knowledge, as well as evidences retrieved from existing literature, we came up with some major insights and further mapped them to the three technological affordances spectra.To specifically note that, these insights are not confined to the proposed technology stack, but rather generalizable to be applied to a wider MR game research context.We hence present the three major insights below, with bracketed numbers indicating the amount of advocatory participants respectively: 1. Seamful Design, Seamless Experience: The use cases that attracted most attentions were located between the intervals of room-sized and pervasive games on Activity Range Spectrum (8).Pervasive MR games are considered to expand and transcend the temporal, spatial and societal boundaries between the game and real world, or the "magic circle", which exists in conventional games according to Huizinga (2020) and Montola (2011).Designers and developers may have to well balance the immersiveness and the situation awareness simultaneously, which is greatly distinguished from ordinary pervasive games or fully immersive VR games.For example, several participants (4) mentioned that it may on occasion break a coherent gameplay if players need to put on/down their goggles from time to time in an outdoor gaming scenario, or when the players need to perform the action of scanning an RFID tag with a reader.This issue can be partially addressed by concealing the technologies underneath the surrounding environment and objects without users' explicit awareness of their presence.We also suggest seamful design, proposed by Chalmers et al (2003), as a practical approach to associate underlying technologies and the seams they create with meaningful interpretations in consistency with the context of the gameplay.One such example is the work by Yi et al (2020), where a mediation device for museum exhibition visitors was proposed.When the visitors hold the stethoscope-like device close to an exhibited object or panel, which was embedded with RFID tags, and move along the surface, he/she can see augmented virtual contents through the scope such as the cardiovascular system inside the human body or the internal texture and structure of a rock sample.By employing an analogy of stethoscope, it successfully established a meaningful mapping between users' action of holding a VR display with the behavior of exploring knowledge itself.This kind of metaphorical design is also able to immerse uses into a playful and consistent experience, without distracting their focus from the physical targets to the virtual contents.2. Think Outside the Screen: We found that participants generally favor the TUI and NUI on the right side of User Interface Spectrum more than the conventional GUI in screen-based games on the left end.This observation aligns with some established design guidelines such as the one by Bowman et al (2004).It is commonly agreed that user interface for MR/XR systems should be native and dedicated to the fully or partially immersive environment, however, in practice we found that it is not always easy for designers and developers to truly think outside the screen.Auxiliary props like RFID were reported explicitly helpful for pivoting designers and developers around alternative user interfaces involving physical entities and body movement (8).The participants also expressed their appreciation towards RFID's ability to foster unambiguous and unobtrusive identification and tracking without complicated electronic engineering and programming (4).As a single RFID tag affords only binary input as either activated or deactivated, some participants also expressed their expectation for a better granularity of motion detection (3).It is possible to mitigate the issue by a more sophisticated arrangement, e.g. an RFID tag array with dedicated gesture recognition algorithm developed by Wang et al (2018), or incorporating extra sensor units like IMU, so as to capture continuous, fine-grained motion data.However, our experience as well as previous research by Mueller et al (2018) both confirmed that an engaging game experience can benefit more from the innate ambiguity of embodied interaction than from a fully reliable and accurate sensing/tracking utility.A good example is the party game, 1-2-Switch by Nintendo, which allow players to enjoy filling the blank where the joycon cannot sense, by their own bodily improvisation and performative play.Therefore, one may need to consider what will be the best match between the intended game experience and the granularity of user interaction, e.g. is full-body motion tracking the core to the gameplay?Again, this probably requires MR game designers and developers to think outside the technology framework.3. Play with Virtuality and Reality: Referring to Feedback Control Spectrum, our user study results manifest the participants' needs for enhanced physical entity actuation (1) and in some occasions a full-fledged virtual-physical synchronization (2).The former is consistent with our previous finding by Xiao et al (2022a) in a way that existing smart daily objects and environments were suggested to provide new play opportunities and new in-game feedback modality.For example, suppose a player enters a dark and icy dungeon in a game, it will thus actuate the connected smart home appliances to lower down the room temperature and dim the ambient light, so as to create a responsive physical environment as part of an enhanced and holistic game experience.In the same vein, research community are seeking to cohere vestibular and proprioceptive systems with visual stimulus, e.g.aligning vertical floor vibrations with cannon bombing in a VR game (Jung et al (2022)), or utilizing the sense of gravity as a resource for game design (Hämäläinen et al (2015)) etc.In our user study, the participants had the concerns that when players interact with RFID-embedded physical objects, virtual feedback might not be adequate especially when there are multiple similar objects nearby at the same time.It implies a need for enhanced physical feedback or even a higher-level synchronization between the status of physical entities and their virtual counterparts.Currently, the most relevant research identified in this direction was MR based robotic games.
While it is technically feasible to incorporate more sensing and actuating utilities into MR games, rather than simply mutual mirroring between a physical entity and its digital representation, a more sophisticated mechanism entails more future research efforts in this highly context-dependent and open-ended area.

Limitations
The proposed technology stack relies on smartphones coupled with cardboard goggles to make an abridged mobile VR display.The hardware limitations in computational capacity and the graphic processing ability of current smartphones result in a compressed 3D graphic quality inferior to dedicated displays.Two participants in our user study have mentioned this issue.Another limitation lies in that currently RFID is not among standard smartphone configurations, although NFC offers similar utility but with a much shorter read range.It thus requires extra RFID reader devices, while there are mobile readers with preinstalled Android OS, we have not yet tested the proposed technology on them.It has been our intention that the technology stack settles in the form of an open source project, and users will have the full access to the source code.To configure and compose their own MR game applications, for now, users need to directly modify part of the source code, e.g.C# scripts in Unity, CSV files at the RFID server end etc.The user study results suggested that it was inadequate to expose the programming interface as bare source code.To this end, we are considering that a tangible programming interface may facilitate end-user development (EUD) and collaborations within a mixed reality environment.This will be directing our future work in the next iteration, and user elicitation and evaluation will be further conducted throughout the process.
As for research process and experiment design per se, the result of questionnairebased measurement was restricted to the limited sample size (p=15).Also, the participants' perception of the proposed technology stack and related tools may be confined to the limited use experience during the co-design workshop.In the future, the researchers plan to organize further evaluation, e.g. by comparing the proposed technology stack with conventional development methods.

Contributions
In this article, we first concluded the technological affordances of MR game systems from current state-of-the-art literature.The identified technological affordances encompassed three different spectra, namely 1) activity range, 2) user interface and 3) feedback control, each of which reflected a gradation between virtuality and reality from the left to the right extremity.By mapping MR game systems at specific intervals on the spectra, we believe that it will help MR game designers and developers establish a precise and clearer vision of their target user experience, prior to an actual implementation stage.
Secondly, we proposed a general-purpose technology stack for MR game design and development, based on our reflection of the previous technological affordance spectra.It consisted of three distinct functional modules, respectively 1) mobile VR module, 2) RFID-based interactive module and 3) outdoor positioning module.To fully evaluate the proposed technology stack, a user co-design workshop was conducted, and a total of 15 participants took part in the workshop with 4 conceptual game designs generated as outcome.According to the user survey and interviews, we received highly positive feedback about the perceived innovativeness, the perceived motivation, the willingness of future use of the proposed technology stack while a moderate feedback about the learning difficulty.Future applications and potential improvement were identified from the user interview.
Last but not least, we have further discussed three generalizable design insights in the principal results subsection, including: 1) seamful design, seamless experience, 2) think outside the screen, and 3) play with virtuality and reality.We believe these insights, as a reflection of our previous empirical research and speculative design practice, will contribute to the collective knowledge body and facilitate the design and development of future MR serious games and gamified applications in the coming era of metaverse.

Fig. 1
Fig. 1 Research Structure and procedure

Fig. 2
Fig. 2 Technological affordance spectra of MR game literature 2. User Interface: User-perceived access point to interact with the MR system.At the left extremity of this spectrum locates graphical user interface (GUI), where conventional WIMP (windows, icon, menu, pointer)-style elements or their 3D counterparts are directly transplanted into an MR environment.Most mobile-based pervasive games adopted this kind of user interface, such as the work by Samodelkin et al (2016); Cavallo and Forbes (2016); Linner et al (2005).Following GUI, we have witnessed the use of tangible user interface (TUI).In MR context, TUI often involves physical objects and surfaces that are not intentionally devised as input devices, forming a hybrid game experience that blends physical entities and digital contents.Some examples include the work by Harley et al (2017); Oswald et al (2014); Jing et al (2017); Yannier et al (2013); Hong et al (2017).The right extremity is natural user interface (NUI), including but not limited to voice commands (Wang et al (2019); Prompolmaueng et al (2021)), gaze (Wang et al (2019)), gestures (Wang et al (2019); Swearingen and Swearingen (2018); Smith and Graham (2010)) and body motion (Cheok et al (2002); Oswald et al (2014); Lupetti

Fig. 4
Fig. 4 Proposed MR game technology stack

Fig. 6
Fig. 6 Four user-generated game design concepts

Table 1
User survey questionnaire

Table 2
Semi-structured user interview questions