Yaying Zhang
Abstract
This work explores the design of stick-shaped tangible user interfaces (TUI) for Extended Reality (XR). While sticks are widely used in everyday objects, their applications as a TUI in XR have not been systematically studied. We conducted a participatory design session with twelve experts in XR and HCI to investigate the affordances of stick-based objects and how to utilize them in XR. As a result, we present a taxonomy of stick-based objects’ affordances and propose three types of stick-based XR controllers and their dynamic variations. The paper discusses design considerations for selecting the appropriate stick-based form in XR TUI design.
1 INTRODUCTION
Tangible User Interface (TUI) expands the scope of Human-Computer Interaction (HCI) by diversifying user interaction methods beyond traditional keyboard, mouse, and touchscreen paradigms. With the advancement of TUI, researchers have been exploring the relationship between shapes and hand grips to inform TUI design. One of the most recent works in the field is from Serrano et al., which explored how the shapes of handheld freeform devices affect interaction [17]. Other studies focused on shape-changing objects [14, 15, 16]. Researchers have also studied hand gestures with objects [2, 4, 9, 18, 27].
In this work, we also look at shape design and hand-grasping gestures on the shape. But we specifically explore the shape of stick and how to adopt this form into Extended Reality (XR) User Interface (UI) design. Here, we define a stick as an object with a long, thin, and cylindrical shape. We are particularly interested in stick shapes instead of compact shapes like balls or disks because not only can sticks be held in a user’s hand, but they can also contact or link to other objects, ground, or walls. This could potentially afford interesting interactions in XR. As we all know, sticks are a common shape used in daily objects. This shape can be found in a diverse range of contexts, including our professional lives (in pens), education (teaching sticks), sports (rackets), and general household items (kitchen utensils and tools).
As anticipated, this shape is widely adopted in HCI and XR as a TUI. Most commercial Virtual Reality (VR) controllers are shaped as a short stick in hand, such as Quest or Vive controllers [11, 23]. Academic research has also suggested diversified XR controller designs of stick form. For instance, Harders et al. used a sharp-tipped pen to perform high-precision operations in Augmented Reality (AR) [7]. As another example, Zhao et al. proposed a cane that is attached to programmable brakes to simulate touching virtual objects for blind users’ VR experience [26]. Each of these designs adopted some affordance of stick shape like Harders et al.’s precision interface used a thin stick’s tip for accurate pointing [7], and Zhao et al.’s VR cane works as an extension for the user’s arm to detect the environment [26]. However, there is no study yet that has systematically explored the design space of the stick-based TUI in XR. There are more usages of stick shape design that are found in daily usages, but not fully adopted in XR yet. For example, its feature of supporting and balancing the body might be helpful for stable operations in XR, and its leveraging feature might be used to save effort.
The two research questions (RQs) that guided this work are:
RQ1. | Stick-Based Objects’ Affordances: What daily objects are stick-based, and what are their affordances? | ||||
RQ2. | Design Insights of Stick-Based Design in XR TUI: How to apply stick-based design in XR TUI? In what scenarios might such designs be beneficial? | ||||
To understand these questions, we conducted a participatory design session with twelve experts in XR and HCI. Participants brainstormed on the affordance of daily stick-based objects and designed stick-based XR controllers on paper. The data collected allowed us to develop a taxonomy for stick-based objects in terms of functionality, holding, movement, and contact (Section 4.1). Additionally, it led to the design of three types of stick-based XR controllers and two types of dynamic variations (Section 4.2). We discussed the design considerations for these forms on flexibility, support, and control aspects to aid in selecting the appropriate form for specific design intentions (Section 5).
2 RELATED WORK
Our work contributes to TUI knowledge by studying stick shapes on their functionalities, hand-grasping gestures, and applications in XR.
2.1 Shape Studies
We study the stick shape, especially identifying how their diverse shapes (such as length or thickness) afford distinct functionalities. With this, we can further infer how stick-based TUI can solve existing issues in XR. To the best of our knowledge, we have not seen shape studies for stick-based objects. However, some studies have cast light on freeform or shape-changing designs. For example, Majken et al. explored shape-changing interfaces and identified eight types for shape transformation [14]. Roudaut et al. proposed the notion of “Shape Resolution” to describe the flexibility of shape-changing device [16]. As another example, Serrano et al. arranged participatory design sessions to understand how shape affected interaction for freeform handheld devices [17]. They provided insights such as the tradeoff between holding and interacting and the benefit of using metaphors and docking for feature discoverability. Similarly, He et al. [8], via a participatory workshop, explored design possibilities for using multiple tangible cubes for interacting with visualizations in XR. We used a similar participatory design method to involve experts in brainstorming and prototyping stick-based XR controllers and elicit their insights on the design space.
2.2 Grasp Gesture Studies
In our study, we are also interested in analyzing the gestures for interacting with stick-based objects. This could inform XR controller designs regarding the appropriate types of sensors and control and optimal locations to embed them. Regarding hand gesture studies, Cutkosky proposed a grasp taxonomy [4], which discussed various hand grasping types for more precision or more power for compact or long-shaped objects. Zheng et al. later examined these grasping gestures with housemaids and machinists’ daily activities and identified the most common grasp gestures for these activities [2, 27]. Sharma et al. studied single finger gestures when a user grasps an object [18]. Zhou et al. proposed Gripmarks [28] to detect the handheld object shape by the user’s griping gesture and thus construct an interactable surface on top of the handheld object. These prior works provided a grounded understanding of grasping gestures in general, but none had been conducted specifically for stick-based objects and in the context of XR interaction, which is what this work focuses on.
2.3 Stick-Based XR Controller Studies
There are a variety of existing XR controller designs that adopt stick shapes. For instance, Microsoft Research proposed VR canes for blind or low-vision users to experience virtual environments [20, 26]. Some researchers explored stick-shaped handheld controllers that are shape-changing to simulate weight and drag (e.g., [12, 19, 22, 24, 25]). Strasnick et al. considered linking two controllers in different dynamics [21]. We will refer to more existing stick-based XR controller studies along with our taxonomy presented in Section 4.2 and discuss the applications of stick-based design in XR that have not been explored in Section 5.
3 METHOD
We invited 12 participants (6F/6M, aged 25.6 on average, 4 in the United States and 8 in China). We recruited participants who have expertise in XR, hardware design, and computer-aided 3D design. Table 1 summarizes participants’ self-reported expertise. They participated in the study in groups of 2 or 3. The study consisted of a 1-hour participatory design session with 4 activities. The study began with participants listing out names of stick-based objects to the best of their abilities. These answers were displayed in real-time on a shared screen using the Mentimeter presenter tool1, allowing participants to draw inspiration from one another. Secondly, the participants were asked to write down the affordances of stick-based objects, considering their usage, holding method, and how they contact other objects. For instance, one of their answers was “Hammer: (Usage) Break stuff / push stuff with a lot of force; (Hold) grab by the not metal part, wherever it is easy to swing; (Contact) stuff to be smashed”. Participants’ answers are also posted on a shared screen to inspire each other.
The first two activities aimed to familiarize participants with various stick-based objects and their associated affordances, while the next two activities guided the participants in exploring the application of these objects in XR interaction. The third activity asked the participants to write down scenarios where stick-based designs could be employed in XR interaction. For example, P5 answered, “A cane stick can be used so people have something to hold on to during certain experiences that will help them not fall or lose balance. (Like) a hike experience; (or) something that requires walking (and people need extra help).” The experimenter discussed the answers with the participants and asked follow-up questions as necessary.
Lastly, participants were instructed to create a stick-based XR controller design on paper or a whiteboard. We provided a set of design aspects to prompt more concrete creations: 1) grip method, 2) interaction with virtual objects, 3) buttons/controls/sensors, and 4) the feedback it provides to the user. Participants were given approximately 15 minutes to complete their designs, after which they presented and explained how their designs addressed the four aspects. They were also asked to compare the pros and cons of their designs with free hand control and conventional XR controllers like the Meta Quest controller. The experimenter posed follow-up questions as necessary.
Participants’ textual responses and conversation recordings were analyzed through open coding, axial coding, and selective coding. We present our preliminary findings in the next section.
4 RESULTS
4.1 Stick-Based Objects and Their Affordances
The stick-based objects listed by our participants are displayed in Figure 1. The objects most mentioned include “cane”, “racket or bat”, “pen”, and “gear lever”. From our initial analysis, we present the taxonomy of stick-based objects’ affordances via four aspects: functionality, contact with the user, moving direction, and contact with other objects.
Figure 1: The world cloud of stick-based objects that participants mentioned. The same item with different names was merged to reflect the mentioned frequency (e.g., “walking stick” and “cane” mean the same, thus represented both as “cane” in the world cloud result).
4.1.1 Functionality.
We conclude the functionality that stick-based objects offer into these types:
F1 – Point or Position: Teaching stick, remote control, gear lever, etc. The stick is usually long and straight. And the user tilts its angle to point to the right direction.
F2 – Support or Handle (to Resist Gravity): Including objects to support people, such as a cane or prosthetic leg, and objects to support things, such as a clothes hanger or table leg. The stick either stands on the ground vertically with one end and provides support on the other end or is fixed horizontally to let objects hang on its bar-shaped body.
F3 – Manipulate Other Objects: Many sticks are designed to work with other objects. The two most common manipulations are moving objects and deforming objects. Moving objects includes these ways: pinch to move (chopsticks, pliers, etc.), punch/hit to move (racket, hammer, etc.), lever to move (crowbar, etc.), pull to move (fishing rod). It can also deform other objects in these ways: pressing flat (rolling pin etc.) or poking holes (needle, fork, toothpick, etc.).
F4 – Enable the User to Move Forward: Like rowing oars, hiking sticks, and ice axes. This type of object is used one per hand and touches the ground alternately to assist user movement. It functions as an extension of human legs but is controlled by hands. Tools like hiking sticks serve both F4 and F2. While implements like hiking sticks fulfill dual functions of F4 and F2, others, such as the rowing oar, are only in the F4 category.
F5 – Write or Draw: Including varying types of pens. The user holds the stick with ink or paint on the tip to work on paper.
F6 – Miscellaneous: We also discovered other less common functionalities, including connecting objects (skewers, rods, connecting wheels), transporting (hollow sticks for other things to pass through, such as pipes, straws, etc.), saving space (things like cloth hangers to organize objects vertically to save space).
4.1.2 Contact with the User.
Some stick-shaped objects, such as a tripod, or pipe, have no contact with a human and function by themselves without a user. We are more interested in objects that have a user and focus on where and how to hold the stick.
Holding positions on stick – A stick shaped object can be held near the end of the stick, such as racket, spoon, crowbar, near the middle of the stick: such as pen, chopsticks, and on the top of the stick, such as cane, gear lever. Sometimes, one holds a stick with both hands. This is usually to increase the swinging power, and the hands are usually on either the end or the middle of the stick, e.g., baseball bat, polearms.
Holding methods – Previous research already discussed gripping gestures of how a general object could be held in hand [2, 3, 5, 27], which could be applied to stick-based objects too. For instance, the GRASP taxonomy [5] introduced Power, Intermediate, and Precision grasps with variations of thumb abducted or adducted. In addition to existing gripping gestures taxonomy which mainly focused on hand holding, we observed that the stick could be held with a user’s hand and elbow, such as some canes, or a user’s hands and shoulder, such as carrying a pole. It could also be not held by a hand, but attached to a user’s body, such as prosthetics.
4.1.3 Movements.
We found four types of movements while using sticks in daily objects, as summarized below.
M1 – Linear Movements: The user holds the stick and moves in a straight direction. This includes pulling/pushing (such as fishing rod, door handle or drawer handle), poking (needle, etc.), supporting (the user pushes down on the stick to support their body up, such as cane).
M2 – Rotational Movements: These movements include pivoting (where the stick is put on a pivot point, and the user moves the end of the stick around, like a gear lever. It could also be press down one end to lift the other end up, such as crowbar), rolling such as using a rolling pin to press or flatten a piece of dough, and stirring to mix content in a container.
M3 – Complex Traced Movements: A lot of stick-based objects are interacted with by swinging, such as sword, rackets, paddle, wand. Another unique but important movement is writing with pens. Participants also mentioned throwing movements used when using a dart or javelin to throw toward a target, and bending motions are used when using a pole or pole vault to bend or flex the stick-shaped object to gain momentum or height.
4.1.4 Interacting with Other Objects.
Not all stick-based objects have to interact with other objects while being used. For example, dumbbells are used in the hands for exercise without needing to touch other objects. However, many stick-shaped objects are used to interact with other objects. The ways they interact or contact can be:
C1 – Touching on the Side of the Stick: It could be with one end held by a hand, and the other end contacts objects on the side of the stick, such as a baseball bat, hammer, or spoon. This is usually used for applying a direct force onto an object, like striking. It could also be held by two ends by hands and the middle in contact with an object, typically used for applying pressure, like a rolling pin.
C2 – Touching on the Tip of the Stick: This could be the object attached to the tip of the stick to be pulled, usually for transmitting force, such as a fishing rod or some dog toy, etc. Another case is like a pen or needle, to interact with a surface-shaped object like a piece of paper. A third case is like a cane, which uses its tip to contact the ground to provide support and stability for the user. Another case is where both ends are connected to objects like drawer handles, which are typically used for pulling, too.
C3 – Immerse the Stick In: One end of the stick is held by hand, and the other end is immersed in a liquid object (typically used for stirring), like a stir stick to mix a liquid-based object.
Figure 2: The forms for stick-based designs in XR Controllers and their variations. Red, green, and blue areas represent areas where controls can be put for the thumb, index, and middle fingers, respectively.
4.2 Stick-Based Design in XR Controllers
In this section, we will outline the noticeable patterns derived from these designs. Participants’ designs can fall into these forms:
4.2.1 Basic Stick.
The most basic form is a short stick held in hand with the thumb resting on the side and the other four fingers wrapping around the stick (as shown in Figure 2-A1). It is the most frequently mentioned design proposed by P2, 3, 4, 7, 8, and 11. Its usage adopts multiple movements that we discussed in Section 4.1.3, including pulling, swinging, levering, and stirring. P11 also proposed another grasp pattern using this form, shown in Figure 2-A2, for which the index finger is straight on the stick, used to press the stick. The colored areas marked in the figure are where buttons or controls should be for each pattern. Current XR controllers like Quest or Vive controllers are in this form.
The Basic Stick’s A1 and A2 pattern can also adopt a weight-changing variation that uses mechanisms to change the center of weights, thus simulating different holding feelings (P8 and P11, see Figure 2-A.V1), similar to ElastOscillation [22], VibroWeight [24].
4.2.2 Long Stick (Cane).
Building upon the basic stick form, when we integrate a long stick into it so that it meets the ground, it evolves into what we refer to as the Long Stick or Cane form. This variant is the second most prevalent form from participants’ submissions. In this form, the user’s movement is limited in that the user basically can only orbit the hand-holding part around it touches the floor, but the user also gets support from the ground, which can potentially help the user to keep balance or make the hand movement more stable. This form also has multiple patterns in terms of where the long stick attaches to the short stick. It could be from the end (P6, Figure 2-B1), from the middle (P2, Figure 2-B3), from the top (P4, 5, 10, Figure 2-B2, B4). P5 even mentioned a U-shaped connection like a shove handle (Figure 2-B5). Each of these variations has different holding gestures and where buttons and controls can be placed, as illustrated in Figure 2. Participants usually give the thumb more controls, such as a big touchpad (P9 and P10) or a T-shaped button bar (P6, Figure 2-B1).
For the usage of this form factor, participants commonly pointed out this cane form can be used for accessibility, specifically in assisting users with mobility challenges while using the XR experience (P5, 6, and 10). P6 also suggested that the cane form offers stability, which could be beneficial in tasks like environment scanning.
Furthermore, the design also includes variations of adjustable length options (P2, 11, and 12, Figure 2-B.V1) to accommodate various needs. Besides the normal cane form, P8 created a design to extend the stick’s length by pulling out a rope (Figure 2-B.V2). She felt this provided a unique dynamic of “controlling a soft element with a rigid one,” and it contrasted the rigid strength of the stick with the flexibility of the rope.
4.2.3 Thin Stick.
This design is a thin, light stick that can be held between fingers (rather than using the whole palm). One such design is pen shape design (P6, 7, 8, Figure 2-C1). Its tip can be used for accurate selection, and its end can embed a button to press and/or be used as an eraser. P12’s design also adopted a thin stick, and it features a symmetric design where the stick’s both ends works interchangeably. This provides the flexibility of holding either end, front or back. This is useful in cases like medical lab simulation, where the operator often uses tools, such as pipettes, with different gestures (Figure 2-C2, C3) and conventional controllers are not able to facilitate such need. Another benefit of this form is that it is lightweight and easy to carry. P11 said it could be attached to XR glasses. Control-wise, because it’s too small to have many buttons on its body, participants mainly adopt squeezing and tabbing gestures (P7 and P11).
This form leverages the mobility of both the fingers and the wrist and enhances flexibility in movement. Hence, its potential usage spans fields that need precision and extensive motion. Participants pointed out it can be used in scenarios like office work (P7), drawing (P6), medication (P12), and teaching (P7).
4.2.4 Modular Design.
Some participants also applied a modular design in their work. Both P4 and P5 developed a cane design that features interchangeable upper and lower segments that screw to the main rod. The upper segments are designed for diverse handling styles (e.g., sword handle or shove handle). The lower segment can accommodate various types of end pieces, such as shove head or tripod end, and it is capable of hosting sensors that can potentially detect when the cane strikes the floor. The main rod contains batteries and is the center of weight.
Participant P1 introduced another modular design that comprises numerous short sticks with magnets. These sticks can be assembled in a multitude of configurations to cater to diverse needs. For instance, a pair of sticks can be connected at the center to create a scissor-like motion, or multiple sticks can be assembled to replicate the form of a gun.
4.2.5 Employing Both Hands.
The above forms can be manipulated with a single hand, thereby enabling users to operate two controllers, one in each hand, to enable more operations. The left and right controllers can be identical or different in form. Despite that, P12 also mentioned that a long stick can be designed to be held with two hands, and controls and pressure sensors can be located on the stick according to the user’s grip.
5 DISCUSSION
5.1 Flexibility
We noticed that the cane form, basic stick, and thin stick can lie along a spectrum of movement flexibility. Cane form offers the least flexibility, as it only relies on the user’s elbow movements because the user’s hands can only pivot around the point where the cane meets the ground. Without this constraint, the basic stick allows for using both elbow and wrist movements. However, since the user has to hold the stick in their palm, they lose some finger flexibility. Thin sticks, which are a step further, can be maneuvered with the user’s fingers. Consequently, it is the most flexible form and could be used in scenarios that require more diverse movements, such as simulating sculpting or medical operations in XR.
5.2 Support
However, the cane form does not sacrifice flexibility for nothing in return. It provides the user with arm support. A common issue in XR interaction is the “Gorilla Arm” effect, where the user accumulates arm fatigue due to holding the arm in the air over time. Providing support is a plausible solution to address this issue. Beyond reducing arm fatigue, providing support can potentially stabilize movements, which can lead to more accurate operation. For instance, P6 proposed to use a cane design to rotate and scan 3D environments stably. However, if the user needs to do more than pivoting around the cane’s landing spot, such as lifting and swinging the stick to simulate a sword, it will cause more fatigue than short sticks.
On the other hand, the basic and thin stick could potentially offer the user support, too. For example, when we write on a table with a pen, the table supports our arm. Researchers also proposed designs where a thin stick’s tip is mounted to a robot-arm-like structure [1, 6, 7, 10, 13]. Although such designs were intended for better tracking or providing haptic feedback, we feel this could also be providing support to a user’s arm against gravity. However, when support is added, movements will be limited to some degree. When we draw on a table, it is a 2D platform that the pen is moving on. When we move with a robot arm, we only move where the arm allows us to move and with more friction than usual. It would be ideal if we could find an ergonomic form that provides enough flexibility while supporting the user’s arm. We encourage future researchers to consider this direction.
5.3 Controls
Participants commonly designed their buttons or controls for the thumb, index, and middle fingers. In particular, thumbs are given a larger operation area, such as a T-shaped space (P6) or a touchpad (P10). This is consistent with findings from [18], where the authors found these three fingers are considered much easier to perform a vast majority of gestures with than the ring and little fingers. In 4.2, we already illustrated the positions on each stick form where control buttons could be placed. In general, the thin stick has the fewest opportunities to put many buttons due to its size. Thus, for thin sticks, participants mainly designed tab or squeeze interactions like on the Apple pencil (P7, P11) or buttons along its body where the thumb might touch (P6). P7 stated that this form sacrificed control clarity for portability. Further, when the thumb, index, and middle fingers hold the thin stick, they are locked in place. As Serrano et al. revealed from handheld design [17], there is a trade-off between holding and interacting. Whereas in the basic stick or long stick forms, the stick is largely managed by the user’s palm and the two less flexible fingers, which frees up the thumb, index, and middle fingers to perform more intricate interactions on controls like on touchpads, thumbsticks, or sliders.
Besides receiving the user’s hand input, it is also important for an XR controller to interact with other objects as described in Section 4.1.4. Thus, tracking is important. Participants generally proposed the controller would need positional and rotational tracking (P4, 5, 8, 11, and 12) to locate itself in VR or the digital twin space in AR. It can also have sensors on the tip in order to detect events such as the cane hitting the floor (P3, P6) or the pen touching an object to get its color (P6). Moreover, XR could be more fun and powerful with ubiquitous computing, where stick controllers can contain sensors to detect smart objects and trigger reactions, which could be used in cases like tangible classrooms for kids or sports practice rooms.
6 CONCLUSION
This paper explored the use of stick-based design in XR TUI design. By studying the affordances of daily stick-based objects with a participatory design session, we presented a taxonomy for stick-based objects in terms of functionality, holding, movement, and object contact. We presented three types of stick-based XR controllers and discussed design considerations for these forms. This research contributes to systematically exploring the design space of stick-based TUI in XR. Future research can continue to build upon this work by addressing identified research gaps, such as leveraging the use of the cane form to mitigate arm fatigue in XR.
ACKNOWLEDGMENTS
The authors thank Dr. Brennan Jones for helping with our ideation, study framing, and giving feedback on our work. We also thank the participants who volunteered their time to join the user studies and the reviewers whose insightful comments and suggestions helped improve our paper.
Footnotes
⁎ Corresponding authors
- Footnote
Supplemental Material
Available for Download
- Jacopo Aleotti, Giorgio Micconi, and Stefano Caselli. [n. d.]. Programming manipulation tasks by demonstration in visuo-haptic augmented reality. In 2014 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE) Proceedings (2014-10). 13–18. https://doi.org/10.1109/HAVE.2014.6954324Google Scholar
Cross Ref
- Ian M Bullock, Joshua Z Zheng, Sara De La Rosa, Charlotte Guertler, and Aaron M Dollar. 2013. Grasp frequency and usage in daily household and machine shop tasks. IEEE transactions on haptics 6, 3 (2013), 296–308.Google ScholarDigital Library
- M.R. Cutkosky. [n. d.]. On grasp choice, grasp models, and the design of hands for manufacturing tasks. 5, 3 ([n. d.]), 269–279. https://doi.org/10.1109/70.34763Google ScholarCross Ref
- M.R. Cutkosky. 1989. On grasp choice, grasp models, and the design of hands for manufacturing tasks. IEEE Transactions on Robotics and Automation 5, 3 (1989), 269–279. https://doi.org/10.1109/70.34763Google ScholarCross Ref
- Thomas Feix, Javier Romero, Heinz-Bodo Schmiedmayer, Aaron M Dollar, and Danica Kragic. 2015. The grasp taxonomy of human grasp types. IEEE Transactions on human-machine systems 46, 1 (2015), 66–77.Google ScholarCross Ref
- Colin Gallacher, Arash Mohtat, Steve Ding, and József Kövecses. [n. d.]. Toward open-source portable haptic displays with visual-force-tactile feedback colocation. In 2016 IEEE Haptics Symposium (HAPTICS) (2016-04). 65–71. https://doi.org/10.1109/HAPTICS.2016.7463157 ISSN: 2324-7355.Google ScholarCross Ref
- Matthias Harders, Gerald Bianchi, Benjamin Knoerlein, and Gabor Szekely. 2009. Calibration, Registration, and Synchronization for High Precision Augmented Reality Haptics. IEEE Transactions on Visualization and Computer Graphics 15, 1 (2009), 138–149. https://doi.org/10.1109/TVCG.2008.63Google ScholarDigital Library
- Shuqi He, Haonan Yao, Luyan Jiang, Kaiwen Li, Nan Xiang, Yue Li, Hai-Ning Liang, and Lingyun Yu. 2024. Data Cubes in Hand: A Design Space of Tangible Cubes for Visualizing 3D Spatio-Temporal Data in Mixed Reality. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, 33 pages. https://doi.org/10.1145/3613904.3642740Google ScholarDigital Library
- Thea Iberall. 1987. The nature of human prehension: Three dextrous hands in one. In Proceedings. 1987 ieee international conference on robotics and automation, Vol. 4. IEEE, 396–401.Google ScholarCross Ref
- Thomas H Massie and J K Salisbury. [n. d.]. The PHANTOM Haptic Interface: A Device for Probing Virtual Objects. ([n. d.]).Google Scholar
- Meta. 2024. Meta Quest 2 Controller. https://www.meta.com/quest/accessories/quest-2-controllers/. Accessed: 1-Jan-2024.Google Scholar
- Diego Monteiro, Hai-Ning Liang, Xian Wang, Wenge Xu, and Huawei Tu. 2021. Design and Development of a Low-Cost Device for Weight and Center of Gravity Simulation in Virtual Reality. In Proceedings of the 2021 International Conference on Multimodal Interaction (Montréal, QC, Canada) (ICMI ’21). Association for Computing Machinery, New York, NY, USA, 453–460. https://doi.org/10.1145/3462244.3479907Google ScholarDigital Library
- T. Nojima, D. Sekiguchi, M. Inami, and S. Tachi. [n. d.]. The SmartTool: a system for augmented reality of haptics. In Proceedings IEEE Virtual Reality 2002 (2002-03). 67–72. https://doi.org/10.1109/VR.2002.996506 ISSN: 1087-8270.Google ScholarCross Ref
- Majken K. Rasmussen, Esben W. Pedersen, Marianne G. Petersen, and Kasper Hornbæk. 2012. Shape-Changing Interfaces: A Review of the Design Space and Open Research Questions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 735–744. https://doi.org/10.1145/2207676.2207781Google ScholarDigital Library
- Majken K. Rasmussen, Giovanni M. Troiano, Marianne G. Petersen, Jakob G. Simonsen, and Kasper Hornbæk. 2016. Sketching Shape-Changing Interfaces: Exploring Vocabulary, Metaphors Use, and Affordances. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 2740–2751. https://doi.org/10.1145/2858036.2858183Google ScholarDigital Library
- Anne Roudaut, Abhijit Karnik, Markus Löchtefeld, and Sriram Subramanian. 2013. Morphees: Toward High "Shape Resolution" in Self-Actuated Flexible Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 593–602. https://doi.org/10.1145/2470654.2470738Google ScholarDigital Library
- Marcos Serrano, Jolee Finch, Pourang Irani, Andrés Lucero, and Anne Roudaut. 2022. Mold-It: Understanding How Physical Shapes Affect Interaction with Handheld Freeform Devices. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 402, 14 pages. https://doi.org/10.1145/3491102.3502022Google ScholarDigital Library
- Adwait Sharma, Michael A. Hedderich, Divyanshu Bhardwaj, Bruno Fruchard, Jess McIntosh, Aditya Shekhar Nittala, Dietrich Klakow, Daniel Ashbrook, and Jürgen Steimle. [n. d.]. SoloFinger: Robust Microgestures while Grasping Everyday Objects. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2021-05-07) (CHI ’21). Association for Computing Machinery, 1–15. https://doi.org/10.1145/3411764.3445197Google ScholarDigital Library
- Jotaro Shigeyama, Takeru Hashimoto, Shigeo Yoshida, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2019. Transcalibur: A Weight Shifting Virtual Reality Controller for 2D Shape Rendering Based on Computational Perception Model. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3290605.3300241Google ScholarDigital Library
- Alexa F. Siu, Mike Sinclair, Robert Kovacs, Eyal Ofek, Christian Holz, and Edward Cutrell. 2020. Virtual Reality Without Vision: A Haptic and Auditory White Cane to Navigate Complex Virtual Worlds. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376353Google ScholarDigital Library
- Evan Strasnick, Christian Holz, Eyal Ofek, Mike Sinclair, and Hrvoje Benko. 2018. Haptic Links: Bimanual Haptics for Virtual Reality Using Variable Stiffness Actuation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3174218Google ScholarDigital Library
- Hsin-Ruey Tsai, Ching-Wen Hung, Tzu-Chun Wu, and Bing-Yu Chen. 2020. ElastOscillation: 3D Multilevel Force Feedback for Damped Oscillation on VR Controllers. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376408Google ScholarDigital Library
- Vive. 2024. VIVE Controller. https://www.vive.com/ca/accessory/controller/. Accessed: 1-Jan-2024.Google Scholar
- Xian Wang, Diego Monteiro, Lik-Hang Lee, Pan Hui, and Hai-Ning Liang. 2022. VibroWeight: Simulating Weight and Center of Gravity Changes of Objects in Virtual Reality for Enhanced Realism. In 2022 IEEE Haptics Symposium (HAPTICS). 1–7. https://doi.org/10.1109/HAPTICS52432.2022.9765609Google ScholarCross Ref
- André Zenner and Antonio Krüger. 2019. Drag:On: A Virtual Reality Controller Providing Haptic Feedback Based on Drag and Weight Shift. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300441Google ScholarDigital Library
- Yuhang Zhao, Cynthia L. Bennett, Hrvoje Benko, Edward Cutrell, Christian Holz, Meredith Ringel Morris, and Mike Sinclair. 2018. Enabling People with Visual Impairments to Navigate Virtual Reality with a Haptic and Auditory Cane Simulation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3173574.3173690Google ScholarDigital Library
- Joshua Z. Zheng, Sara De La Rosa, and Aaron M. Dollar. [n. d.]. An investigation of grasp type and frequency in daily household and machine shop tasks. In 2011 IEEE International Conference on Robotics and Automation (2011-05). 4169–4175. https://doi.org/10.1109/ICRA.2011.5980366 ISSN: 1050-4729.Google ScholarCross Ref
- Qian Zhou, Sarah Sykes, Sidney Fels, and Kenrick Kin. 2020. Gripmarks: Using Hand Grips to Transform In-Hand Objects into Mixed Reality Input. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3313831.3376313Google ScholarDigital Library
Index Terms
- Designing Stick-Based Extended Reality Controllers: A Participatory Approach
Recommendations
User Experience Guidelines for Designing HMD Extended Reality Applications
Human-Computer Interaction – INTERACT 2019AbstractWith the rise of Extended Reality (XR) technologies, such as head mounted displays (HMD) for Virtual Reality (VR), Mixed Reality (MR), and Augmented Reality (AR), designers are presented with many unique challenges and opportunities when creating ...
A Framework for Virtual Reality with Tangible Augmented Reality-Based User Interface
In this paper, we propose a framework for virtual reality, I2-NEXT, which enables users to interact with virtual objects by tangible objects in immersive networked virtual environment. The primary goal of this framework is to support rapid development ...
A brain-computer interface for extended reality interfaces
SIGGRAPH '17: ACM SIGGRAPH 2017 VR VillageExtended reality (XR) technologies, such as augmented reality (AR) and virtual reality (VR), remain limited in their interaction modalities. Prevailing interaction methods such as hand gestures and voice recognition prove awkward in XR environments, ...
Comments