Human to Robot Hand Motion Mapping Methods: Review and Classification

In this article, the variety of approaches proposed in the literature to address the problem of mapping human to robot hand motions are summarized and discussed. We particularly attempt to organize under macrocategories the great quantity of presented methods that are often difficult to be seen from a general point of view due to different fields of application, specific use of algorithms, terminology, and declared goals of the mappings. First, a brief historical overview is reported, in order to provide a look on the emergence of the human to robot hand mapping problem as a both conceptual and analytical challenge that is still open nowadays. Thereafter, the survey mainly focuses on a classification of modern mapping methods under the following six categories: direct joint, direct Cartesian, task-oriented, dimensionality reduction based, pose recognition based, and hybrid mappings. For each of these categories, the general view that associates the related reported studies is provided, and representative references are highlighted. Finally, a concluding discussion along with the authors' point of view regarding future desirable trends are reported.

Abstract-In this article, the variety of approaches proposed in the literature to address the problem of mapping human to robot hand motions are summarized and discussed. We particularly attempt to organize under macrocategories the great quantity of presented methods that are often difficult to be seen from a general point of view due to different fields of application, specific use of algorithms, terminology, and declared goals of the mappings. First, a brief historical overview is reported, in order to provide a look on the emergence of the human to robot hand mapping problem as a both conceptual and analytical challenge that is still open nowadays. Thereafter, the survey mainly focuses on a classification of modern mapping methods under the following six categories: direct joint, direct Cartesian, task-oriented, dimensionality reduction based, pose recognition based, and hybrid mappings. For each of these categories, the general view that associates the related reported studies is provided, and representative references are highlighted. Finally, a concluding discussion along with the authors' point of view regarding future desirable trends are reported.

I. INTRODUCTION
T HE employment of mapping algorithms in order to replicate human hand (HH) motions into robot hands (RHs) is frequent in several applications. The reasons lie in the advantages of introducing the human cognitive consciousness and physical dexterity in the control process, planning of actions, error supervision/recovery, and learning that can be delegated (totally or partially) to the human operator. The two principal fields in which human to robot hand mapping (HRM) methods are used are teleoperation and learning by demonstration [1].
In teleoperation, data measured from the operator's HH are used to control in real time the motion of an RH. RHs can present up to 25 degrees of freedom (DOF) (highly anthropomorphic artificial hand), making it difficult for the operator to generate commands for such a high dimensional input space. On the other side, the HH is a natural holder of multi-DOF and multifinger information to reproduce the operator's intent on the RH. The goal is to grasp objects, interact with the environment (e.g., pressing a button) or express intentions by means of gestures [2]. Examples of modern RH teleoperation include the following: robotic arms/hands in space shuttles and on the International Space Station, robots for inspection/repairing of underwater pipes and cables for oil and communication industries, medical robots for microsurgical interventions, wearable assistive robots for human physical training/rehabilitation, manipulation of hazardous materials in chemical and nuclear plants, and other applications including warehousing, agriculture, constructions, and mines [3], [4], [5], [6]. Teleoperation of RHs by HH mapping can apparently seem a straightforward and low-complexity solution to realize highly dexterous behaviors. Unfortunately, despite the advances achieved in the last years, it results from evidence to be a nontrivial problem, both in conceptual and analytical terms [7].
Differently, in learning by demonstration, data collected from the human operator are not used for an online control. Indeed, motion measurements from the HH are exploited as a source of human skill information, used to improve the dexterity and the behavior of autonomous RHs. It can, therefore, be seen as an advanced teaching technique of the robot by the human. Examples of typical applications exploiting learning by demonstration are preshaping and grasp planning for object gripping, optimization of postures for stabilizing manipulation, execution of RH transportation paths and execution of human-like movements [8], [9], [10], [11].
Let us introduce the reference terminology and concepts of the human to robot mapping problem illustrated in Fig. 1. The HRM represents the algorithm that associates available kinematic measurements (KM) of the HH to related kinematic commands (KC) for the RH. The KM are the output of the primary hand (PH) subsystem, which is given by the combination of the HH and the sensing equipment (SE) used to measure HH's motions. Thereafter, the KC generated by the HRM are taken as inputs by the target hand (TH) subsystem, which is composed of a hand controller (HC) translates the commands to motor's references for the RH.
From a general point of view, the root problem of HRM lies in the dissimilarities between the TH and the PH subsystems, since the real HH finger motions cannot be determined in practice. This comes from the difficulties in knowing realistic HH's physical quantities. Phalanx locations and lengths, and inconstant joint rotation centers differ between every operator This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ Fig. 1. General scheme of an HRM system (left) and nomenclature (right). and are de facto not measurable in vivo, except by invasive techniques not applicable for practical purposes (such as implantation of metal markers in the HH's bones followed by radioscopy, tomography, or magnetic resonance [12], [13]). In practice, only noninvasive techniques can be used, such as sensorized gloves, hand exoskeletons, and vision-based systems. This also implies that the HRM quality will be influenced by the limited KM information available from each specific SE. For example, for those approaches that directly estimate HH joint angles or fingertip positions, a model of the HH kinematics must be derived [14] from the KM, making the accuracy of the HH model of paramount importance. A particular consequence of these observations relates with the kinematic dissimilarities between the HH model and the RH geometry. If the RH presents a high level of anthropomorphism [15], then the HRM strategy will be less complex, on the other side, the mapping problem becomes more difficult and undetermined when the RH's level of anthropomorphism decreases. Indeed, in principle, if the HH model is endlessly accurate and the RH kinematics coincides with the one of the HH model, then the mapping will be the identity. However, in real situations, the RH will not have exactly the same kinematics as the HH. This means that, even when the HH model is highly accurate, HH motions cannot be directly replicated on the RH, and some adaptation has to be adopted [11].
It is difficult to design solutions that present predictable behaviors of the TH and are not limited to a small subset of motions/postures. Several works have attempted to define and guarantee a correct HRM in such terms. Two fundamental subproblems can be identified as follows: i) a meaningful and accurate measurement of the HH by the SE and ii) the design of an HRM algorithm in order to obtain a proper motion imitation by the TH. In this review, we will not focus on the former subproblem, which is addressed principally by sensorized gloves (e.g., [16], [17], [18], [19], [20]), hand exoskeletons (e.g., [21]), and vision systems (e.g., [9], [22], [23], [24], [25], [26]), but on the second subproblem that does not have a general solution yet. In particular, we review literature approaches for the HRM problem in terms of positional transformations. The design of the SE, devices for force feedback and the consideration of dynamics-related aspects are outside the scope of this survey. Furthermore, relatively recent approaches based on biological signals allow teleoperation by means of neural-consistent lowdimensional input subspaces, such as electromyography-based controls [27], [28], [29], but we will not focus on this kind of works/aspects. Note that the mapping problem arises also in the control of wearable supernumerary robotic fingers [30], [31], [32], although in this article, we put our focus specifically on RHs. Another important aspect regards also the mapping of the human arm, along with the HRM. Although a comprehensive literature review on human-to-robot arm mapping is outside the scope of this survey, we report, in the tables related to the different mapping methods, whether a mapping of the arm was realized in the considered work, and how this was performed. Indeed, for certain systems/applications, the HRM alone is not adequate, and also the arm needs to be mapped; at the same time, some HRM could be extended also for the mapping of the arm.
A great number of different approaches are reported in the literature to address the HRM problem. In such a prolific research branch, ways of approaching the problem, terminology, and a general view on the state-of-the-art are partially blurred. This is due to the overlapping of different applications, producing objective difficulties in identifying relevant studies and key scientific challenges. Therefore, for a better comprehension of the state-of-the-art and to foster a more homogeneous development of novel advances, in this review, we report an organic and systematic view on the techniques in the scientific literature related to HRM (see the Appendix for details on the review criteria). In the literature, the only survey attempting a comprehensive review of the HRM methods is the one by Li et al. [33]. However, such survey is affected by some lack of generality, due to a pure computer science point of view without a robotics approach to the rationalization of the literature. It is mainly biased on analysing computer vision based SE for teleoperation applications, disregarding important purposes of the HRM, like programming-by-demonstration and grasp planning. Also, mapping methods based on kinematic synergies and hybrid approaches are not considered. Furthermore, the mapping categories introduced by Li et al. [33] are based on technological aspects, introducing a terminology that is not general when an HRM is analyzed independently from the technological aspects (typically, multiple SE are appropriate for a given mapping). Differently, in this article, we provide a robotics overview on the problem, giving a general definition of an HRM, a deep analysis of the main scientific challenges, and a careful classification of the mapping method categories independently from technological aspects. Only then, we provide a discussion on SE, RH, and applications. Specifically, first, a brief temporal overview of the historical emergence of scientific challenges in HRM is reported. In this way, a complete picture, methodological, and temporal, of state-of-the-art and open challenges is given. Thereafter, articles are referenced and organized in the following six categories: 1) direct Cartesian mappings; 2) direct joint mappings; 3) task-oriented mappings; 4) dimensionality reduction based mappings; 5) pose recognition based mappings; 6) hybrid mappings. Each category is treated in a dedicated section in order to report the following: 1) a general view of the aspects common to the works of the same category; 2) the relevant aspects and differences that characterize specific studies; 3) a table containing representative articles of the category, summarizing typology of SE, typology of RH, and the metrics used to report results; 4) a table reporting qualitative comparisons; 5) an analysis of technological aspects and applications.

II. HISTORICAL EMERGENCE OF THE HRM PROBLEM IN SCIENTIFIC LITERATURE
This section provides an historical overview of the evolution of the founding scientific challenges of the HRM problem, which are not recent, instead they emerged along with pioneering investigations during the last century. Since such challenges are still relevant and unsolved nowadays, it is significant to have an overview on their historical evolution.
Early applications of the mapping of HH motions to robotic grasping devices can be found starting from the 1940s [3], [34], [35]. Nuclear scientists and engineers have been, from the first attempts, familiar with the transformation from human to RH motions in order to handle radioactive material [36]. Other remarkable historical applications are related to upper limb rehabilitation [37], and underwater [38] and space robotics [39], [40], dated from early 1950 s to late 1980 s.
The necessity of formal studies on the transformation from human to robot hand motions emerged along with the development of anthropomorphic RHs during the 1980s [41], [42], [43]. The HRM was formulated for the first time in 1992 by [44] as the problem of "transforming human hand positions to functionally equivalent robot hand positions, possibly preserving generality." In such work, the primary fundamental challenge of the HRM was identified in the presence of differences in the kinematic structures between PH and TH. Another important step in the historical evolution of HRM was the introduction of automatic goniometric measurements of the HH, which were previously performed only by manual usage of calipers with bony landmarks [43], [45], [46]. In 1989-1990, the virtual programming languages (VPL) DataGlove [47]) was used [48] reporting accuracies comparable with manual goniometric measurements. This gave birth to the study of analytical mapping methods. The first direct joint mapping (see Section III) was proposed in 1989, in early studies with the Utah/MIT Hand [49]. In 1990, a direct Cartesian mapping (see Section IV) was proposed for the first time by [50]. Another fundamental challenge for the HRM problem was identified by [43] as the mapping imprecision due to the computation of inverse kinematics in presence of incongruous workspaces between PH and TH. In 1993, a calibration/optimization process was employed for the first time [14], [51] stating its general necessity in order to improve the HRM performance. However, in the same year, other works [46], [52] demonstrated that this accuracy limitation remains predominant also by optimizing the portion of TH workspace safely reachable by PH poses.
A successive important development was the use of modern machine learning in 1998 [64] (other early nonanalytical approaches can be found in [65], [66], and [67]). Importantly, Fischer et al. [64] showed that the accuracy using machinelearning-based optimizations did not outperform previous techniques, and generalized the HRM scientific challenges to a modern formulation, stating that hand mappings to be successful should satisfy the following two requirements: i) the complete exploitation of the RH workspace to avoid dexterity limitations, and ii) the totality of PH finger poses having a realizable correspondence in the TH workspace. In particular, it was highlighted that optimization procedures do not fulfill these two requirements per se. Even more, the two requirements cannot be fulfilled in general, and therefore, the effort of research in HRM (as reviewed and classified in the following sections) is devoted to get around this primary difficulty that sum up with the other scientific challenges outlined earlier.

A. Description
A direct joint mapping is based on associating the values of PH joint angles-provided by the SE-directly to corresponding joints of the TH. This kind of HRM is possible when the kinematic structure of the RH allows some meaningful correspondence between joints of the PH and TH. Then, a linear relation between n selected PH and TH joints can be enforced as where θ PH i is the selected ith PH joint angle, θ TH i is the ith corresponding TH joint angle, and the constants k i and c i are parameters determined by means of: 1) empirical techniques; 2) optimization; 3) manual error and trial. A particular extension of the linear mapping (1), less used in more recent works, is the computation of the relationship between the selected PH and RH joints by means of least-square fit of m predefined (θ TH , θ PH ) couples. It is then possible to consider the matrix equation where H ∈ R m×n and R ∈ R m×n collect the m PH and TH predefined joint configurations, and K is the matrix computed by using the pseudoinverse of H, H + , as K = H + R. Then, K is used to map any generic PH joint configuration θ PH according to A block scheme of the direct joint mapping is shown in Fig. 2.

B. Literature Review
Several direct joint mappings simply implemented a one-toone mapping of the PH joint measurements onto the TH, without further processing or adjustment. The feasibility of one-to-one joint mapping were analyzed in [55] during the execution of gestures by the operator wearing a dataglove, demonstrating that state-of-the-art SE can introduce averaged angle errors up to around 7 degrees when anthropomorphic RH are remotely controlled. One-to-one mapping using a low-cost easy-to-replicate  dataglove was compared with the popular LeapMotion sensor in [63], the former showing a higher fidelity of hand pose sensing, tested on a virtual reality RH. Other approaches using low-cost datagloves were used to telecontrol five-fingered anthropomorphic RH, e.g., [60], [61], [62]. In [58], a one-toone joint mapping realized with markerless vision is assisted by a dataglove-based dataset of discretized HH motions. For particular precision requirements, as in the remarkable case of the Da Vinci robotic surgical telemanipulator, one-to-one joint mapping is accompanied by an exoskeleton-based SE, which exactly matches the low-complexity kinematics of the RH [57].
More elaborated approaches of direct joint mapping include empirical processing of PH joint measurements and optimization of HH models. The evaluation of three direct joint mapping methods is carried out in [53], considering the following: 1) dataglove's joint value adjustments in order to take into account sensor couplings during hand motions; 2) one-to-one mapping using marker-based vision as SE; 3) the usage of an HH model optimized by means of an iterative procedure with vision-based tracking of the HH fingertips.
A direct joint mapping was used for the Deutsches Zentrum für Luft-und Raumfahrt/Harbin Institute of Technology (DLR/HIT) II RH in [54], adjusting dataglove measurements by means of gains computed on the basis of HH calibration motions. As reported in [56], when the anthropomorphism level of the RH kinematics is pronounced, even little empirical adjustments of SE measurements can improve the transfer of human manipulation skills onto the TH. Other related works include the computation of a correction matrix for dataglove measurements based on index and thumb fingertips tracking [59], selection of a subset of joint measurements in order to match the number of TH joints [68], exploitation of the measurements of human phalanges relative orientations along with an HH model in order to estimate PH joints [69]. Table I reports a list of representative works using direct joint mappings.

C. Remarks
Direct joint mapping is the simplest realization of an HRM [49]. Indeed, simplicity represents a considerable advantage of this method, which can be rapidly implemented when fast HRM solutions are required. Another advantage is related to the fact that direct joint mapping preserves the PH shape on an anthropomorphic TH, allowing a better reproduction of HH gestures and increased predictability of the TH motions for the teleoperator. This means that the operator can more easily learn how to move her/his hand in order to compensate for inaccurate TH behaviors [43]. On the other hand, disadvantages are related to difficulties, for the operator, in reproducing wanted fingertip poses. This is due to the fact that kinematic differences between PH and TH do not allow direct joint mapping to preserve correctness and intuitiveness of the Cartesian fingertip positions. Furthermore, especially if the TH presents a nonanthropomorphic structure, an empirical selection of PH joints to be mapped is necessary, producing a loss of information. Typically, ad hoc adjustments include the following: 1) specific heuristics (i.e., loss of generality); 2) joint averaging (i.e., loss of functional information); 3) discarding of selected joints (i.e., underutilization of the input space). Nowadays, direct joint mapping is implemented for fast, simple, and inaccurate HRM solutions. See Table II, for qualitative evaluations of representative direct joint mappings.

A. Description
In direct Cartesian mappings, Cartesian positions and orientations of the TH fingertips (i.e., the fingertip poses) are imposed based on the fingertip poses of the PH. The latter are obtained by computation of forward kinematics given an HH model and joint angle measurements by the SE. Therefore, for each finger of the PH a forward kinematic function F(•) has to be considered for the definition of the fingertip pose p PH , resulting for the ith finger as a function of the n finger's joint angles. Thereafter, each PH fingertip pose p PH i needs to be represented in the TH workspace. Such latter "raw" fingertip poses may be then additionally processed in order to apply scaling, optimization, or ad hoc transformations based on specific design criteria. We refer to the processed TH fingertip poses as p TH . Finally, an inverse kinematic function I(•) is required to determine the joint angles for each TH finger obtaining the ith finger's vector of m joint angles The conceptual scheme of this mapping is reported in Fig. 3.

B. Literature Review
Different versions of direct Cartesian mappings have been proposed in the literature. In [70] and [71], the authors used a computer-vision-trained neural network to map the output of a dataglove on the DLR Hand II, focusing on the fingertip positions and neglecting the posture of the phalanges. Differently, other works used an explicit model of the HH in order to obtain Cartesian space information from SE measurements. In [72], fingertip poses are mapped on a virtual TH for ambulatory analysis purposes of the HH. In [73], PH fingertips are obtained using forward kinematics based on dataglove measurements, and then directly projected in the TH workspace for inverse kinematic calculation.
A distinct direct Cartesian mapping approach consists in scaling the workspace of the PH in order to increase the similarity with the TH workspace. The mapping algorithm proposed in [74] uses a simple scale factor to adapt the workspaces of the PH fingers, just before imposing the Cartesian poses on the TH. With a different perspective, the workspace scaling proposed by [75] for a five-fingered hand is realized by a combination of scaling  factors and 3-D roto-translations, resulting in the identification of a multidimensional linear transformation, which is performed using an automatic Jacobian-based iterative algorithm.
Another method adopted in some direct Cartesian mappings consist in processing the fingertip poses in the TH workspace using optimization procedures. In [76], PH fingertip poses are obtained via a marker-less vision, and, thereafter, a cost function based on the distance and orientation among fingertips in the TH workspace is designed in order to optimized the final Cartesian references. Nonlinear constrained optimization is applied in [77] for the Cartesian mapping of the thumb fingertip, with the aim of keeping the index and thumb last phalanx relative orientations within the limits of a (friction) cone surface.
On the other hand, other studies process fingertip poses focusing on the PH and TH number of fingers and their fingertip coordination. The tracking of PH index and thumb fingertips is realized using marker-based [78] and markerless [79] vision, and then used to control the closure motion of a two-fingered parallel gripper. In [80], a "virtual finger" concept is introduced in order to capture the PH ring-pinkie coordination and obtain a unique Cartesian reference for the middle fingertip of the AllegroHand (i.e., a RH composed of the thumb and only three fingers). Table III reports representative studies using direct Cartesian mapping.

C. Remarks
Direct Cartesian mapping is the simplest HRM when higher precision of TH fingertip positions is required. Indeed, its major advantage relates to the fact that PH Cartesian positions are directly imposed on the TH, which results as more appropriate for the execution of precision grasps and in-hand manipulation. Furthermore, scaling and optimization techniques can be easily applied to adapt the direct Cartesian mapping for hands of different sizes. Major disadvantages of direct Cartesian mapping are related to the fact that the shape of the PH is not preserved on the TH, affecting the execution of gestures, volar power grasps, and nonprehensile manipulation. Further limitations are related to size and shape differences between PH and TH workspaces, which means that some PH configurations will generate control references that are not achievable by the TH [43]. Finally, in case the operator needs to compensate with PH motions for some inaccuracy due to HH model and/or SE imprecisions, this may require contorted HH motions that are counterintuitive for the operator. For these reasons, practical realizations of the direct Cartesian mapping are affected by loss of generality and functional information. Further qualitative evaluations are summarized in Table IV for representative direct Cartesian mappings.

A. Description
A HRM approach that directly considers PH-TH kinematic dissimilarities is the task-oriented mapping, which consists in encapsulating motion information into a suitable task-oriented description. Such description is defined by a set of parameters that are independent of the hand kinematic structures, since they are defined in the task domain instead that in the kinematic domain. The conceptual block diagram of the task-oriented mapping is illustrated in Fig. 4. As reported in the figure, the PH motion information is encapsulated in an image description, in order to i) solve the mapping problem in the task domain (i.e., the image domain) and, thereafter, ii) apply the obtained result back to the TH in the kinematic domain. Besides this approach general description, which holds for any kind of taskoriented representation, specific realizations are discussed as follows.

B. Literature Review
Task-oriented mappings presented in the literature have been basically limited to applications in which the RH is used to perform object grasping. Therefore, they may be seen as "object-oriented" mappings, as a particular realization of the task-oriented paradigm. Particularly, grasping involving only RH fingertips have been mainly considered, dedicating less or no attention to, for example, gesture executions, keyboard typing or volar grasps.
One of the first uses of the task-oriented mapping concept can be found in [81] for learning by demonstration purposes. In such work, the surfaces of a parallelepiped-shaped object are identified by means of a vision system. Then, this information is used to encapsulate the measurements of an PH's dataglove into an object-specific geometric description, which is used to instruct a task planner for autonomous grasping actions for a 1-DOF two-fingered gripper. Subsequently, in the pioneering study of [82], the concept of virtual object has been introduced for the realization of a pinch grasping task-oriented mapping. Specifically, a fictitious circular object is considered to be held between the PH index and thumb fingertips, and exploited as task-oriented representation to capture their motion. Thereafter, the mapping for the TH is realized by means of an empirical nonlinear scaling of the virtual object size and centroid location. Finally, the scaled virtual object is used to impose fingertips poses on the TH and compute the joint angles for a planar two-fingered RH by means of inverse kinematics.
An extension of [82] from the planar to the 3-D case was presented in [83], [86], and [87]. In these works, the virtual circular object is used for task-oriented mapping of tripodal grasps (i.e., considering only PH thumb, index, and middle fingers) on a three-fingered RH. The reported results show that the approach produces a higher number of PH configurations that are actually reachable in the TH workspace with respect to the usage of direct Cartesian mapping. A further generalization of the virtual object method was proposed in [11] and [88], considering a spherical virtual object, instead of a circular one, whose radius and center vary dynamically on the basis of PH fingertip poses. In these works, it is also shown how the virtual object concept can be exploited to teleoperate RHs-in a taskoriented manner-independently from their specific kinematic structure (this was not completely achieved in [82], [83], [86], and [87]). Furthermore, it was demonstrated [90] that the approach proposed in [11] and [88] can even be abstracted from the definition of the virtual object shape itself. Specifically, the grasping task-oriented information is represented by a homogeneous transformation encapsulating the nonrigid deformation of predefined points of the PH fingers (e.g., PH fingertips in [90]). The same transformation is then imposed to related empirically selected reference points of the RH fingers (e.g., TH fingertips in [90]), and finally joint values are obtained by means of inverse kinematics. The virtual object approach was also demonstrated to guarantee stability of robotic grasps when used in conjunction with higher level passivity-based controllers [84]. Applications to bilateral telemanipulation can be found in [89] and [85].
Representative task-oriented mappings are reported in Table V.

C. Remarks
Many HRM involve the teleoperation of a nonanthropomorphic RH, making direct joint and direct Cartesian mappings particularly unsatisfactory. In such cases, a remarkable advantage of task-oriented mappings consists in reducing the control complexity while allowing the usage of a wide range of different TH. De facto, this enables HRM applications characterized by reduced expenses for the RH fabrication, by virtue of higher simplicity and mechatronic reliability. On the other hand, disadvantages of task-oriented mappings mainly rely on the general intuitiveness of the teleoperation and predictability of the TH motions from the operator point of view. Indeed, although task-oriented mappings allow some sort of abstraction of the HRM from the PH and TH kinematics, this is true only in the sense of the considered task-oriented view of the mapping problem. Then, even for slightly different purposes, their applicability can likely result very low. Thus, such aspects still require more investigations. See Table VI for representative task-oriented mappings qualitative evaluations.

A. Description
In dimensionality reduction based mappings, some kind of coordination for the variables of the TH input space is introduced. In this way, a dimensionality reduction is enforced by encoding the PH motions into a related subspace, and, thereafter, decoding such low-dimensional information into TH motion inputs. A conceptual block scheme is reported in Fig. 5. Dimensionality reduction-based mappings were exploited by several works in the literature to foster effective real-time HRM for both fully actuated and underactuated RH.

B. Literature Review
The most common realization of dimensionality reductionbased mappings is performed by observing how humans grasp objects, and tracking HH motions in order to formally describe HH joint correlations. This was done in the pioneering work by Santello et al. [96], which introduced the so called hand postural synergies, defining the low-dimensional subspace used by humans to control their hands. In other studies, the low-dimensional subspace was instead computed by following empirical rules in order to match specific requirements, and/or enforce the encoding/decoding of specifically identified PH motions. Indeed, dimensionality reduction-based mappings were exploited in the literature for a wide range of purposes, such as the following: 1) teleoperation; 2) synthesis of specific control systems; 3) planning for automatic grasping tasks; 4) human-inspired RH mechanical designs. Examples of these applications are reviewed as follows.
In [97] and [98], the authors derive a continuous subspace of the TH configuration space thanks to an analysis of HH motions during grasping (explicitly inspired on the concept of postural synergies as introduced in [96]). The reported results show the ability of performing autonomous grasp planning with different kinds of RH-either anthropomorphic or not-successfully avoiding unfeasible configurations and collisions during operations in cluttered environments. In an extension of these works [99], the grasp planner, using dimensionality reduction-based mapping, integrates also a teleoperation-aided definition of the relative orientation between RH and objects. Other relevant works adopted dimensionality reduction based mappings exploiting postural synergies to inform planning and control during grasping actions with a fully actuated RH [100], [101]. In [102], a successful extension to an underactuated anthropomorphic RH of the mapping proposed in [100] is presented. A review on strategies for transferring HH postural synergies to RH is reported in [103].
A different concept is proposed in [10] and [111], where the subspace for the dimensionality reduction based mapping is computed in accordance to given task-oriented specifications. In such works, the authors exploit the concept of principal motion directions that can be seen, simplifying, as the task-specific counterpart of the postural synergies of Santello et al. [96]. The principal motion directions are computed directly in the TH joint space, and then they are used for the implementation of a grasping planner, also including collision avoidance [112]. Interestingly, in [113], an explicit comparison is reported between "generic" postural synergies and "task-specific" principal motion directions, showing that they produce comparable performances, therefore demonstrating the suitability of the principal motion direction approach. Other works define subspaces for dimensionality reduction based mappings with respect to other arbitrary criteria, e.g., [19], [114], [115].
A direct use of dimensionality reduction-based mappings for teleoperation can be found in [91] and [95], where Cartesian space-based synergies are used to map unconstrained PH motions on the TH. A continuous teleoperation subspace is derived by means of empirical considerations in [7]. Simpler, completely ad hoc solutions for the implementation of dimensionality reduction for teleoperation are explored for two-dimensional [94] and one-dimensional [29], [92], [93] TH input spaces. Furthermore, HRM-based on dimensionality reduction were also proved to be particularly effective to foster natural control with robotic prosthetic hands [116] and bio-signal based interfaces for advanced RH teleoperation [117], [118], [119], [120]. Table VII reports representative works using dimensionality reductionbased mappings.

C. Remarks
Several applications involving RH can benefit from the availability of a reduced complexity HRM. In this relation, dimensionality reduction-based mappings allow us to define a subdimensional teleoperation space with the double advantage of dealing with both problems of i) high dimensionality of the input space for anthropomorphic RH and ii) pronounced kinematic differences for nonanthropomorphic RH. Indeed, such advantage makes dimensionality reduction based mappings very suitable to transfer human skills to RH autonomous grasping planners. In contrast, disadvantages are related to fact that the lower complexity of the HRM is obtained at the cost of limiting the possible motions of the TH to the ones that can be generated according to the defined mapping subspace. This, in addition to the reduced number of possible TH configurations, may require a non-negligible learning effort of the mapping by the operator. Learning rates, and therefore, the related intuitiveness of the HRM, require additional investigations and generalizations. Qualitative evaluations of representative dimensionality reduction-based mappings are compactly reported in Table VIII.

A. Description
A multifunctional HRM can be realized by means of the hand posture recognition-based mapping. In this category of mapping methods, the behavior of the TH is not continuous, yet is restricted to a certain number of predefined configurations/grasps/motions. That is, a discrete set of distinct actions is available for the teleoperation of the TH. A conceptual block scheme is reported in Fig. 6. Looking at the figure, the motion information coming from the PH is processed to compute relevant data features, and then used to recognize specific PH gesture/grasp types, i.e., specific hand postures. Thereafter, based on the output of this recognition process, one of the predefined TH motions is enabled and executed. Different specific implementations of the hand posture recognition process have been tested, mostly based on machine learning techniques.

B. Literature Review
Several works used hand posture recognition-based mappings to realize teaching by demonstration systems. The aim is to obtain a natural programming of the RH, exploiting different PH gestures/grasps classification techniques. In [1], a hierarchical neural network is used to recognize precision versus power grasps from dataglove measurements. In [121], a support vector machine based classification of PH grasp postures allows the programming of assembly tasks performed by a TH. In [122], the measurements of an SE composed by a markerless vision system and a dataglove are used to recognize PH grasp postures by means of a decision tree and, consequently, to execute predefined force-controlled TH grasps. HRM for teaching by demonstration purposes based on nonparametric learning techniques can be found in [9], [123], and [124]. In such works, k-nearest neighbourhood algorithms are used to classify PH postures for the activation of TH grasping patterns.
Hand posture recognition based mapping is largely exploited also for teleoperation applications. In [106], a shallow neural network is trained for the online recognition of PH gestures for the activation on-the-fly of predefined TH grasps. Hidden Markov models are used to identify PH grasp sequences in [8], along with the support of a neural network for choosing between predefined TH configurations. More recently, deep learning techniques were applied for the PH posture recognition [104], [125] using markerless vision data.
Other uses of hand posture recognition based mappings for teleoperation were based on Gaussian mixture models [107], [109] and Bayesian networks [26]. Interestingly, in [108], the measurements from a dataglove-based SE are exploited to recognize the object to be grasped, and then a predefined grasping motion of the TH is activated on the basis of a compatibility index. Differently, in [110] and [105], the result of the PH posture recognition based on machine learning is used to drive the transitions of finite state machines in order to identify the action to be executed by the TH. Representative hand posture recognition mappings are reported in Table IX.

C. Remarks
The main advantage of the hand posture recognition based mapping is related to its use in applications characterized by a known set of requested TH actions. In this case, the behavior of the TH can be discretized, and the HH postures used to activate specific predefined and precise configurations/motions. In this way, the operator can learn how to modulate the PH by her/his own hand, as a sort of advanced remote controller. Furthermore, another advantage is that the mapping method can be easily implemented in an incremental fashion, i.e., new functions can be added when necessary. The principal disadvantages of hand posture recognition based mapping are i) the limited possibility of continuous control of the TH, and (ii) the fact that the number of TH predefined actions increases as much as the complexity of the application increases, which can make the recognition of correct PH postures more and more difficult. Qualitative evaluations are listed in Table X for representative hand posture recognition-based mappings.

A. Description
HRM methods that do not directly belong to one of the previous categories are here considered, referred to as hybrid mappings. HRM of this kind can either consist of the following: 1) relevant ad hoc solutions; 2) implementations that can be described as a merging of previously mentioned mapping methods along with slight modifications/adaptations; 3) explicit hybrid combinations of other mapping approaches, with adjoining definition of related transitions during the teleoperation. For these mappings, we do not provide, by their heterogeneous nature, a conceptual block scheme.

B. Literature Review
In [126], a customized mapping is proposed for teleoperation of an ambidextrous robot for space applications. Specifically, a voice command recognition is implemented to instruct "primitive tasks" to the TH and, thereafter, the teleoperation switches to a continuous HRM in which a dataglove modulates the opening/closing motion of TH preselected grasps by means of a dimensionality reduction-based mapping (see Section VI).
In the approach presented in [127], first, markerless vision is used to observe PH and impose a predefined grasp to the TH, second, the same SE is used to estimate the distance between PH thumb and index fingertips in order to map opening/closing motions. A different ad hoc HRM is introduced in [128], referred to as "hidden robot concept." In such approach, the TH and its scenario are hidden to the teleoperator by means of a virtualized intermediate representation. In this way, the HRM problem is split in the subproblems of determining a suitable intermediate representation and, thereafter, translate the obtained virtual behavior for the real TH.
Other kinds of hybrid mappings embrace the concept of shared control [129]. In [130], the implementation of a hand posture recognition based mapping (see Section VII) is combined with an autonomous "fine tuning" performed by the TH to improve grasp stability. With a similar hybrid structure, in [131], the hand posture recognition based mapping is used for TH preshape selection, and autonomous grasping motions are executed when the RH palm touches an object, detected by means of a tactile sensor. Differently, in [132], the possibility of switching between a shared control in which the operator only selects the TH shape for an autonomous grasping motion and a direct joint mapping (see Section III) is provided. In [133], a shared control mapping is proposed in order to allow the operator to command the TH motion that determines the orientation of a remote grasped object. A number of works combine direct joint and Cartesian mappings (see Sections III and IV) with the goal of preserving, on the TH, PH shape and fingertip positions within a single HRM. In the hybrid mapping presented in [134], a transition between direct joint mapping and predefined TH precision grasps is enforced by means of a fuzzy-based classifier. Such approach is also used for a dual arm system in [135], but with an additional integration of a collision avoidance algorithm. In [136], a classifier uses measurements from a dataglove SE to discriminate if a precision grasp is being performed by the PH, and in that case, the HRM switches from direct joint to direct Cartesian mapping. Differently, in [137], the transition between direct joint and direct Cartesian mappings is realized analytically and continuously, in an independent fashion for each TH finger. In particular, the distance between PH thumb and finger fingertips is exploited to allow a smooth sigmoidal spatial transition between the TH motions obtained from direct joint and direct Cartesian mappings.
A list of representative works implementing hybrid mappings is shown in Table XI.

C. Remarks
The approach to the HRM problem based on combining different methods represents a promising road to attempt overcome current limitations. Therefore, the advantage of hybrid mappings lies in the possibility of exploiting specific approaches to solve multiple RH teleoperation subproblems, for example, following criteria of modularity and/or merging within the hybrid mapping structure. Of course, the algorithm design complexity increases, and transition behaviors and/or redefinitions of mapping strategies available in the literature are delicate operations that can be considered as a disadvantage of this kind of HRM. Qualitative evaluations of representative hybrid mappings can be found in Table XII.

IX. DISCUSSION: APPLICATIONS AND DESIRABLE TREND
In the previous sections, we provided a review of the methods presented in the literature for the problem of HRM. The existing approaches have been reviewed and classified, under general and terminology-consistent criteria, into six mapping  categories. In this section, further discussion is reported, especially analysing appropriateness aspects for different utilizations of the mapping methods and outlining desirable trends for future developments.

A. Technological Aspects and Applications
In the following, we will discuss the distribution of real applications and SE/RH types emerging from the considered works, with particular reference to two figures reporting aggregated results over the different mapping approaches. In Fig. 7, the main SE and RH types of HRM are listed, and their combined usage in the different mapping methods is reported. It must be highlighted that the results in Fig. 7 were obtained considering that a single work can contain multiple combinations. In Fig. 8, the principal applications of HRM are listed, among which: "motion evaluation" means that a mapping method has been applied just to evaluate the performance of TH motions with respect to specific evaluation criteria based or not on HH motions/user intent; the terminologies "simple power grasping" and "simple precision grasping" refer to the fact that power/precision grasps have been applied on standard shapes, such as spheres and cuboids, without using real world objects; "DLO grasping" stays for "daily living object grasping"; and the remaining applications are self-explanatory. In this case, a single work can also be related to multiple applications. The "motion evaluation" is the most frequent application because it is very common for evaluating the mappings in scientific studies, therefore, in the following, we will focus more on the other reported applications.
1) Direct Joint Mapping: 1) Technological aspects (cf. Fig. 7): Direct joint mapping is the most obvious and used mapping method, and is implemented mostly using datagloves for the control of anthropomorphic RH. The reason is attributable to the fact that HH joint angle values are easily obtained from datagloves. Vision systems are also quite usual SE as far as adequate implementation/processing is adopted to obtain HH joint angle measurements, and the necessity of a grounded sensory system is acceptable. 2) Applications (cf. Fig. 8): Direct joint mapping is consistently used in applications regarding the execution of simple power grasps, where lower fingertip position accuracy is acceptable in exchange for greater intuitiveness and HH finger shape preservation. However, in applications demanding for high reliability standards, the usage of a minimal complexity RH (two-fingered single joint gripper) with a hand exoskeleton as SE can be adopted for matching the required specifications (such as surgery, refer also to Fig. 7). 2) Direct Cartesian Mapping: 1) Technological aspects (cf. Fig. 7): The pair "dataglove SE-anthropomorphic RH" is the most adopted solution also for direct Cartesian mappings. Indeed, the availability of HH joint angles allows an immediate computation of forward kinematics to obtain PH fingertip poses that can be more intuitively mapped on anthropomorphic grasping devices. On the other hand, vision systems provide direct measurement of PH fingertip positions, and also are a very appropriate solution for precision grasping (especially marker-based vision) with both three-fingered grippers or anthropomorphic RH. 2) Applications (cf. Fig. 8): Since direct Cartesian mapping does not conserve PH finger shapes due to the PH-TH kinematic dissimilarities issue, it has been mainly adopted for testing the grasping of DLO (in which an intuitivenessprecision tradeoff can be desirable for several grasps/tasks, e.g., holding a cup from its handle, pressing keyboard buttons, grab a pencil), pick-and-place applications, and simple precision grasps.

3) Task-Oriented Mapping:
1) Technological aspects (cf. Fig. 7): For the implementation of task-oriented mappings, the usage of datagloves as SE is mainly preferred. Indeed, the aim is to have HH measurements for easily computing forward kinematics, on which obtain RH-type-independent descriptions, to generalize the mapping for multiple robotic devices. Accordingly, task-oriented mapping has been employed with different types of RH, much with two-and three-fingered grippers but also with anthropomorphic RH. 2) Applications (cf. Fig. 8): Task-oriented mapping has been demonstrated to be suitable for assembly applications and simple precision grasping. In such kind of tasks, objectoriented approaches are highly useful for mapping of HH to nonanthropomorphic RH. However, it is necessary to highlight that some aspects related to the TH motion predictability and intuitiveness for the operator still need to be properly addressed. Indeed, the performance evaluation of task-oriented mappings is mainly limited to motion evaluations (i.e., mostly relegated to academic research).

4) Dimensionality Reduction Based Mapping:
1) Technological aspects (cf. Fig. 7): The utilization of dimensionality reduction based mapping is mostly used with anthropomorphic and three-fingered RH. Being an approach that aims at simplifying the control complexity, it can be seen as a "conceptual" mapping that can be applied to RH with different kinematic structures. Anyway, more applications have been realized using anthropomorphic RH, whereas different types of SE have been adopted. Either joint or Cartesian spaces can be used for computing general human postural synergies or task-specific motion directions, for which reason different kinds of SE can be adequate based on the particular requirements. 2) Applications (cf. Fig. 8): Dimensionality reduction-based mapping has been mainly applied for power grasping tasks and DLO grasping (e.g., for prosthetic applications). This kind of mapping approach is also a very used solution for planning pick-and-place and tool use tasks. However, the predictability of the motions produced on the TH should be additionally investigated.

5) Hand Posture Recognition Based Mapping:
1) Technological aspects (cf. Fig. 7): The hand posture recognition based mapping is used for diverse combinations of SE and RH types. Indeed, note that: i) the pose recognition is often performed using machine learning, which can work with inputs from different SE, and ii) the realization of discrete, predetermined sets of postures/motions can be obtained on different RH without critical problems. 2) Applications (cf. Fig. 8): Typically, the operator is required to learn how to configure her/his HH in order to activate the correct predefined behavior of the TH. Also nonverbal communication or virtual reality applications are common, in which grasps are not a primary goal, but hand posture recognition mapping have been successfully used also for grasping activities, such as simple precision grasping and pick-and-place tasks. However, an evident limitation is related to the fact that only a discrete number of motions/actions/functionalities is controllable, normally with a very little possibility of continuous control. As a matter of fact, this bounds the level of naturalness of the HRM. 6) Hybrid Mapping: 1) Technological aspects (cf. Fig. 7): Hybrid mapping methods have mostly been developed for utilization with anthropomorphic RH, since they attempt at solving the complexity of the hand mapping on multiarticulated RH by merging different methods. Datagloves have been the most used SE type, but also hand exoskeletons and vision systems have been used for hybrid mapping based on requirements. 2) Applications (cf. Fig. 8): This method have been used in a very different kinds of applications. The more explored are grasping applications related to assembly and tool use, which are very challenging tasks in both daily living and industrial contexts. The realization of a structured framework including multiple HRM methods can hold a great potential for a wide range of applications.
Of course, this introduces an additional research question, that is, how to successfully combine such different methods, in terms of logic, temporal, and spatial transitions. More investigation is surely needed in this direction.

B. Desirable Trends
In the following some desirable future trends/directions of research for new developments and improvements in the problem of HRM are discussed.
1) Objectives and Metrics: First of all, a more systematic definition of the objectives of the mapping methods is needed. This is very important because it would help different works focusing on the same unresolved issues, in a manner such that they can "speak the same language" in defining common goals. At the same time, also a better understanding of the meaning and value of already presented results would be obtained, with clear advantages. Indeed, it would help to understand common research directions for incremental advances in the HRM problem. This should be also accompanied by the definition of standard evaluation metrics, which until today have been mostly based on qualitative evaluations in the great majority of studies, hindering a systematic and fruitful comparison among different approaches.
2) Naturalness and Intuitiveness: Two specific aspects that should be considered for novel developments are the naturalness and intuitiveness of the mappings from the user point of view. Unfortunately, these concepts are basically disregarded in state-of-the-art solutions. Indeed, they still need to be formally defined even when they are undoubtedly important for the overall mapping performance. A formal definition and measuring of HRM intuitiveness and naturalness would certainly guide to novel points of view in the method designs and specification of evaluation metrics. The naturalness should concern the operator's ability-given a certain mapping-in executing motions on the TH in the same way as it happens with her/his own hand. Differently, intuitiveness should be related to the operator's learning rate of the functionalities provided by a given mapping, which, for instance, should not surpass a certain required threshold. Studies in these directions can be very useful for novel perspectives and advances in HRM.
3) A Priori Kinematic Information: Another point, we highlight for novel developments in HRM is the exploitation of a priori information related to the hand kinematic structures. Indeed, this aspect is not considered in the approaches available in the literature. With a priori information we mean that, once a certain TH/RH kinematics is modeled/given, there are several considerations that are readily available (or immediately computable) such as: the shape of the hand/fingers workspace, velocity ellipsoids [138], workspace regions in which thumb and fingers can get in contact, and position and orientation of the finger bases with respect to the palm. These are just some of the aspects available "for free" from a priori kinematic information that still need to be investigated and fully exploited for mapping purposes. They can give useful insights for the development of more meaningful methods, and also for generalization on the basis of a both geometrical and topological (or, with a more anthropocentric terminology, anatomical) knowledge, which is definable and available for all hands.

4) Feedback of Mapping Information:
A novel feature that could be introduced in HRM methods is a feedback of specific mapping parameters/descriptors, which should be provided to the operator during online usage. This would allow a closed-loop control and learning of the mapping functionalities, since the user would be aware-by means of the feedback-of the evolution of such parameters/descriptors. In other words, the mapping should be interactive, and the HRM performance would not be defined anymore by the sole mapping characteristics, but also by the user-mapping coadaptation by means of the feedback information (i.e., the user communicates information to the mapping algorithm via HH motions, and the mapping communicate information to the user by means of the feedback). To give an example, the feedback interface could be used to inform the operator about unsafe HH motions. The user would be informed, during the execution of HH motions, about the "distance" from unsafe HH configurations that do not have a direct correspondence in the TH workspace. In such a way, unpredictable TH motions due to workspaces incongruity can be mitigated by the user herself exploiting the feedback information. In practice, the feedback can be generated by using one of the wearable human-robot interfacing modalities extensively studied in the literature, such as auditory [139], electrical [140], visual [141], or mechano-haptic stimulations [142].

5) Incrementally Updatable HRM:
An interesting approach to face the mapping of HH motions onto a kinematically dissimilar RH would be to develop mapping algorithms updatable to novel functionalities. That is, not to approach the problem with the aim of developing a monolithic mapping algorithm for being used in all possible situations, but, instead, to make the algorithm easily updatable whenever requested by the user. The best framework for implementing this kind of behavior is probably provided by the incremental learning paradigm [143], [144], i.e., the machine learning approach in which input data can be used to update the existing algorithm capabilities. To give a more practical example, a user could provide to an incremental machine learning algorithm some training data specifically related to precision grasp capabilities (necessity of preserving precision of the fingertip positioning), because such capability is needed in the short term. If subsequently, another kind of capability is needed, let us say index pointing (necessity of preserving finger shapes), new training data will be provided to the incremental learning algorithm to update its functionalities to the novel objectives. Note that this kind of incremental approach could also be used to make mapping algorithms adaptable to different kinds of RH.

APPENDIX REVIEW CRITERIA
The literature search at the base of this review work was conducted in accordance with criteria and guidelines outlined by the preferred reporting items for systematic reviews and meta-analyses (PRISMA) approach [145]. In this relation, and looking at the flowchart of the literature search process reported in Fig. 9, the following four stages were carried out as follows: 1) identification of the articles (identification stage); 2) screening of the acceptable papers (screening stage); 3) selection of relevant articles (eligibility stage); 4) inclusion of relevant articles in the present review work (inclusion stage).  At the identification stage, the literature search was performed from the electronic databases [146] IEEE Xplore, Web of Science, ScienceDirect, ACM Digital Library, Scopus, Springer-Link, and Google Scholar. All these databases support the usage of advanced operators, and therefore, the key search terms were selected and combined as follows (for clarity, operators are bolded and key terms are in italic): [("robot" OR "robotic" OR "multifingered" OR "dexterous" OR "prosthetic" OR "artificial" OR "human") AND "hand"] AND [("teleoperation" OR "telemanipulation") OR ("human skill transfer" OR "human skill imitation" OR "teaching by demonstration" OR "programming by demonstration" OR "learning from demonstration" or "human-like") OR ("virtual reality" OR "interface" OR "interfacing")] OR [("grasp" OR "hand" OR "motion" OR "human to robot" OR "kinematic") AND ("map" OR "mapping")] OR ["hand" AND ("exoskeleton" OR "motion tracking")] OR [("sensorized" OR "data") AND "glove"] where the operator "" (i.e., the quotes operator) allows exactmatch search, and the operators OR and AND allow the application of boolean rules. In addition to the search with keywords and operators, we also identified other articles by manual search of conference and journal papers, book chapters, and the bibliography of the identified articles themselves, in order to ensure coverage and avoid biases of automatic-only search. At the screening stage, 12 900 articles were screened in order to remove duplicates and grey literature, resulting in 411 articles selected. At the eligibility stage, we applied specific inclusion and exclusion criteria for the selection of relevant articles, as listed in Table XIII. As a result, 327 articles were excluded. Therefore, as reported at the Inclusion Stage of Fig. 9, a total of 84 articles were included and discussed, reviewed, and classified in the present survey.