Estimating Finger Joint Motions Based on the Relative Sliding of Layered Belts

This article introduces a new method for measuring finger joints’ motion based on the relative sliding motions of belts placed on the finger. Conventional mechanical methods used in wearable devices for measuring hand motions, such as a method using resistive sensors, require calibration for the difference in the fingers’ shapes of individuals. In contrast, the proposed method measures the metacarpophalangeal (MP), proximal interphalangeal (PIP), and distal interphalangeal (DIP) joints flex motion of a finger except for the thumb without the above individual calibration using the difference in the sliding amount of belts on the finger surface. The simple equations show that the relative sliding amount of belts is proportional to joint angles with a factor of the belt thickness. Therefore, the method only need to know the belts’ thickness beforehand to calculate the joint angles from the relative belt positions but not the fingers’ shapes. Experiments with a prototype showed that the proposed method estimated the joint motion of the finger model driven by servomotors with an accuracy of less than 1° in three kinds of motions. The device was in an early stage of development as a data glove, but the accuracies were comparable to other wearable devices. The layered belts mechanism proposed here can provide an option to design a device to realize simplified hand motion measurement by eliminating cumbersome calibration processing for each execution. Furthermore, the simple measurement principle also has an advantage in robustness to noise, such as the drift seen in the devices using inertial measurement units (IMUs).

H AND manipulation is an essential function for daily life, 23 and the hand is capable of the most subtle motions by 24 the human body. Capturing motions of the hand, especially 25 the fingers, is crucial to understanding hand manipulation, 26 evaluating individual functions for manipulation, and using 27 such information in applications. This has been a motivation 28 to the development of many devices for measuring human 29 finger motions, which include a simple goniometer to a com- range of fields requiring accurate hand motion, such as the 37 teleoperation of robot hands [4] and manipulation in virtual 38 reality (VR) [5]. A significant disadvantage of this type is 39 the complicated setup and requirement of space for cam-40 eras. Devices that use changes in an electromagnetic field to 41 measure hand motions have a simpler setup [6] but require 42 wired markers and restrict the environment to no metallic 43 objects around the measurement target. The demand for a 44 compact device with a simple setup has led to the develop-45 ment of hand-tracking techniques that use a single camera 46 [7], [8], depth camera [9], or stereo camera. The leap motion 47 controller system (UltraLeap Ltd., U.K.) is a successfully 48 commercialized device that uses a stereo camera for accurate 49 measurement of hand motions [10], and it has been adopted in 50 many fields, such as gaming in VR and rehabilitation [11] and 51 diagnosis [12] in medicine. However, it has a lower estimation 52 accuracy than motion capture systems [13], and the available 53 space for measurement is limited.

54
The second type is wearable devices, which include elec-55 tronic goniometers and data gloves. This type is advanta-56 geous, because it avoids spatial requirements or the occlusion 57 of people and objects to cameras. These advantages make 58 this type viable in various situations, such as rehabilitation 59 [14], [15], diagnosis [16], manipulation in VR [17], and 60 teleoperation of a robot [18]. Many researchers have proposed 61 one of the most well-known commercial products that use 69 this approach [19], but many other similar devices have been 70 developed as well [20], [21], [22]. Fiber-optic sensors are also 71 widely used in wearable devices for measuring hand motions, 72 because they offer flexibility and a lightweight [23], [24]. angles [25], [26], [27]. Alternatively, small inertial measure-83 ment units (IMUs) can be placed on each finger phalange to 84 measure the joint angles directly based on the relative motion 85 estimated by the IMUs [14], [28] The basic idea of the method and the mechanism to realize 104 it is the following. The proposed method uses a simple 105 mechanism consisting of layered belts placed along the finger 106 surface, as shown in Fig. 1(c). The shapes of the overlapping 107 regions of the belts are similar to each other in Euclidean 108 geometry, which means their scaling is different, but the shape 109 is the same. Furthermore, the outer curves of these regions 110 are congruent by uniform scaling, and the thicknesses of 111 the belts and the joint angle enable the calculation of the 112 difference in lengths of the curves. This relationship means 113 that the differences in lengths of the outer belt surfaces and 114 their thicknesses give the estimation of the joint angle. The 115 proposed method lets us estimate several joint angles by 116 overlapping an appropriate number of belts and fixing their 117 ends at appropriate points and realizes direct measurement of 118 joint angles with a simple mechanism, and it does not require 119 calibration for individuals.

120
The main aim of this article is to prove this new method 121 through validation experiments, introducing the proposed 122 method. Section II introduces the relationship between the 123 lengths of overlapped belts and joint angles in more detail. 124 Section III presents the experiment performed to validate 125 the proposed method, where the joint angles of a serial 126 link system were estimated. Section IV concludes this article 127 and discusses future works and potential applications of the 128 proposed method.

II. METHOD 130
The proposed method is based on our previous work on 131 estimating the rotational center of the lumbar spine and its 132 motion [33]. Here, how the proposed method measures finger 133 motions is introduced. The scope is limited to motions of the 134 finger's metacarpophalangeal (MP), proximal interphalangeal 135 (PIP), and distal interphalangeal (DIP) joints except for the 136 thumb within the sagittal plane (i.e., flexion and extension, not 137 adduction and abduction). However, the proposed method is 138 applicable to 3-D motions in principle. Fig. 2 shows a model 139 of the relationship between the motions of finger phalanges 140 and a belt placed on the finger. Only the distal and medial 141 phalanges are shown for simplicity, and the distal phalange 142 moves around the medial phalange. The followings are the 143 assumptions for simplification. 144 1) The belt thickness is constant during the finger's motion. 145 In reality, the thickness of the belt changes as it bends 146 to maintain its volume, although the amount of change 147 depends on the belt material. A mechanism to satisfy 148 this assumption is proposed in Section III.

149
2) The phalanges are rigid bodies. As further assumption, where r 1 and r 2 are the distances from the center of rotation to Substituting (1) and (2) into (3) and (4) gives Equation (5) where c is a constant of integration, θ is the joint angle 200 between the two phalanges, and l 1 and l 2 are the displacements 201 of the lower and upper wires, respectively. The advantage of 202 this method is that the joint angle between phalanges can 203 be measured directly if the belt thickness is known without 204 requiring any information about individual phalanges.
205 Equation (6) shows the calculation of a single joint angle 206 from the displacement of wires placed along the lower and 207 upper sides of a belt. This calculation means it is possible to 208 follow the joint angle while the joint moves by measuring the 209 change of wire displacements. The extension of this calcula-210 tion realizes the estimation of multiple joint angles during a 211 finger motion by overlapping belts on the finger and measuring 212 their relative sliding motions using wires placed on these belts. 213 Fig. 4 shows how the proposed method measures the following 214 finger joints: DIP, PIP, and MP. Let the angles of these joints be 215 θ 1 , θ 2 , and θ 3 , respectively. The mechanism of the proposed 216 method consists of three belts, and the bottommost belt is 217 fixed to the distal phalange. Let all belts have a constant 218 thickness d during finger motions. Two wires run along the 219 lower and upper surfaces of the belt from the belt ends.

220
The displacements of the lower and upper wires along the 221 metacarpal bone are represented by l 1 and l 2 , respectively.

222
The difference between these displacements is proportional to 223 the sum of the joint angles where c 1 depends on the initial joint angles and wire dis- following relationship with the belt thickness and joint angles: where c 2 also depends on the initial joint angles and wire wire. This results in the following relationship: where c 3 is a constant. Herein, c 1 , c 2 , and c 3 are zero if 241 the appropriate joint angles and wire displacements are set.
242 From (7)-(9) Equations (10)- (12) are rewritten using them as In the experiment to validate this method in Section III, 280 the above way to estimate joint angles from three wires' 281 displacements is adopted.

III. EXPERIMENT 283
The validation of the proposed method is conducted using 284 the prototype of the device and the serial joint link system with 285 servomotors instead of a finger for stable verification, as shown 286 in Fig. 5. Specifically, the validation aims to test whether the 287 relative displacements of the three belts enable us to estimate 288 the joint angles during finger motion and their accuracy. The 289 joint angles estimated from measured relative displacements of 290 the belts are compared with the joint angles directly measured 291 by the rotation angle sensors embedded in servomotors. The 292 estimation with the proposed system is applied to two different 293 shapes of links that imitated human finger bone shapes to 294 validate the proposed method is that unaffected by target finger 295 shapes.     with the finger surface by the fabrics smoothly sliding each 329 other. Fig. 7 shows the structure of a belt composing eight 330 nylon fabrics. The belt had a total thickness of 2.00 mm. The 331 structure of the belt realized a low bending elasticity to fit the 332 curved surface and a constant belt thickness during relative 333 sliding motions. These properties were essential to ensure that 334 the assumptions described in Section II held. As explained 335 in Sections I and II, the proposed mechanism consists of 336 three belts placed on the finger surface. In the experiment, 337 the end of the bottom belt was directly glued to the nail part 338 of the distal phalange model for stability, assuming it would 339 be firmly fixed to a fingertip or nail by a band or glue in 340 actual applications. The ends of the middle and top belts were 341 fixed to the belt underneath in the order explained in Fig. 4. 342 The other sides of the belts were allowed to shift as the belts 343 bent according to joint flexion. Two spandex fibers attached 344 to the top nylon fabric of each belt apply a tensile force to 345 the belt, respectively, as shown in Fig. 8. Additional nylon 346 fabric is placed under the bottom belt to place the sensor to 347 measure relative slides of the belts, and another pair of spandex 348 fibers also pull this fabric. Spandex fibers were adopted here 349 because of their nonlinear elasticity, such as rubber elasticity. 350 A spandex fiber shows the property that the increase in the 351 pulling force becomes moderate according to its extension. 352 This property is helpful to prevent the rise of pulling force 353 on the belts for finger flexion and avoid the problems on the 354 fixation of the belts on the fingertip or the problem in user 355 comfortability. The belts were made based on the size of the 356 middle finger model used in the experiment, but it is possible 357 to use it to measure the motion of another finger except for 358 the thumb if the end of each belt is on each phalange. Three 359 wires were attached to the top of the three belts, respectively. 360 A wire-in pulse coder (WP20-030-1S, LEVEX Corporation, 361 Japan) was used to measure the wire displacements caused 362 by the relative slidings of the belts. This tube-shaped sensor 363 measured the displacement of a magnetic wire in the tube 364 based on the change in the voltage waveform caused by the 365 coil's inductance. The stroke of the wire-pulse coder is 30 mm, 366 and its measurement resolution is 0.015 mm. From (16)- (18), 367 the ideal resolution of the joint angle estimation is 0.0075 • . 368 Furthermore, the dynamic response of the wire-pulse coder is 369 4 kHz. Therefore, the proposed system has the potential to 370 capture the motion at 30 • per second of each joint, although 371 the other factors, such as the friction between the belts, may 372 affect it. Independent amplifiers connected to the tube-shaped 373 Fig. 9. Schematic of the system used to capture the data obtained by the proposed device for joint angle estimation and joint link system. sensor by cables amplify the signals. Fig. 9 shows the whole 374 system to control the joint link system and obtain the amplified  triggers the motors of the joint link system to move, and the 418 computer starts to record the data from the A-D converter 419 simultaneously. The joint link system moves by setting the 420 appropriate target angles at 10 Hz to reach the target angles in 421 10 s. After 10 s, the joint link system stops and moves back 422 to the initial posture, where every joint angle is 0 • . During 423 the motion, the servomotors send the joint angles used as 424 the ground truth of the estimation to the laptop. They also 425 send the pulse signal simultaneously to the A-D converter to 426 match the ground truth with the estimated angles. The above 427 procedure was repeated ten times for each motion to evaluate 428 the repeatability and stability of the joint angle estimation by 429 checking the estimation variance among trials. The joint angles 430 estimated using (16)- (18) were evaluated by comparing them 431 with the ground truth based on the recorded pulse signal from 432 the microcontroller. 433 Finally, the system estimated the joint angles for the same 434 three motions with different shapes of the phalanx model. 435 One of the contributions of the proposed method is that the 436 estimation does not require calibration for the difference in the 437 finger shape of individuals, as explained in Sections I and II. 438 This experiment aims to show this insensitivity using the 439 different contact shapes between the belt and the bones with 440 the same measurement condition. The cover was attached to 441 the region of contact between the belt and the bones on the 442 phalanx model, as shown in Fig. 10. The wire displacements 443 and joint angles were recorded during the three joint motions 444 in the previous experiment and compared the estimated joint 445 and changing the friction. These tests will be future work to 467 make our proposed mechanism into an application.    Fig. 12 are the mean of ten trials. A slight shift can be seen 487 in the estimated values from their ground truth, especially 488 in the large joint angles. However, the proposed method 489 well estimated the difference in the joint angles during each 490 motion. Table I shows the RMSE of the estimated θ 1 , θ 2 , 491 and θ 3 . Motions 1-3 in Table I correspond to the motions in 492 Fig. 12(a)-(c), respectively. The estimation accuracy was less 493 than 1 • in RMSE for all joints and conditions. Table II shows 494 the mean of the standard deviation of the joint estimation 495 in each joint and motion among ten trials. The standard 496 deviations were slight among trials concerning the estimation 497 errors. Therefore, the measurement itself is stable, and it seems 498 the errors come from some mechanical problem, such as the 499 nonuniformity of the fabrics. Furthermore, the RMSE and the 500 mean of SD show that the error increases toward the tip of 501 the finger. It seems the frictional force between the fabrics 502 affects this increase of error in the estimation.   Table I correspond to the motions in Fig. 13(a)-(c), respec-510 tively, and Table I  The mechanism, which directly measures the angles them-548 selves, not velocity or acceleration, has an advantage in 549 stability and reliability. Some commercial devices that require 550 calibration to individual users show high accuracy and stability 551 even in a dynamic situation. For example, CyberGlove guar-552 antees measurement nonlinearity of less than 0.6% over the 553 joint range. Even though the proposed method demonstrated 554 only minor errors in the experiment, the error should be 555 minimized to be competitive with such commercial products. 556 The experimental errors can be attributed to the rigidity of 557 the mechanical components used to guide the belts and wires 558 and the elasticity of the belts. The nonlinearity observed in 559 the wire displacement indicates these problems. Adopting a 560 new belt material that is elastic in bending and that has high 561 tensile strength will help increase the measurement accuracy 562 and stability.

563
The potential of the proposed method also lies in the fact 564 that its principle of measuring joint angles is very different 565 from other methods. While an optical motion capture system 566 can estimate the joint angles with high accuracy, it usually 567 does not entirely match the actual joint angles, because the 568 markers on the finger slide without careful maker displacement 569 [3], [35]. In contrast, the proposed method directly measures 570 the relative angles between rigid bodies. Theoretically, this 571 means that the sliding motion of the device on the finger 572 should not affect the measurement accuracy. Therefore, the 573 proposed method is a strong option for designing a measure-574 ment device for hand motions.

575
In this study, the validity of the principle of measuring in 576 the proposed method was demonstrated, and the evaluation 577 was conducted only on the model of a human finger. As the 578 device attached to the finger model in Fig. 5, it can be attached 579 to a human hand by fixing the end of the device to the fingertip 580 and fixing the spandex fibers to the wrist. The sliding of the 581 belts on the finger and the sliding of wire-pulse coders on 582 the back of the hand realize the joint angle estimation as 583 the experiments in this article. However, there will be several 584 problems to apply our joint angle estimation mechanism to 585 practical application from the laboratory environment. First, 586 the belts must be stable on the finger and need to prevent it 587 from slipping off. For example, it is necessary to devise such 588 a way as making it into a glove shape. It is also necessary 589 to consider the influence of the finger's inward and outward 590 motion, which this article ignores. Ideally, these motion does shape must be a barrier for making it common technology.

619
Further work is needed to realize its practical application con-620 sidering these problems, but this study showed the possibility 621 of the calibration-free data glove. include multiagent robotic systems, embodied-brain systems science, 787 design support for large-scale production/material handling systems, and 788 human behavior analysis and support. 789 Dr. Ota was a recipient of the Fellowships from the Robotics Society 790 of Japan (RSJ) in 2016 and the Japan Society of Mechanical Engi-791 neers (JSME) in 2021.