Novel Kinect-Based Method to Assess 3D Reachable Workspace in Musculoskeletal Shoulder Dysfunctions: Case Reports

Objective: To demonstrate the feasibility and usefulness of applying Kinect sensor-based 3D reachable workspace assessment in four cases of musculoskeletal shoulder dysfunction. Methods: Reachable workspace assessments were performed using a Kinect sensor on 4 individuals with various shoulder dysfunctions (shoulder impingement syndrome, frozen shoulder, chronic shoulder dislocation, and post-surgical repair of supraspinatus tear). Upper extremity active range of motion was captured by the Kinect sensor and reconstructed to provide an intuitive graphical representation of each individual’s 3D reachable workspace. For tracking each individual’s progress during rehabilitation, both total and quadrant reachable workspace relative surface areas (RSA) were serially monitored over varying follow-up periods spanning 92 to 197 days. Results: The newly-developed Kinect sensor-based reachable workspace assessment was capable of measuring changes in range of motion (ROM) of individuals with musculoskeletal shoulder dysfunctions over time, reflecting improvement in 3 out of 4 presented cases. Conclusion: The demonstration cases indicate that the newly-developed sensor-based 3D reachable workspace analysis could be beneficial in clinical evaluation and during rehabilitation of various upper extremity musculoskeletal conditions. The 3D reachable workspace analysis provides a more intuitive and useful quantitative global upper extremity functional outcome measure and a means to better engage the patients through visualization of individualized reachable workspace to track their progress during rehabilitation.


Introduction
Musculoskeletal upper extremity dysfunctions are common and can have significant impact on an individual's daily life, occupation, and quality of life. In the most recent census report, nearly 20 million Americans out of the total U.S. population of 303.9 million reported upper extremity disability [1]. Of those, approximately 12.2 million people reported difficulty reaching objects overhead and 17.2 million were reported to have difficulty lifting a 10-pound object.
The proximal upper extremity and shoulder dysfunction has a multifactorial etiology depending on age, risk factors, occupation, activities of daily living (ADLs) and trauma including sports related injuries [2]. Clinical evaluation and monitoring of these shoulder dysfunctions can vary depending on the diagnosis and severity, as well as the clinical setting and context. However, there appears to be no clear consensus on an easy-to-administer global upper extremity functional outcome measure that can quickly provide information about an individual's overall upper extremity function or impairment.
Recently, there has been a rising interest in the field of rehabilitation medicine regarding the use of new technologies as a tool to assess physical function and track progress of therapy [3]. In terms of upper extremity functional assessment, there are a variety of traditional clinical outcome measures that are available, [4][5][6][7][8][9][10][11] but one of the most commonly used measure in multiple conditions may be the range of motion (ROM) evaluation because of its relative costeffectiveness and simplicity [12]. Currently, the primary measurement tool for upper extremity ROM is the manual goniometry; however, proper goniometric assessment is both time-consuming and effortintensive, with a caveat that reliability can vary depending on the experience of the evaluator.
With recent advances in sensor technologies, researchers have introduced a new methodology for the assessment of upper limb function based on the concept of 3D reachable workspace obtained using a depth-ranging sensor [13][14][15][16]. The newly-developed kinect (Microsoft, Redmond, WA, USA) sensor-based system with a customized software is capable of tracking the upper extremity motion and reconstructing a 3D graphical representation of an individual's reachable workspace [17,18]. The technical details of the developed system has already been published [17]. When evaluated simultaneously against a full-scale motion capture system, the Kinectacquired reachable workspace was found to be comparatively robust with high test-retest reliability and minimal data loss. In addition, the developed system has demonstrated excellent validity and high reliability, with sensitivity to detect changes in upper extremity reachable workspace of various neuromuscular conditions [19][20][21]. The simple set-up requiring only a commercially available and costeffective Kinect sensor in combination with a computer/laptop system takes approximately 1 minute to capture and near instantaneously reconstruct an individual's 3D reachable workspace. Originally, the Kinect-based upper extremity functional assessment tool was developed for neuromuscular conditions (muscular dystrophies and motor neuron disease); however, the versatility and applicability of the system may be extended towards various musculoskeletal shoulder dysfunctions. This paper explores new application of the developed 3D reachable workspace assessment tool and outcome measure in 4 patients with musculoskeletal shoulder dysfunctions undergoing rehabilitation.

Case-1
A 35-year old left hand-dominant woman presented with symptoms of continuous pain and limited ROM of the right shoulder after being rear-ended in a motor vehicle accident. A sports medicine physician evaluated her one-month post-injury and recommended outpatient therapy to improve her shoulder ROM. The therapist's initial assessment of the patient's shoulder ROM showed severe limitations in all directions. The patient attended a total of five 1-hour therapy sessions over the course of three weeks. Prior to her first therapy, she underwent the Kinect-based 3D reachable workspace assessment. A graphical example of the patient's normal unaffected side (left) reachable workspace is shown in Figure 1. A line graph demonstrates the difference in the reachable workspace relative surface area (RSA) between her unaffected arm and the affected arm over a span of 190 days ( Figure 2). The Kinect-based RSA provides quantitative and informative data about the reachability of the arms-the unaffected arm showing total RSA of 90% (of the full frontal hemisphere which is within normal range) while on the last day of evaluation, the affected arm still lagged in reachability with a total RSA of 74%. Furthermore, the graphical RSA provides additional information regarding the deficits in reachable workspace-the affected arm shows considerable reduction in the reachability of the upper two quadrants. While the lower two quadrants are relatively well preserved and similar to that of the unaffected arm. Overall, the longitudinal follow up data shows that there was significant improvement in the ROM of the affected arm after undergoing physical therapy sessions.

Case 2:
A 56-year old woman with left shoulder adhesive capsulitis (frozen shoulder) and labral tear underwent outpatient therapy one-month prior to her Kinect-based evaluations. She previously underwent a variety of treatments because of her persistent shoulder dysfunction. She first received a platelet-rich plasma (PRP) injection in her left supraspinatus tendon six-months before her Kinect-based evaluations which is depicted in Figure 3. Exactly one-month prior to her first Kinect visit, she also received a cortisone injection. The patient received oral prednisone for five days just prior to her first Kinect reachable workspace assessment. At the initial Kinect assessment, she was noted with a persistent mild restriction in shoulder ROM that is shown by a slight reduction in RSA of her affected arm compared to her unaffected arm. The serially collected reachable workspace RSAs during the patient's rehabilitation process over the span of 92 days are shown in Figure 3B. The Kinect RSA data showed continued reachability impairment and limitations in the affected arm's ROM. Commonly in refractory frozen shoulder cases, recovery is a lengthy process. The Kinect motion-sensor is able to detect changes over time and it showed that this patient did not fully recover her active ROM over this 3-month period. As the data demonstrated, the patient did not show significant improvement in her shoulder ROM. However, an observed decline in RSA of the affected arm was shown despite undergoing multiple treatment methods and physical therapy sessions.

Case 3:
A 67-year old man with a past history of right supraspinatus tear (>50% thickness tear) underwent surgical repair and post-operative rehabilitation. Four weeks after his shoulder repair surgery, he underwent his first Kinect reachable workspace assessment (marked as Day 0 in Figure 4A). He had severely restricted shoulder ROM showing very reduced RSA that was essentially limited to the lower two quadrants (Figure 4). At his last Kinect session, his reachable workspace showed significant improvement. He regained enough strength to move against gravity and demonstrated ROM within all four quadrants as depicted in (Day 103). Although his right shoulder ROM is still not completely congruent with his unaffected arm, it can be readily appreciated that the patient has made good recovery of his reachable workspace. Five and a half months after his surgery, he was given medical clearance to begin light recreational activities.

Case 4:
A 31-year-old right-hand dominant woman presented with a history of recurrent right shoulder anterior dislocation over a span of fourteen years. Her first shoulder dislocation occurred at sixteen years of age from a surfing accident. Exactly one year later the same injury occurred due to over extension of her right shoulder. The patient underwent reconstructive surgery and by six months post-operatively, she had regained 100% of her shoulder ROM. However, her shoulder was dislocated for the third time at the age of thirty-one and her ROM was very limited. Within days of her right shoulder dislocation, she underwent her first Kinect reachable workspace analysis with follow up data collections over the span of 111 days during her rehabilitation ( Figure 5). Initially, she had significant pain and she had very limited shoulder ROM resulting in essentially zero reachable workspace RSA. The timing of the patient's five therapy visits is also shown in Figure  5A. After the second therapy session, the patient's Kinect reachable workspace demonstrated continued improvement at a slow but steady pace. Over the course of 111 days with therapy, the patient fully regained all ROM of her affected arm (79% RSA) comparable to that of her unaffected arm (82%). Although the patient's active shoulder ROM was regained, her shoulder strength had not yet returned to her pre-injury level. In order to gain a better understanding of the strength deficit in her affected arm, the Kinect-based reachable workspace assessment was conducted under various loading conditions (500-and 1,000-gram wrist weights) ( Figure 6). The patient's reachable workspace in the unaffected arm remained stable with both light and moderately heavy wrist weights. However, the affected arm's reachable workspace can be seen to contract due to the wrist weight. This demonstrates that although her ROM has returned, the data suggests the patient may still benefit with a continued strengthening program.

Discussion
The Microsoft Kinect is a motion-sensing device originally developed as a peripheral tool for use with the Xbox 360 gaming console. Kinect allows users to control and interact with the gaming console without needing to attach markers on the body, which is a major benefit for use in rehabilitation. Recently, researchers have begun to utilize the cost-effective and commercially-available Kinect sensor for medical rehabilitation purposes [3].  This report's investigators had originally developed a novel sensorbased upper extremity reachable workspace assessment tool and outcome measure for use in neuromuscular conditions [17][18][19][20][21]. The foundational concept and framework for utilizing the sensor-based reachable workspace as a global upper extremity functional outcome measure has been previously discussed in published studies [17][18][19][20][21]. However, its use and application in an otherwise healthy population with various musculoskeletal shoulder dysfunctions has not yet been explored. In this case report, four patients with different shoulder dysfunctions were evaluated using the newly-developed Kinect-based reachable workspace assessment system, and their progress through rehabilitation was tracked serially over a course of 92 to 197 days (representing 3-4 serial data collections for each patient during this time). This collection of cases, revealed the sensitivity of reachable workspace measure to detect longitudinal changes in three out of the four patients. We evaluated the ability of RSA to detect change over time and found it easy to discriminate ROM improvement in an affected arm by directly comparing it to the un-affected arm. This comparison is powerful as it allows for a patient to serve as his/her own control. Although the shoulder dysfunctions presented with a range of severity, in general, physical therapy lasted up to six months. RSA evaluations matched this guidance and treatment plan ultimately showing ROM improvements over this time period.
The patients reported an appreciation of the simplicity and the short amount of time it took for the Kinect evaluation to be completed. Additionally, the patients commented positively on the intuitive graphical illustration of their reachable workspace, which was easier to understand and to visualize their individual shoulder's functional deficits. This may provide a more engaging interaction between the patients and their treating clinicians when both can look at the same data and formulate therapeutic plans to achieve their goals of improving ROM and function. Although we did not provide the reachable workspace outcome measure to the respective therapists taking care of each of the patients, our experience thus far has been positive regarding feedback from therapists who also appreciate the scalable and time-efficient features of the newly developed Kinectbased reachable workspace assessment system. This case report represents an initial exploration into utilizing technology-based automated physical function assessment tools and outcome measures. The positive results from the demonstration cases of shoulder dysfunction assessed by the developed Kinect reachable workspace assessment tool and outcome measure suggest promise and clinical utility. As illustrated in this case report, this tool is not just used for neuromuscular diseases but is generalizable to various upper extremity dysfunctions. Kinect allows users to control and interact with the gaming console without needing to attach markers on the body, which is a major benefit for use in rehabilitation. Not only is the Kinect accurate, affordable and home accessible it has a future application beyond the office setting such as use with Telemedicine [18]. The use of Virtual Reality technology has become a major breakthrough for use in the rehabilitation setting. The results indicate that the new methodology for assessment based on the 3D reachable workspace of upper limb function could be beneficial in clinical evaluation of upper extremity musculoskeletal conditions. The RSA of the workspace could provide more sensitive and useful global upper extremity functional measurements [17]. The advanced features of Kinect will be helpful for clinicians and therapists to assess patient performance and track improvements in ROM over the course of rehabilitation.
Although additional studies are needed to further evaluate the validity, reliability, and sensitivity of this system reachable workspace is promising to become a global upper extremity functional outcome measure applicable to both the neuromuscular and musculoskeletal populations. The simple, quick, automated, intuitive, and costeffective features of the developed system, in addition to its advantages of scalability and quantitative nature of the outcome measurement, will be helpful for both clinicians to better assess patient performance and track improvements in ROM over the course of rehabilitation.