Maksym Perepichka
ABSTRACT
Processing motion capture data from optical markers for use in computer animations presents numerous technical challenges. Artifacts caused by noise, marker swaps, and marker occlusions often require manual intervention of a professionally trained marker tracking artist that spends large amounts of time and effort fixing these issues. Existing automatic solutions that attempt to fix marker data lack robustness due to either failing to properly detect and fix marker paths, or generating solutions that are challenging to integrate within current animation pipelines. In this paper, we present a method that robustly identifies invalid marker paths, removes the associated segments and generates new kinematically correct paths. We start by comparing the kinematic solutions generated by commercial software against the one generated by the state-of-the-art methods, using this information to determine which animation keyframes are invalid. Subsequently, we regenerate marker paths from the neural network based method [Holden 2018] and use a sophisticated marker filling algorithm to combine them with the original marker paths at sections where we detect the original data to be invalid. Our method outperforms alternatives by generating solutions that are both closer to the ground truth and more robust, allowing for manual intervention if required.
- Andreas Aristidou, Daniel Cohen-Or, Jessica K Hodgins, and Ariel Shamir. 2018. Self-similarity Analysis for Motion Capture Cleaning. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 297–309.Google Scholar
- Andreas Aristidou and Joan Lasenby. 2013. Real-time marker prediction and CoR estimation in optical motion capture. The Visual Computer 29, 1 (2013), 7–26.Google ScholarCross Ref
- Jan Baumann, Björn Krüger, Arno Zinke, and Andreas Weber. 2011. Data-Driven Completion of Motion Capture Data.. In VRIPHYS. 111–118.Google Scholar
- Mickael Begon, Pierre-Brice Wieber, and Maurice Raymond Yeadon. 2008. Kinematics estimation of straddled movements on high bar from a limited number of skin markers using a chain model. Journal of biomechanics 41, 3 (2008), 581–586.Google ScholarCross Ref
- Samuel Buss. 2004. Introduction to Inverse Kinematics with Jacobian Transpose, Pseudoinverse and Damped Least Squares methods. (2004).Google Scholar
- Judith Butepage, Michael J Black, Danica Kragic, and Hedvig Kjellstrom. 2017. Deep representation learning for human motion prediction and classification. In Proceedings of the IEEE conference on computer vision and pattern recognition. 6158–6166.Google ScholarCross Ref
- Peter Andreas Federolf. 2013. A novel approach to solve the “missing marker problem” in marker-based motion analysis that exploits the segment coordination patterns in multi-limb motion data. PloS one 8, 10 (2013), e78689.Google ScholarCross Ref
- Katerina Fragkiadaki, Sergey Levine, Panna Felsen, and Jitendra Malik. 2015. Recurrent network models for human dynamics. In Proceedings of the IEEE International Conference on Computer Vision. 4346–4354.Google ScholarCross Ref
- Michael Gleicher. 2001. Comparing constraint-based motion editing methods. Graphical models 63, 2 (2001), 107–134.Google Scholar
- Ø Gløersen and P Federolf. 2016. Predicting Missing Marker Trajectories in Human Motion Data Using Marker Intercorrelations. PLoS ONE 11, 3 (2016), e0152616.Google Scholar
- Lorna Herda, Pascal Fua, Ralf Plankers, Ronan Boulic, and Daniel Thalmann. 2000. Skeleton-based motion capture for robust reconstruction of human motion. In Proceedings Computer Animation 2000. IEEE, 77–83.Google ScholarDigital Library
- Daniel Holden. 2018. Robust solving of optical motion capture data by denoising. ACM Transactions on Graphics (TOG) 37, 4 (2018), 165.Google ScholarDigital Library
- Eugene Hsu, Sommer Gentry, and Jovan Popović. 2004. Example-based control of human motion. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics symposium on Computer animation. Eurographics Association, 69–77.Google ScholarDigital Library
- Ashesh Jain, Amir R Zamir, Silvio Savarese, and Ashutosh Saxena. 2016. Structural-RNN: Deep learning on spatio-temporal graphs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5308–5317.Google ScholarCross Ref
- Simon J Julier and Jeffrey K Uhlmann. 1997. New extension of the Kalman filter to nonlinear systems. In Signal processing, sensor fusion, and target recognition VI, Vol. 3068. International Society for Optics and Photonics, 182–193.Google Scholar
- Wooyoung Kim and James M Rehg. 2008. Detection of unnatural movement using epitomic analysis. In 2008 Seventh International Conference on Machine Learning and Applications. IEEE, 271–276.Google ScholarDigital Library
- Taras Kucherenko, Jonas Beskow, and Hedvig Kjellström. 2018. A neural network approach to missing marker reconstruction in human motion capture. arXiv preprint arXiv:1803.02665(2018).Google Scholar
- Jehee Lee and Sung Yong Shin. 1999. A hierarchical approach to interactive motion editing for human-like figures. In Siggraph, Vol. 99. 39–48.Google Scholar
- Lei Li, James McCann, Nancy Pollard, and Christos Faloutsos. 2010. Bolero: a principled technique for including bone length constraints in motion capture occlusion filling. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Eurographics Association, 179–188.Google Scholar
- Lei Li, James McCann, Nancy S Pollard, and Christos Faloutsos. 2009. Dynammo: Mining and summarization of coevolving sequences with missing values. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 507–516.Google ScholarDigital Library
- Guodong Liu and Leonard McMillan. 2006. Estimation of missing markers in human motion capture. The Visual Computer 22, 9-11 (2006), 721–728.Google ScholarDigital Library
- Xin Liu, Yiu-ming Cheung, Shu-Juan Peng, Zhen Cui, Bineng Zhong, and Ji-Xiang Du. 2014. Automatic motion capture data denoising via filtered subspace clustering and low rank matrix approximation. Signal Processing 105(2014), 350–362.Google ScholarCross Ref
- Utkarsh Mall, G Roshan Lal, Siddhartha Chaudhuri, and Parag Chaudhuri. 2017. A deep recurrent framework for cleaning motion capture data. arXiv preprint arXiv:1712.03380(2017).Google Scholar
- Liu Ren, Alton Patrick, Alexei A Efros, Jessica K Hodgins, and James M Rehg. 2005. A data-driven approach to quantifying natural human motion. In ACM Transactions on Graphics (TOG), Vol. 24. ACM, 1090–1097.Google Scholar
- Abraham. Savitzky and M. J. E. Golay. 1964. Smoothing and Differentiation of Data by Simplified Least Squares Procedures.Analytical Chemistry 36, 8 (1964), 1627–1639. https://doi.org/10.1021/ac60214a047Google Scholar
- Wei Shen, Ke Deng, Xiang Bai, Tommer Leyvand, Baining Guo, and Zhuowen Tu. 2012. Exemplar-based human action pose correction and tagging. In 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1784–1791.Google ScholarCross Ref
- Hyun Joon Shin, Jehee Lee, Sung Yong Shin, and Michael Gleicher. 2001. Computer puppetry: An importance-based approach. ACM Transactions on Graphics (TOG) 20, 2 (2001), 67–94.Google ScholarDigital Library
- Vicon Software. 2019. Vicon Shogun. https://www.vicon.com/products/software/Google Scholar
- Seyoon Tak and Hyeong-Seok Ko. 2005. A physically-based motion retargeting filter. ACM Transactions on Graphics (TOG) 24, 1 (2005), 98–117.Google ScholarDigital Library
- Mickaël Tits, Joëlle Tilmanne, and Thierry Dutoit. 2018. Robust and automatic motion-capture data recovery using soft skeleton constraints and model averaging. PLOS ONE 13, 7 (07 2018), 1–21. https://doi.org/10.1371/journal.pone.0199744Google Scholar
- Eric A Wan and Rudolph Van Der Merwe. 2000. The unscented Kalman filter for nonlinear estimation. In Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No. 00EX373). Ieee, 153–158.Google ScholarCross Ref
- Xin Wang, Qiudi Chen, and Wanliang Wang. 2014. 3D human motion editing and synthesis: A survey. Computational and Mathematical methods in medicine 2014 (2014).Google ScholarCross Ref
- Xinyi Zhang and Michiel van de Panne. 2018. Data-driven Autocompletion for Keyframe Animation. In MIG ’18: Motion, Interaction and Games (MIG ’18).Google Scholar
Recommendations
Robust solving of optical motion capture data by denoising
Raw optical motion capture data often includes errors such as occluded markers, mislabeled markers, and high frequency noise or jitter. Typically these errors must be fixed by hand - an extremely time-consuming and tedious task. Due to this, there is a ...
Mobilizing mocap, motion blending, and mayhem: rig interoperability for crowd simulation on Incredibles 2
SIGGRAPH '18: ACM SIGGRAPH 2018 TalksThe stylized world of Incredibles 2 features large urban crowds both in everyday situations and in scenes of panicked mayhem. While Pixar's now academy award winning animation software, Presto, has allowed us to create expressive and nuanced rigs for ...
Robust online motion capture labeling of finger markers
MIG '16: Proceedings of the 9th International Conference on Motion in GamesPassive optical motion capture is one of the predominant technologies for capturing high fidelity human skeletal motion, and is a workhorse in a large number of areas such as bio-mechanics, film and video games. While most state-of-the-art systems can ...
Comments