Jie Tan
Skip Abstract Section
Skip Supplemental Material SectionAbstract
We present a general approach for simulating and controlling a human character that is riding a bicycle. The two main components of our system are offline learning and online simulation. We simulate the bicycle and the rider as an articulated rigid body system. The rider is controlled by a policy that is optimized through offline learning. We apply policy search to learn the optimal policies, which are parameterized with splines or neural networks for different bicycle maneuvers. We use Neuroevolution of Augmenting Topology (NEAT) to optimize both the parametrization and the parameters of our policies. The learned controllers are robust enough to withstand large perturbations and allow interactive user control. The rider not only learns to steer and to balance in normal riding situations, but also learns to perform a wide variety of stunts, including wheelie, endo, bunny hop, front wheel pivot and back hop.
Supplemental Material
Available for Download
zip
a50-tan.zip (178.2 MB)
Supplemental material.
- Allen, B., and Faloutsos, P. 2009. Evolved controllers for simulated locomotion. In Motion in Games, Lecture Notes in Computer Science, 219--230. Google Scholar
Digital Library
- Andrews, S., and Kry, P. 2013. Goal directed multi-finger manipulation: Control policies and analysis. Computers & Graphics 37, 7, 830--839. Google ScholarDigital Library
- Auslander, J., Fukunaga, A., Partovi, H., Christensen, J., Hsu, L., Reiss, P., Shuman, A., Marks, J., and Ngo, J. T. 1995. Further experience with controller-based automatic motion synthesis for articulated figures. ACM Trans. Graph. 14, 4 (Oct.), 311--336. Google ScholarDigital Library
- BBC. 2005. Bicycle chosen as best invention. BBC News.Google Scholar
- Boyan, J. A., and Moore, A. W. 1995. Generalization in reinforcement learning: Safely approximating the value function. In Advances in Neural Information Processing Systems 7, MIT Press, 369--376.Google Scholar
- Carvallo, M. E. 1900. Théorie du mouvement du monocycle et de la bicyclette. Journal de L'Ecole Polytechnique 5.Google Scholar
- Chambaron, S., Berberian, B., Delbecque, L., Ginhac, D., and Cleeremans, A. 2009. Implicit motor learning in discrete and continuous tasks: Toward a possible account of discrepant results. Handbook of Motor Skills: Development, Impairment, and Therapy, 139--155. Google ScholarDigital Library
- Collins, R. N. 1963. A mathematical analysis of the stability of two-wheeled vehicles. PhD thesis, University of Wisconsin. Google ScholarDigital Library
- Coros, S., Beaudoin, P., and van de Panne, M. 2009. Robust task-based control policies for physics-based characters. ACM Trans. Graph. 28, 5 (Dec.), 170:1--170:9. Google ScholarDigital Library
- Coros, S., Beaudoin, P., and van de Panne, M. 2010. Generalized biped walking control. ACM Transctions on Graphics 29, 4, Article 130. Google ScholarDigital Library
- Coros, S., Karpathy, A., Jones, B., Reveret, L., and van de Panne, M. 2011. Locomotion skills for simulated quadrupeds. ACM Transactions on Graphics 30, 4. Google ScholarDigital Library
- da Silva, M., Abe, Y., and Popović, J. 2008. Interactive simulation of stylized human locomotion. In ACM SIGGRAPH 2008 Papers, ACM, New York, NY, USA, SIGGRAPH '08, 82:1--82:10. Google ScholarDigital Library
- de Lasa, M., and Hertzmann, A. 2009. Prioritized optimization for task-space control. In International Conference on Intelligent Robots and Systems (IROS). Google ScholarDigital Library
- Geijtenbeek, T., and Pronost, N. 2012. Interactive Character Animation Using Simulated Physics: A State-of-the-Art Review. Computer Graphics Forum 31, 8, 2492--2515. Google ScholarDigital Library
- Geijtenbeek, T., van de Panne, M., and van der Stappen, A. F. 2013. Flexible muscle-based locomotion for bipedal creatures. ACM Transactions on Graphics 32, 6. Google ScholarDigital Library
- Grzeszczuk, R., and Terzopoulos, D. 1995. Automated learning of muscle-actuated locomotion through control abstraction. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, 63--70. Google ScholarDigital Library
- Hansen, N. 2009. The CMA Evolution Strategy: A Tutorial.Google Scholar
- Heidrich-Meisner, V., and Igel, C. 2008. Evolution strategies for direct policy search. In Proceedings of the 10th International Conference on Parallel Problem Solving from Nature: PPSN X, Springer-Verlag, Berlin, Heidelberg, 428--437. Google ScholarDigital Library
- Hinton, G. E. 2007. Learning multiple layers of representation. Trends in Cognitive Sciences 11, 428--434.Google ScholarCross Ref
- Hodgins, J. K., Sweeney, P. K., and Lawrence, D. G. 1992. Generating natural-looking motion for computer animation. In Proceedings of the Conference on Graphics Interface '92, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 265--272. Google ScholarDigital Library
- Hodgins, J. K., Wooten, W. L., Brogan, D. C., and O'Brien, J. F. 1995. Animating human athletics. In SIGGRAPH, 71--78. Google ScholarDigital Library
- Jones, D. E. H. 1970. The Stability of the bicycle. Physics Today 23, 34--40.Google ScholarCross Ref
- Klein, F., and Sommerfeld, A. 1910. Stabilität des fahrrads. Über die Theorie des Kreisels, Ch. IX, Section 8, 863--884.Google Scholar
- Kooijman, J. D. G., Meijaard, J. P., Papadopoulos, J. M., Ruina, A., and Schwab, A. L. 2011. A Bicycle Can Be Self-Stable Without Gyroscopic or Caster Effects. Science 332, 6027 (Apr.), 339--342.Google ScholarCross Ref
- Kwon, T., and Hodgins, J. 2010. Control systems for human running using an inverted pendulum model and a reference motion capture sequence. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, SCA '10, 129--138. Google ScholarDigital Library
- Laszlo, J., van de Panne, M., and Fiume, E. 1996. Limit cycle control and its application to the animation of balancing and walking. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, NY, USA, SIGGRAPH '96, 155--162. Google ScholarDigital Library
- Levine, S., and Koltun, V. 2013. Guided policy search. In ICML '13: Proceedings of the 30th International Conference on Machine Learning.Google Scholar
- Levine, S., Wang, J. M., Haraux, A., Popović, Z., and Koltun, V. 2012. Continuous character control with low-dimensional embeddings. ACM Trans. Graph. 31, 4 (July), 28:1--28:10. Google ScholarDigital Library
- Meijaard, J. P., Papadopoulos, J. M., Ruina, A., and Schwab, A. L. 2007. Linearized dynamics euqations for the balance and steer of a bicycle: a benchmark and review. Proceedings of the Royal Society A.Google Scholar
- Mordatch, I., de Lasa, M., and Hertzmann, A. 2010. Robust Physics-Based Locomotion Using Low-Dimensional Planning. ACM Transactions on Graphics 29, 3. Google ScholarDigital Library
- Muico, U., Lee, Y., Popović, J., and Popović, Z. 2009. Contact-aware nonlinear control of dynamic characters. In ACM SIGGRAPH 2009 Papers, ACM, New York, NY, USA, SIGGRAPH '09, 81:1--81:9. Google ScholarDigital Library
- Ng, A. Y., and Jordan, M. 2000. Pegasus: A policy search method for large MDPs and POMDPs. In Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, UAI'00, 406--415. Google ScholarDigital Library
- Ng, A. Y., and Russell, S. J. 2000. Algorithms for inverse reinforcement learning. In Proceedings of the Seventeenth International Conference on Machine Learning, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, ICML '00, 663--670. Google ScholarDigital Library
- Ngo, J. T., and Marks, J. 1993. Spacetime constraints revisited. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, NY, USA, SIGGRAPH '93, 343--350. Google ScholarDigital Library
- Peters, J., and Schaal, S. 2008. Reinforcement learning of motor skills with policy gradients. Neural Networks 21, 4 (May), 682--697. Google ScholarDigital Library
- Pratt, J. E., Chew, C.-M., Torres, A., Dilworth, P., and Pratt, G. A. 2001. Virtual model control: An intuitive approach for bipedal locomotion. Int'l J. Robotic Research. 20, 2, 129--143.Google ScholarCross Ref
- Randløv, J., and Alstrøm, P. 1998. Learning to drive a bicycle using reinforcement learning and shaping. In Proceedings of the Fifteenth International Conference on Machine Learning (ICML 1998), Morgan Kauffman, San Francisco, CA, USA, J. W. Shavlik, Ed., 463--471. Google ScholarDigital Library
- Rankine, W. J. M. 1870. On the dynamical principles of the motion of velocipedes. The Engineer.Google Scholar
- Sims, K. 1994. Evolving virtual creatures. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, NY, USA, SIGGRAPH '94, 15--22. Google ScholarDigital Library
- Singh, D. V. 1964. Advanced concepts of the stability of two-wheeled vehicle-application of mathematical analysis to actual vehicles. PhD thesis, University of Wisconsin.Google Scholar
- Smith, R., 2008. Open dynamics engine. http://www.ode.org/.Google Scholar
- Stanley, K. O., and Miikkulainen, R. 2002. Evolving neural networks through augmenting topologies. Evol. Comput. 10, 2 (June), 99--127. Google ScholarDigital Library
- Sutton, R. S., and Barto, A. G. 1998. Introduction to Reinforcement Learning, 1st ed. MIT Press, Cambridge, MA, USA. Google ScholarDigital Library
- Tan, J., Gu, Y., Turk, G., and Liu, C. K. 2011. Articulated swimming creatures. In ACM SIGGRAPH 2011 papers, ACM, SIGGRAPH '11, 58:1--58:12. Google ScholarDigital Library
- Tan, J., Turk, G., and Liu, C. K. 2012. Soft body locomotion. ACM Trans. Graph. 31, 4 (July), 26:1--26:11. Google ScholarDigital Library
- Thrun, S., and Schwartz, A. 1993. Issues in using function approximation for reinforcement learning. In In Proceedings of the Fourth Connectionist Models Summer School, Erlbaum.Google Scholar
- Treuille, A., Lee, Y., and Popović, Z. 2007. Near-optimal character animation with continuous control. ACM Trans. Graph. 26, 3 (July). Google ScholarDigital Library
- Tsai, Y.-Y., Lin, W.-C., Cheng, K. B., Lee, J., and Lee, T.-Y. 2010. Real-time physics-based 3D biped character animation using an inverted pendulum model. IEEE Transactions on Visualization and Computer Graphics 16, 2 (Mar.), 325--337. Google ScholarDigital Library
- van de Panne, M., and Fiume, E. 1993. Sensor-actuator networks. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, NY, USA, SIGGRAPH '93, 335--342. Google ScholarDigital Library
- van de Panne, M., and Lee, C. 2003. Ski stunt simulator: Experiments with interactive dynamics. In Proceedings of the 14th Western Computer Graphics Symposium.Google Scholar
- Van Zytveld, P. 1975. A Method for the Automatic Stabilization of an Unmanned Bicycle. Department of Aeronautics and Astronautics, Stanford University.Google Scholar
- Wang, J. M., Fleet, D. J., and Hertzmann, A. 2009. Optimizing walking controllers. ACM Trans. Graph. 28, 5 (Dec.), 168:1--168:8. Google ScholarDigital Library
- Wang, J. M., Fleet, D. J., and Hertzmann, A. 2010. Optimizing walking controllers for uncertain inputs and environments. ACM Trans. Graph. 29, 4 (July), 73:1--73:8. Google ScholarDigital Library
- Wang, J. M., Hamner, S. R., Delp, S. L., and Koltun, V. 2012. Optimizing locomotion controllers using biologically-based actuators and objectives. ACM Trans. Graph. 31, 4 (July), 25:1--25:11. Google ScholarDigital Library
- Whipple, F. J. W. 1899. The stability of the motion of a bicycle. Quarterly Journal of Pure and Applied Mathematics 30, 312--348.Google Scholar
- Wu, J.-c., and Popović, Z. 2003. Realistic modeling of bird flight animations. In ACM SIGGRAPH 2003 Papers, ACM, New York, NY, USA, SIGGRAPH '03, 888--895. Google ScholarDigital Library
- Ye, Y., and Liu, C. K. 2010. Optimal feedback control for character animation using an abstract model. In SIGGRAPH '10: ACM SIGGRAPH 2010 papers, ACM, New York, NY, USA, 1--9. Google ScholarDigital Library
- Yin, K., Loken, K., and van de Panne, M. 2007. SIMBICON: simple biped locomotion control. In ACM SIGGRAPH 2007 papers, SIGGRAPH '07. Google ScholarDigital Library
- Yin, K., Coros, S., Beaudoin, P., and van de Panne, M. 2008. Continuation methods for adapting simulated skills. ACM Trans. Graph. 27, 3. Google ScholarDigital Library
- Zhao, P., and van de Panne, M. 2005. User interfaces for interactive control of physics-based 3D characters. In Proceedings of the 2005 Symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D '05, 87--94. Google ScholarDigital Library
Index Terms
- Learning bicycle stunts
Recommendations
Road Detection for Reinforcement Learning Based Autonomous Car
ICISS '20: Proceedings of the 3rd International Conference on Information Science and SystemsHuman mistakes in traffic often have terrible consequences. The long-awaited introduction of self-driving vehicles may solve many of the problems with traffic, but much research is still needed before cars are fully autonomous.
In this paper, we propose ...
A Flexible Simulation Environment for Enhanced VRU Research
AutomotiveUI '23 Adjunct: Adjunct Proceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular ApplicationsVirtual (VR) and mixed reality (MR) simulations allow researchers to explore technologies and concepts that are not yet available. In the automotive sector, driving simulators are often used in user studies to investigate the experiences of drivers and ...
Road traffic optimisation based on a learning approach
Road traffic management is an important issue for authorities, road operators and users. Many researchers have worked since many years in order to propose smart solutions. Some of them have been adopted by road operators and applied on road signalling or ...
Comments