Steven S. An
Doug L. James
Skip Abstract Section
Skip Supplemental Material SectionAbstract
We present a practical data-driven method for automatically synthesizing plausible soundtracks for physics-based cloth animations running at graphics rates. Given a cloth animation, we analyze the deformations and use motion events to drive crumpling and friction sound models estimated from cloth measurements. We synthesize a low-quality sound signal, which is then used as a target signal for a concatenative sound synthesis (CSS) process. CSS selects a sequence of microsound units, very short segments, from a database of recorded cloth sounds, which best match the synthesized target sound in a low-dimensional feature-space after applying a hand-tuned warping function. The selected microsound units are concatenated together to produce the final cloth sound with minimal filtering. Our approach avoids expensive physics-based synthesis of cloth sound, instead relying on cloth recordings and our motion-driven CSS approach for realism. We demonstrate its effectiveness on a variety of cloth animations involving various materials and character motions, including first-person virtual clothing with binaural sound.
Supplemental Material
Available for Download
zip
a102-an.zip (378.2 MB)
Supplemental material.
- Baraff, D., and Witkin, A. P. 1998. Large steps in cloth simulation. In Proceedings of SIGGRAPH 98, Computer Graphics Proceedings, Annual Conference Series, 43--54. Google Scholar
Digital Library
- Belongie, S., Malik, J., and Puzicha, J. 2002. Shape matching and object recognition using shape contexts. Pattern Analysis and Machine Intelligence, IEEE Trans. on 24, 4 (April), 509--522. Google ScholarDigital Library
- Bilbao, S. 2009. Numerical Sound Synthesis. Wiley Online Library.Google Scholar
- Bridson, R., Fedkiw, R. P., and Anderson, J. 2002. Robust treatment of collisions, contact, and friction for cloth animation. ACM Transactions on Graphics 21, 3 (July), 594--603. Google ScholarDigital Library
- Brown, C., and Duda, R. 1998. A structural model for binaural sound synthesis. Speech and Audio Processing, IEEE Transactions on 6, 5 (sep), 476--488.Google Scholar
- Cardle, M., Brooks, S., Bar-Joseph, Z., and Robinson, P. 2003. Sound-by-numbers: motion-driven sound synthesis. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA '03, 349--356. Google ScholarDigital Library
- Chadwick, J. N., and James, D. L. 2011. Animating fire with sound. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2011) 30, 4 (Aug.). Google ScholarDigital Library
- Chadwick, J. N., An, S. S., and James, D. L. 2009. Harmonic shells: A practical nonlinear sound model for near-rigid thin shells. ACM Transactions on Graphics 28, 5 (Dec.), 119:1--119:10. Google ScholarDigital Library
- Cho, G., Casali, J., and Yi, E. 2001. Effect of fabric sound and touch on human subjective sensation. Fibers and Polymers 2, 196--202.Google ScholarCross Ref
- Cho, G., Kim, C., Cho, J., and Ha, J. 2005. Physiological signal analyses of frictional sound by structural parameters of warp knitted fabrics. Fibers and Polymers 6, 89--94.Google ScholarCross Ref
- Choi, K.-J., and Ko, H.-S. 2002. Stable but responsive cloth. ACM Transactions on Graphics 21, 3 (July), 604--611. Google ScholarDigital Library
- Cook, P. 2002. Real Sound Synthesis for Interactive Applications. A. K. Peters. Google ScholarDigital Library
- Courshesnes, M., Volino, P., and Magnenat-Thalmann, N. 1995. Versatile and efficient techniques for simulating cloth and other deformable objects. In Proceedings of SIGGRAPH 95, Computer Graphics Proceedings, Annual Conference Series, 137--144. Google ScholarDigital Library
- Dubnov, S., Bar-Joseph, Z., El-Yaniv, R., Lischinski, D., and Werman, M. 2002. Synthesizing sound textures through wavelet tree learning. Computer Graphics and Applications, IEEE 22, 4, 38--48. Google ScholarDigital Library
- Fontana, F., and Bresin, R. 2003. Physics-based sound synthesis and control: crushing, walking and running by crumpling sounds. In Proc. Colloquium on Musical Informatics, 109--114.Google Scholar
- Heeger, D. J., and Bergen, J. R. 1995. Pyramid-based texture analysis/synthesis. In Proceedings of SIGGRAPH 95, Computer Graphics Proceedings, Annual Conference Series, 229--238. Google ScholarDigital Library
- Houle, P., and Sethna, J. 1996. Acoustic emission from crumpling paper. Physical Review E 54, 1, 278.Google ScholarCross Ref
- Huang, G., Metaxas, D., and Govindaraj, M. 2003. Feel the "fabric": an audio-haptic interface. In 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 52--61. Google ScholarDigital Library
- Hunt, A., and Black, A. 1996. Unit selection in a concatenative speech synthesis system using a large speech database. In Acoustics, Speech, and Signal Processing, 1996. ICASSP-96. Conference Proceedings., 1996 IEEE International Conference on, vol. 1, IEEE, 373--376. Google ScholarDigital Library
- James, D. L., Barbič, J., and Pai, D. K. 2006. Precomputed Acoustic Transfer: Output-sensitive, accurate sound generation for geometrically complex vibration sources. ACM Transactions on Graphics 25, 3 (July), 987--995. Google ScholarDigital Library
- Kaldor, J. M., James, D. L., and Marschner, S. 2008. Simulating knitted cloth at the yarn level. ACM Transactions on Graphics 27, 3 (Aug.), 65:1--65:9. Google ScholarDigital Library
- Karplus, K., and Strong, A. 1983. Digital Synthesis of Plucked-String and Drum Timbres. Computer Music Journal 7, 2, 43--55.Google ScholarCross Ref
- Marelli, D., Aramaki, M., Kronland-Martinet, R., and Verron, C. 2010. Time--frequency synthesis of noisy sounds with narrow spectral components. Audio, Speech, and Language Processing, IEEE Transactions on 18, 8, 1929--1940. Google ScholarDigital Library
- Matusik, W., Zwicker, M., and Durand, F. 2005. Texture design using a simplicial complex of morphable textures. ACM Transactions on Graphics 24, 3 (Aug.), 787--794. Google ScholarDigital Library
- O'Brien, J. F., Cook, P. R., and Essl, G. 2001. Synthesizing sounds from physically based motion. In Proceedings of ACM SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, 529--536. Google ScholarDigital Library
- O'Brien, J. F., Shen, C., and Gatchalian, C. M. 2002. Synthesizing sounds from rigid-body simulations. In ACM SIGGRAPH Symposium on Computer Animation, 175--181. Google ScholarDigital Library
- Peltola, L., Erkut, C., Cook, P. R., and Valimaki, V. 2007. Synthesis of hand clapping sounds. Audio, Speech, and Language Processing, IEEE Transactions on 15, 3, 1021--1029. Google ScholarDigital Library
- Picard, C., Tsingos, N., and Faure, F. 2009. Retargetting example sounds to interactive physics-driven animations. In AES 35th International Conference on Audio for Games.Google Scholar
- Pullen, K., and Bregler, C. 2002. Motion capture assisted animation: Texturing and synthesis. ACM Transactions on Graphics 21, 3 (July), 501--508. Google ScholarDigital Library
- Rabiner, L., and Juang, B. H. 1993. Fundamentals of Speech Recognition, united states ed. Prentice Hall, Apr. Google ScholarDigital Library
- Raghuvanshi, N., and Lin, M. C. 2006. Interactive Sound Synthesis for Large Scale Environments. In SI3D '06: Proceedings of the 2006 symposium on Interactive 3D graphics and games, ACM Press, New York, NY, USA, 101--108. Google ScholarDigital Library
- Roads, C. 2004. Microsound. The MIT Press. Google ScholarDigital Library
- Rodet, X., Depall, P., Rodet, X., and Depalle, P. 1992. Spectral envelopes and inverse FFT synthesis. In Proceedings of the 93rd Audio Engineering Society Convention.Google Scholar
- Schwarz, D. 2004. Data-Driven Concatenative Sound Synthesis. PhD thesis, Ircam, Centre Pompidou, University of Paris 6--Pierre et Marie Curie.Google Scholar
- Serra, X., and Smith, Julius, I. 1990. Spectral modeling synthesis: A sound analysis/synthesis system based on a deterministic plus stochastic decomposition. Computer Music Journal 14, 4, pp. 12--24.Google ScholarCross Ref
- Strobl, G., Eckel, G., Rocchesso, D., and le Grazie, S. 2006. Sound texture modeling: A survey. In Proceedings of the 2006 Sound and Music Computing (SMC) International Conference, 61--5.Google Scholar
- Takala, T., and Hahn, J. 1992. Sound rendering. In Computer Graphics (Proceedings of SIGGRAPH 92), 211--220. Google ScholarDigital Library
- Terzopoulos, D., Platt, J., Barr, A., and Fleischer, K. 1987. Elastically deformable models. In Computer Graphics (Proceedings of SIGGRAPH 87), 205--214. Google ScholarDigital Library
- van den Doel, K., and Pai, D. K. 1996. Synthesis of shape dependent sounds with physical modeling. In Intl Conf. on Auditory Display.Google Scholar
- van den Doel, K., Kry, P. G., and Pai, D. K. 2001. FoleyAutomatic: Physically Based Sound Effects for Interactive Simulation and Animation. In Proceedings of ACM SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, 537--544. Google ScholarDigital Library
- Wahba, G. 1990. Spline Models for Observational Data. Society for Industrial and Applied Mathematics, Philadephia, PA.Google Scholar
- Zheng, C., and James, D. L. 2010. Rigid-body fracture sound with precomputed soundbanks. ACM Transactions on Graphics 29, 4 (July), 69:1--69:13. Google ScholarDigital Library
Index Terms
- Motion-driven concatenative synthesis of cloth sounds
Recommendations
Faking dynamics of cloth animation for animated films
AMDO'10: Proceedings of the 6th international conference on Articulated motion and deformable objectsIn this paper we argue for the concept of fake dynamics to allow animators to interactively create visually pleasing animations of cloth models while keeping him/her in full control of the animation process. Existing animation and simulation techniques ...
Animating Wrinkles by Example on Non-Skinned Cloth
The simulation of cloth with rich folds and wrinkles is a computationally expensive process. In this paper, we introduce an example-based algorithm for fast animation of plausible cloth wrinkles. Our algorithm does not depend on a character's pose, ...
Faking Dynamics of Cloth Animation for Animated Films
6th International Conference on Articulated Motion and Deformable Objects - Volume 6169In this paper we argue for the concept of fake dynamics to allow animators to interactively create visually pleasing animations of cloth models while keeping him/her in full control of the animation process. Existing animation and simulation techniques ...
Comments