Skip to main content
Log in

Synthesis of sign language co-articulation based on key frames

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Co-articulation is a language phenomenon. In sign language (SL), it takes the form of impact among adjacent signs, which results in variations of signs from their standard configurations. Standard configuration is the appearance of a sign when it appears singly, without context. Without co-articulation, SL animation based on virtual character will be a simple concatenation of signs. The movement of virtual character will be mechanical, lacking fluency and realism. This paper presents a key frame based SL co-articulation animation scheme aiming at the three most important elements of co-articulation, i.e. hand shape, hand position and SL speed. To generate co-articulation, motion data of signs which appear sequentially is parsed to identify hand shapes and positions included in these signs. Then, co-articulation will be achieved through some modification to the motion data according to the interaction between adjacent hand shapes and adjacent hand positions. SL Speed acts as an adjusting parameter which dynamically impacts co-articulation. Different expression speed will lead to different degree of co-articulation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Chi D, Costa M, Zhao L, Badler N (2000) The EMOTE model for effort and shape. In: Proceedings of the 27th annual conference on Computer graphics and interactive techniques. SIGGRAPH’00, New Orleans, Louisiana, USA, July 23–28, 2000, pp 173–182. doi:10.1145/344779.352172

  2. China Association for the deaf (2003) Chinese sign language. HuaXia Press, Beijing

    Google Scholar 

  3. Conway A, Veale T (1994) A linguistic approach to sign language synthesis. In: People and Computers IX. Cambridge University Press, Cambridge, pp 211–222

    Google Scholar 

  4. Dasgupta T, Basu A (2008) Prototype machine translation system from text-to-Indian sign language. In: 13th International Conference on Intelligent User Interfaces. IUI’08, Canary Islands, Spain, January 13–16, 2008, pp 313–316. doi:10.1145/1378773.1378818

  5. Delorme M, Filhol M, Braffort A (2009) Animation generation process for sign language synthesis. In: 2nd International Conferences on Advances in Computer-Human Interactions. ACHI’09, Cancun, Mexico, February 1–7, 2009, pp 386–390. doi:10.1109/ACHI.2009.29

  6. Grosvald M, Corina D (2008) Location-to-Location coarticulation: A phonetic investigation of American Sign Language. In: Proceedings of the 24th Northwest Linguistics Conference. NWLC’08, Seattle, Washington, May 3–4, 2008, pp 59–66

  7. Jerde TE, Soechting JF, Flanders M (2003) Coarticulation in Fluent Fingerspelling. The Journal of Neuroscience 23:2383–2393

    Google Scholar 

  8. Jerde TE, Soechting JF, Flanders M (2003) Biological constraints simplify the recognition of hand shapes. IEEE Transactions on Biomedical Engineering 50:265–269. doi:10.1109/TBME.2002.807640

    Article  Google Scholar 

  9. Li S, Kong D, Wang L, Sun Y, Huang Q (2012) Optimal control theory based interpolation for Chinese sign language animation. In: The 15th International Conference on Geometry and Graphics. ICGG’12, Montreal, Canada, August 1–5, 2012, pp 397–402

  10. Liddell SK (1984) Unrealized inceptive aspect in American sign language: feature insertion in syllabic frames. Chicago Linguistics Society 20:257–270

    Google Scholar 

  11. Luo W (2008) Study on the Handshape of Chinese sign language. Beijing Normal University, Dissertation

    Google Scholar 

  12. Mauk CE (2003) Undershoot in two modalities: Evidence from fast speech and fast signing. Dissertation, University of Texas at Austin

  13. San-Segundo R, Barra R, Córdoba R et al (2008) Speech to sign language translation system for Spanish. Speech Communication 50:1009–1020. doi:10.1016/j.specom.2008.02.001

    Article  Google Scholar 

  14. Segouat J, Braffort A (2009) Toward the study of sign language coarticulation: Methodology Proposal. In: Second International Conferences on Advances in Computer-Human Interactions. ACHI’09, Cancun, Mexico, February 1–7, 2009, pp 369–374. doi:10.1109/ACHI.2009.25

  15. Shantz M, Poizner H (1982) A computer program to synthesize American sign language. Behavior Research Methods and Instrumentation 14:467–474. doi:10.3758/BF03203314

    Article  Google Scholar 

  16. Sharieh A, Ellia G (2008) Toward international multi-sign languages. In: 19th International Conference on Database and Expert Systems Applications. DEXA’08, Turin, Italy, September 1–5, 2008, pp 622–625. doi:10.1109/DEXA.2008.129

  17. Sosnik R, Hauptmann B, Karni A, Flash T (2004) When practice leads to co-articulation: the evolution of geometrically defined movement primitives. Experimental Brain Research 156:422–438. doi:10.1007/s00221-003-1799-4

    Article  Google Scholar 

  18. Stokoe WC (1978) Sign language structure. Linstok Press, Silver Spring

    Google Scholar 

  19. Stokoe WC, Casterline DC, Croneberg CG (1976) A dictionary of American sign language on linguistic principles. Linstok Press, Silver Spring

    Google Scholar 

  20. Sudarsky S, House D (2000) An integrated approach towards the representation, manipulation and reuse of prerecorded motion. In: Proceedings of Computer Animation. CA’00, Philadelphia, PA, USA, 2000, pp 56–61. doi:10.1109/CA.2000.889036

  21. Ye K, Yin B, Wang L (2009) CSLML: a markup language for expressive Chinese sign language systhesis. Computer Animation and Virtual Worlds 20:237–245. doi:10.1002/cav.307

    Article  Google Scholar 

Download references

Acknowledgments

The work in this paper was funded by National Natural Science Foundation of China (No. 61170104, 61370119, U0935004, 61227004), Beijing Natural Science Foundation (4112008).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shuo Li.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Li, S., Wang, L. & Kong, D. Synthesis of sign language co-articulation based on key frames. Multimed Tools Appl 74, 1915–1933 (2015). https://doi.org/10.1007/s11042-013-1724-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1724-1

Keywords

Navigation