CONFERENCE PROCEEDINGS
Advanced Search
Home > Proceedings > Volume 11526 > Article
Paper
24 June 2020 A framework for multimodal sign language recognition under small sample based on key-frame sampling
Jianyu Wang, Jianxin Chen, Yihao Cai
Author Affiliations +
Proceedings Volume 11526, Fifth International Workshop on Pattern Recognition; 115260A (2020) https://doi.org/10.1117/12.2574424
Event: Fifth International Workshop on Pattern Recognition, 2020, Chengdu, China
Abstract
Sign language recognition is challenging, due to the scarcity of available annotated corpora and the difficulty of large vocabulary. In this paper, we study the task based on a Chinese SL database-DEVISIGN, but it only has a few samples to train the deep network on the scratch. First, we segment the hand to eliminate the disturbance of irrelevant factors. By analyzing the special movement tendency of sign words, we propose two novel Key-frame selection schemes. Since no other datasets can have similar data distribution with our preprocessed data, we invent a novel cross-sampling approach, which successfully prevent the overfitting under small sample. To enhance the diversity of data, we take several samplingbased videos as input, and learn spatiotemporal features based on R(2+1)D-18 layers, which is successful in action recognition tasks. Finally, it is shown that our solution can obtain the state-of-the-art performance.
© (2020) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Citation Download Citation
Jianyu Wang, Jianxin Chen, and Yihao Cai "A framework for multimodal sign language recognition under small sample based on key-frame sampling", Proc. SPIE 11526, Fifth International Workshop on Pattern Recognition, 115260A (24 June 2020); https://doi.org/10.1117/12.2574424
ACCESS THE FULL ARTICLE
ORGANIZATIONAL
Sign in with credentials provided by your organization.
INSTITUTIONAL
Select your institution to access the SPIE Digital Library.
SELECT YOUR INSTITUTION
PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.
PERSONAL SIGN IN
No SPIE Account? Create one
PURCHASE THIS CONTENT
SUBSCRIBE TO DIGITAL LIBRARY
50 downloads per 1-year subscription
Members: $195
Non-members: $335 ADD TO CART
25 downloads per 1 - year subscription
Members: $145
Non-members: $250 ADD TO CART
PURCHASE SINGLE ARTICLE
Includes PDF, HTML & Video, when available
Members: $17.00
Non-members: $21.00 ADD TO CART
PROCEEDINGS
 7 PAGES
DOWNLOAD PAPER SAVE TO MY LIBRARY
GET CITATION
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Image processing
Artificial intelligence
Computer vision technology
Convolutional neural networks
Gesture recognition
Multimodal imaging
RELATED CONTENT
Subscribe to Digital Library
Receive Erratum Email Alert
Back to Top