Abstract
Today’s robotic technologies are not only used in industries but are also capable of making an impact in our day-to-day activities. Therapy robots have been playing a very prominent role in today’s modern world, so we have planned to create a robotic arm that could help kids with disabilities to draw along and build self-confidence in them. For this purpose, we train a co-creative robotic arm that processes the input in the form of speech, sketch and update the image according to the user given feedback. We have used Google’s quick draw data set as an initial training data for the robotic arm. The feedback can be given by the kids in the form of voice, as the sketches are drawn by the robotic arm that can then be trained to suit the needs of the kid and make more creative and personalized drawings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Miner, A., Chow, A., Adler, S., Zaitsev, I., Tero, P., Darcy, A., Paepcke, A.: Conversational agents and mental health: theory-informed assessment of language and affect. In: Proceeding of HAI 2016, Proceedings of the Fourth International Conference on Human Agent Interaction, Biopolis, Singapore, 04–07 October 2016, pp. 123–130 (2016)
Ho, A., Hancock, J., Miner, A.: Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. J. Commun. 68(4), 712–733 (2018)
Edwards, C., Edwards, A., Spence, P.R., Lin, X.: I, teacher: using artificial intelligence (AI) and social robots in communication and instruction. Commun. Educ. 67(4), 473–480 (2018)
Saadatzi, M.N., Pennington, R.C., Welch, K.C., Graham, J.H.: Small-group technology-assisted instruction: virtual teacher and robot peer for individuals with autism spectrum disorder. J. Autism Dev. Disord. (2018). https://doi.org/10.1007/s10803-018-3654-2. Accessed 29 Sept 2018
Kural, O.E., Kiliç, E.: Machine learning based interactive drawing platform. In: 25th Signal Processing and Communications Applications Conference (SIU), Antalya, Turkey (2017)
Google Quick, Draw! Game. https://quickdraw.withgoogle.com/. Accessed 29 Sept 2018
Recurrent Neural Networks for Drawing Classification, PyTorch Tutorials. https://www.tensorflow.org/tutorials/sequences/recurrent_quickdraw. Accessed 28 Sept 2018
Google quick draw dataset. https://github.com/googlecreativelab/quickdraw-dataset. Accessed 29 Sept 2018
Finkel, J.R., Grenager, T., Manning, C.: Incorporating non-local information into information extraction systems by Gibbs sampling. In: Proceedings of the 43nd Annual Meeting of the Association for Computational Linguistics (ACL 2005), pp. 363–370 (2005). http://nlp.stanford.edu/~manning/papers/gibbscrf3.pdf. Accessed 29 Sept 2018
Google word2vec. https://code.google.com/archive/p/word2vec/. Accessed 29 Sept 2018
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Paszke, A.: Reinforcement Learning (DQN) tutorial, PyTorch Tutorials. https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html. Accessed 29 Sept 2018
Duan, Y., Andrychowicz, M., Stadie, B.C., Ho, J., Schneider, J., Sutskever, I., Abbeel, P., Zaremba, W.: One-shot imitation learning. CoRR, abs/1703.07326 (2017). http://arxiv.org/abs/1703.07326. Accessed 29 Sept 2018
Yu, T., Finn, C., Xie, A., Dasari, S., Zhang, T., Abbeel, P., Levine, S.: One-shot imitation from observing humans via domain-adaptive meta-learning. CoRR, abs/1802.01557 (2018). http://arxiv.org/abs/1802.01557. Accessed 29 Sept 2018
Natural Language Toolkit. https://www.nltk.org/. Accessed 29 Sept 2018
Google Cloud Speech-to-Text API. https://cloud.google.com/speech-to-text/. Accessed 29 Sept 2018
Demonstration of Co-Creative Robot. https://www.youtube.com/watch?v=tdAjWc45WLs. Accessed 29 Sept 2018
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Shaik, S.Z., Srinivasan, V., Peng, Y., Lee, M., Davis, N. (2021). Co-creative Robotic Arm for Differently-Abled Kids: Speech, Sketch Inputs and External Feedbacks for Multiple Drawings. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Proceedings of the Future Technologies Conference (FTC) 2020, Volume 3. FTC 2020. Advances in Intelligent Systems and Computing, vol 1290. Springer, Cham. https://doi.org/10.1007/978-3-030-63092-8_68
Download citation
DOI: https://doi.org/10.1007/978-3-030-63092-8_68
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63091-1
Online ISBN: 978-3-030-63092-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)