Skip to main content
SpringerLink
Log in

Neural Network Strategies and Models for Voice Cloning in a Multi-speaker Mode: An Overview

  • Conference paper
  • First Online:
Proceedings of 7th ASRES International Conference on Intelligent Technologies (ICIT 2022)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 685))

Included in the following conference series:

  • 140 Accesses

Abstract

The evolution of data science and the constant challenge of carrying out different processes using a few resources with simultaneous personalization has promoted interest in the development of voice cloning. Nowadays, different machine learning techniques are used, given their efficiency in generating relationships across multiple parameters. In this regard, we evaluated the best-performing models and the different process optimization strategies within this sector, where through neural network models separated modularly by their functionality, it is possible to generate independent processes taking into account the most significant number of linguistic factors in the generation of the voice, thus obtaining significant results of a clear improvement in the whole process of synthesizing the voice of a target speaker.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 299.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Jia Y, Zhang Yu, Weiss RJ et al. (2019) Transfer learning from speaker verification to multispeaker text-to-speech synthesis. https://doi.org/10.48550/arXiv.1806.04558

  2. Dai D, Chen Y, Chen, L et al (2021) Cloning one's voice using very limited data in the wild. In: ICASSP 2022–2022 IEEE international conference on acoustics, speech and signal processing (ICASSP)

    Google Scholar 

  3. Cong J, Yang S, Xie, L et al (2020) Data efficient voice cloning from noisy samples with domain adversarial training. https://doi.org/10.48550/arXiv.2008.04265

  4. Ballesteros DM, Rodriguez-Ortega Y, Renza D, Arce G (2021) Deep4SNet: deep learning for fake speech classification. Exp Syst Appl 184:115465. https://doi.org/10.1016/j.eswa.2021.115465

  5. Zhang H, Lin Y (2022) Improve few-shot voice cloning using multi-modal learning. Comput Intell Nerosci 2022. https://doi.org/10.48550/arXiv.2203.09708

  6. Wu Q, Shen Q, Luan J, Wang Y (2022) Msdtron: a high-capability multi-speaker speech synthesis system for diverse data using characteristic Information. https://doi.org/10.48550/arXiv.2107.03065

  7. Kalchbrenner E, Elsen E, Simonyan E et al. (2018) Efficient neural audio synthesis. In: Proceedings of machine learning research. https://doi.org/10.48550/arXiv.1802.08435

  8. Corentin J (2019) Real-time voice cloning. http://hdl.handle.net/2268.2/6801

  9. Seong JW, Lee W, Lee S (2021) Multilingual speech synthesis for voice cloning. In: 2021 IEEE international conference on big data and smart computing (BigComp).https://doi.org/10.1109/BigComp51126.2021.00067

  10. Perez A, Garcés G, Diaz M et al. (2021) Towards cross-lingual voice cloning in higher education. Eng. Appl Artif Intell 105. https://doi.org/10.1016/j.engappai.2021.104413

  11. Luong HT, Yamagishi J (2020) NAUTILUS: a versatile voice cloning system. In: IEEE/ACM transactions on audio, speech, and language processing, https://doi.org/10.1109/TASLP.2020.3034994

  12. Rodriguez C, Angeles D, Chafloque R, Kaseng F, Pandey B (2020) Deep learning audio spectrograms processing to the early COVID-19 detection. In: 2020 12th international conference on computational intelligence and communication networks (CICN), Bhimtal, India, 2020, pp 429–434. https://doi.org/10.1109/CICN49253.2020.9242583

  13. Huang SF, Lin CJ, Lee HY (2021) Meta-TTS: meta-learning for few-shot speaker adaptive text-to-speech. https://doi.org/10.48550/arXiv.2111.04040

Download references

Author information

Authors and Affiliations

  1. Department of Software Engineering, Universidad Nacional Mayor de San Marcos UNMSM, Calle Germán Amézaga N° 375, 15081, Lima, Peru

    Marco Fura-Mendoza & Ciro Rodriguez

  2. Mechanical Technology Laboratory, Department of Mechanical and Electrical Engineering, Universidad de Piura UDEP, Av. Ramón Mugica 131, 20009, Piura, Peru

    Isabel Moscol-Albañil

  3. System Engineering Faculty, Universidad Nacional Federico Villarreal UNFV, Av. Óscar R. Benavides 450, 15082, Lima, Peru

    Ciro Rodriguez, Pedro Lezama & Yuri Pomachagua

  4. Faculty of Health Sciences, Universidad Peruana de Ciencias Aplicadas UPC, Av. Alameda San Marcos 11, Chorrillos, 15067, Lima, Peru

    Diego Rodriguez

Authors
  1. Marco Fura-Mendoza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Isabel Moscol-Albañil
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ciro Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pedro Lezama
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Diego Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuri Pomachagua
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ciro Rodriguez .

Editor information

Editors and Affiliations

  1. Indian Institute of Information Technology and Management, Gwalior, Gwalior, Madhya Pradesh, India

    Karm Veer Arya

  2. National Institute of Technology, Agartala, Agartala, Tripura, India

    Vipin Kumar Tripathi

  3. Universidad Nacional Mayor de San Marcos, Lima, Peru

    Ciro Rodriguez

  4. University of Pembangunan Jaya, Jakarta, Jakarta, Indonesia

    Eddy Yusuf

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fura-Mendoza, M., Moscol-Albañil, I., Rodriguez, C., Lezama, P., Rodriguez, D., Pomachagua, Y. (2023). Neural Network Strategies and Models for Voice Cloning in a Multi-speaker Mode: An Overview. In: Arya, K.V., Tripathi, V.K., Rodriguez, C., Yusuf, E. (eds) Proceedings of 7th ASRES International Conference on Intelligent Technologies. ICIT 2022. Lecture Notes in Networks and Systems, vol 685. Springer, Singapore. https://doi.org/10.1007/978-981-99-1912-3_21

Download citation

Publish with us

Policies and ethics

Search

Navigation