Skip to main content
SpringerLink
Log in

A Speech Enhancement Method Combining Two-Branch Communication and Spectral Subtraction

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1792))

Included in the following conference series:

  • 587 Accesses

Abstract

Time-Frequency (T-F) domain masking is currently the dominant method for single-channel speech enhancement, while little attention has been paid to phase information. A speech enhancement method, named PHASEN-SS, is proposed in this paper. Our method is divided into two steps, first a deep neural network (DNN) with two-branch communication using a combination of mask and phase for speech enhancement, and then a data post-processing after the DNN processes the noisy speech. PHASEN-SS uses two branches to predict the amplitude mask and the phase separately, which improves the accuracy of prediction by exchanging information between two branches, and then further the enhancement by denoising the residual noise through spectral subtraction. The experiments are conducted on the publicly available Voice Bank + DEMAND dataset, as well as a noisy speech dataset is synthesized with 4 common noises in Noise92 and Voice Bank clean speech according to the specified signal-to-noise ratio (SNR). The results show that the proposed method improves on the original one, and has better robustness to speech containing babble noise at higher SNRs for different SNRs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Berouti, M, Schwartz, R., Makhoul, J.: Enhancement of speech corrupted by acoustic noise. In: ICASSP IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 4, pp. 208–211 (1979)

    Google Scholar 

  2. Lim, J., Oppenheim, A.: All-pole modeling of degraded speech. IEEE Trans. Acoust. Speech Signal Process. 26(3), 197–210 (1978)

    Article  MATH  Google Scholar 

  3. Ephraim Y.: Statistical-model-based speech enhancement systems. In: Proceedings of the IEEE, vol. 80, no. 10, pp. 1526–1555 (1992)

    Google Scholar 

  4. Dendrinos, M., Ba Kamidis, S.G., Carayannis, G.: Speech enhancement from noise: a regenerative approach. Speech Commun. 10(1), 45–57 (1991)

    Article  Google Scholar 

  5. Ephraim, Y., Trees, H.V.: A signal subspace approach for speech enhancement. IEEE Trans. Speech Audio Process. 3(4), 251–266 (1995)

    Article  Google Scholar 

  6. Tamura, S., Waibel, A.: Noise reduction using connectionist models. In: ICASSP, pp. 553–556 (1988)

    Google Scholar 

  7. Parveen, S., Green, P.: Speech enhancement with missing data techniques using recurrent neural networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), pp. 733–736 (2004)

    Google Scholar 

  8. Lu, X.G., Tsao, Y., Matsuda, S., et al.: Speech enhancement based on deep denoising autoencoder. In: Conference of the International Speech Communication Association, ISCA, pp. 436–440 (2013)

    Google Scholar 

  9. Pascual, S., Bonafonte, A., Serrà, J.: SEGAN: speech enhancement generative adversarial network. Interspeech, 3642–3646 (2017)

    Google Scholar 

  10. Abdulatif, S., Armanious, K., Guirguis, K., et al.: Aegan: time-frequency speech denoising via generative adversarial networks. EUSIPCO, pp. 451–455 (2020)

    Google Scholar 

  11. Pan, Q., Gao, T., Zhou, J., et al.: CycleGAN with dual adversarial loss for bone-conducted speech enhancement. CoRR.2021:2111.01430

    Google Scholar 

  12. Yasuda, M., Koizumi, Y., Mazzon, L., et al.: DOA estimation by DNN-based denoising and dereverberation from sound intensity vector. CORR.2019:1910.04415

    Google Scholar 

  13. Yasuda, M., Koizumi, Y., Saito, S., et al.: Sound event localization based on sound intensity vector refined by DNN-based denoising and source separation. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 651–655 (2020)

    Google Scholar 

  14. Le, X., Chen, H., Chen, K., et al.: DPCRN: dual-path convolution recurrent network for single channel speech enhancement. In: Interspeech, pp. 2811–2815 (2021)

    Google Scholar 

  15. Pandey, A., Wang, D.: Dense CNN with self-attention for time-domain speech enhancement. IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 1270–1279 (2021)

    Google Scholar 

  16. Jansson, A., Sackfield, A.W., Sung, C.C.: Singing voice separation with deep u-net convolutional networks: US20210256994A1 (2021)

    Google Scholar 

  17. Williamson, D.S., Wang, Y., Wang, D.L.: Complex ratio masking for monaural speech separation. IEEE/ACM Trans. Audio Speech Lang. Process. 24(3), 483–492 (2016)

    Article  Google Scholar 

  18. Yin, D., Luo, C., Xiong, Z., et al.: Phasen: a phase-and-harmonics-aware speech enhancement network. In: Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence (AAAI).2020: 9458–9465

    Google Scholar 

  19. Hu, G., Wang, D.L.: Speech segregation based on pitch tracking and amplitude modulation. In: IEEE Workshop on Applications of Signal Processing to Audio & Acoustics, pp. 553–556 (2002)

    Google Scholar 

  20. Srinivasan, S., Roman, N., Wang, D.L.: Binary and ratio time-frequency masks for robust speech recognition. Speech Commun. 48(11), 1486–1501 (2006)

    Article  Google Scholar 

  21. Wang, Y., Narayanan, A., Wang, D.L.: On training targets for supervised speech separation. IEEE/ACM Trans. Audio Speech Lang. Process. 22(12), 1849–1858 (2014)

    Article  Google Scholar 

  22. Paliwal, K., Wójcicki, K., Shannon, B.J.: The importance of phase in speech enhancement. Speech Commun. 53(4), 465–494 (2011)

    Article  Google Scholar 

  23. Erdogan, H., Hershey, J.R., Watanabe, S., et al.: Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 708–712 (2015)

    Google Scholar 

  24. Selvaraj, P., Eswaran, C.: Ideal ratio mask estimation using supervised DNN approach for target speech signal enhancement. 42(3), 1869–1883 (2021)

    Google Scholar 

  25. Zhou, L., Jiang, W., Xu, J., et al.: Masks fusion with multi-target learning for speech enhancement. Electr. Eng. Syst. Sci. arXiv e-prints (2021)

    Google Scholar 

  26. Zhang, L., Wang, M., Zhang, Z., et al.: Deep interaction between masking and mapping targets for single-channel speech enhancement. CORR.2021:2106.04878

    Google Scholar 

  27. Choi, H.S., Kim, J.H., Huh, J., et al.: Phase-aware speech enhancement with deep complex U-Net In: ICLR. 2019:1903.03107

    Google Scholar 

  28. Routray, S., Mao, Q.: Phase sensitive masking-based single channel speech enhancement using conditional generative adversarial network. Comput. Speech Lang. 71, 101270 (2021)

    Article  Google Scholar 

  29. Takahashi, N., Agrawal, P., Goswami, N., et al.: PhaseNet: discretized phase modeling with deep neural networks for audio source separation. In: Interspeech, pp. 2713–2717 (2018)

    Google Scholar 

  30. Takamichi, S., Saito, Y., Takamune, N., et al.: Phase reconstruction from amplitude spectrograms based on von-Mises-distribution deep neural network. In: IEEE 2018 16th International Workshop on Acoustic Signal Enhancement (IWAENC), pp. 286–290 (2018)

    Google Scholar 

  31. Masuyama, Y., Yatabe, K., Koizumi, Y., et al.: Deep griffin-lim iteration. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 61–65 (2019)

    Google Scholar 

  32. Phan, H., Nguyen, H.L., Chen, O.Y., et al.: Self-attention generative adversarial network for speech enhancement. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7103–7107 (2021)

    Google Scholar 

  33. Soni, M.H., Shah, N., Patil, H.A.: Time-frequency masking-based speech enhancement using generative adversarial network. In: ICASSP, pp. 5039–5043 (2018)

    Google Scholar 

  34. Kim, J.H., Yoo, J., Chun, S., et al.: Multi-domain processing via hybrid denoising networks for speech enhancement. CoRR.2018:1812.08914

    Google Scholar 

  35. Valentini-Botinhao, C., Wang, X., Takaki, S., et al.: Investigating RNN-based speech enhancement methods for noise-robust Text-to-Speech. In: 9th ISCA Speech Synthesis Workshop, SSW, pp. 146–152 (2016)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61170093).

Author information

Authors and Affiliations

  1. Hubei Provincial Engineering Research Center for Intelligent Textile and Fashion, Wuhan, 430200, China

    Ruhan He, Yajun Tian, Zhenghao Chang & Mingfu Xiong

  2. School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan, 430200, China

    Ruhan He, Yajun Tian, Zhenghao Chang & Mingfu Xiong

  3. State Key Laboratory of Silicate Materials for Architectures, Wuhan University of Technology, Wuhan, 430070, China

    Yongsheng Yu

Authors
  1. Ruhan He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yajun Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yongsheng Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhenghao Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mingfu Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yajun Tian .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Indore, Indore, India

    Mohammad Tanveer

  2. Indian Institute of Information Technology - Allahabad, Prayagraj, India

    Sonali Agarwal

  3. Kobe University, Kobe, Japan

    Seiichi Ozawa

  4. Indian Institute of Technology Patna, Patna, India

    Asif Ekbal

  5. University of Innsbruck, Innsbruck, Austria

    Adam Jatowt

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

He, R., Tian, Y., Yu, Y., Chang, Z., Xiong, M. (2023). A Speech Enhancement Method Combining Two-Branch Communication and Spectral Subtraction. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Communications in Computer and Information Science, vol 1792. Springer, Singapore. https://doi.org/10.1007/978-981-99-1642-9_10

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-1642-9_10

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-1641-2

  • Online ISBN: 978-981-99-1642-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation