We introduce VoiceFilter-Lite, a single-channel source separation model that runs on the device to preserve only the speech signals from a target user, as part of a streaming speech recognition system. Delivering such a model presents numerous challenges: It should improve the performance when the input signal consists of overlapped speech, and must not hurt the speech recognition performance under all other acoustic conditions. Besides, this model must be tiny, fast, and perform inference in a streaming fashion, in order to have minimal impact on CPU, memory, battery and latency. We propose novel techniques to meet these multi-faceted requirements, including using a new asymmetric loss, and adopting adaptive runtime suppression strength. We also show that such a model can be quantized as a 8-bit integer model and run in realtime.
Cite as: Wang, Q., Moreno, I.L., Saglam, M., Wilson, K., Chiao, A., Liu, R., He, Y., Li, W., Pelecanos, J., Nika, M., Gruenstein, A. (2020) VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition. Proc. Interspeech 2020, 2677-2681, doi: 10.21437/Interspeech.2020-1193
@inproceedings{wang20z_interspeech,
author={Quan Wang and Ignacio Lopez Moreno and Mert Saglam and Kevin Wilson and Alan Chiao and Renjie Liu and Yanzhang He and Wei Li and Jason Pelecanos and Marily Nika and Alexander Gruenstein},
title={{VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition}},
year=2020,
booktitle={Proc. Interspeech 2020},
pages={2677--2681},
doi={10.21437/Interspeech.2020-1193}
}