Robust Self Supervised Speech Embeddings for Child-Adult Classification in Interactions involving Children with Autism

Rimita Lahiri, Tiantian Feng, Rajat Hebbar, Catherine Lord, So Hyun Kim, Shrikanth Narayanan

We address the problem of detecting who spoke when in child-inclusive spoken interactions i.e., automatic child-adult speaker classification. Interactions involving children are richly heterogeneous due to developmental differences. The presence of neurodiversity e.g., due to Autism, contributes additional variability. We investigate the impact of additional pre-training with more unlabelled child speech on the child-adult classification performance. We pre-train our model with child-inclusive interactions, following two recent self-supervision algorithms, Wav2vec 2.0 and WavLM, with a contrastive loss objective. We report 9-13% relative improvement over the state-of-the-art baseline with regards to classification F1 scores on two clinical interaction datasets involving children with Autism. We also analyze the impact of pre-training under different conditions by evaluating our model on interactions involving different subgroups of children based on various demographic factors.

doi: 10.21437/Interspeech.2023-1447

Cite as: Lahiri, R., Feng, T., Hebbar, R., Lord, C., Kim, S.H., Narayanan, S. (2023) Robust Self Supervised Speech Embeddings for Child-Adult Classification in Interactions involving Children with Autism. Proc. INTERSPEECH 2023, 3557-3561, doi: 10.21437/Interspeech.2023-1447

@inproceedings{lahiri23_interspeech,
  author={Rimita Lahiri and Tiantian Feng and Rajat Hebbar and Catherine Lord and So Hyun Kim and Shrikanth Narayanan},
  title={{Robust Self Supervised Speech Embeddings for Child-Adult Classification in Interactions involving Children with Autism}},
  year=2023,
  booktitle={Proc. INTERSPEECH 2023},
  pages={3557--3561},
  doi={10.21437/Interspeech.2023-1447}
}