The effects of audio and video correlation and lip synchronization

This paper investigates the interaction effect of audio and video, and studies lip synchronization (lip sync). The study shows a comprehensive evaluation of achievable audio and video quality undertaken based upon different sets of impairments between audio and video, prior to transmission. The tests have been conducted on two different task scenarios, i.e. passive communication and interactive communication (person to person). The research concentrates on quantifying the effects of network impairments (packet loss) on perceived audio and video quality, as well as finding the correlations between audio and video in multimedia applications. The results presented in this paper show the strong interaction dependency between audio and video. It was justified that video has a unique benefit on multimedia quality for its psychological effects. The findings also concluded that the sensory interactions, and the attention given to a particular aspect of performance, are clearly content‐dependent.