Pitch processing in language and music is generally considered to engage overlapped neural correlates. Previous studies on musicians showed that the ability of pitch processing in music could be transferred to language. It is known that music training can facilitate neural processing of speech.  However, the underlying neural mechanisms of pitch processing in language and music are not fully understood, especially in non-musicians. Using magnetoencephalography (MEG), we presented a pitch anomaly paradigm which consists of language/music phrases ending in either congruous or incongruous tone/pitch to non-musicians. We found distinctive brain activity patterns between two groups of participants with varying (high vs. low) musical pitch perceptual abilities. The brain-behavior results showed a positive correlation between performance of musical pitch tasks and activation of the left frontotemporal cortical regions elicited by lexical tones. Our results suggested that the cross-domain effect of language and music could be generalized to people without formal music training.