Dysarthria is a speech disorder that mostly occurs as a symptom of neurodegenerative and other neuromuscular diseases. The speech of patients with dysarthria becomes distorted, the articulation of phonemes (especially that of consonants) is poor, the intelligibility and naturalness are impaired. Because dysarthria is progressive (similarly to the other symptoms of the main disease), patients may have difficulties using speech-controlled Ambient Assisted Living systems that could be a great help for them in daily life. In this paper, an adaptive speech recognition framework is introduced that is able to handle gradually occurring changes in the speech quality of the user. The presented technique can adapt to these changes while the speech interpretation accuracy of the system will not decrease, even in cases of noisy or incorrect training data.