Objects detection from an image is one of the basic problems of study in computer vision. In particular, moving objects detection is important, as they may be those to which one should pay attention in, e.g., walking or driving. This paper proposes a method of detecting moving objects by inferring backgrounds frame by frame. The proposed method can cope with dynamical change of the background including sudden illumination change. Experimental results show satisfactory performance of the proposed method.