Facial expression recognition (FER) in an uncontrolled environment is difficult due to changes in occlusion, illumination, noise, and personal attributes. A deep learning enhanced gender conditional random forest (G_DRF) is proposed for FER in an uncontrolled environment. In order to reduce the influence of occlusion, illumination, low image resolution, etc., our method extracts robust facial features by deep multi-instance learning. Then a G_DRF is devised to address the facial personal attributes’ influence, such as gender variation by conditional RF. A large number of experiments were conducted on the public CK+, BU-3DEF, and LFW face databases. The experimental results showed that the proposed method had better performance and robustness than the state-of-the-art methods. The recognition rates of CK+, BU-3DEF, and LFW were 98.83%, 90%, 60.58%, respectively. Compared with other advanced deep learning methods that require a large number of training samples, the proposed method needs a small number of training samples and achieves better results.