Tracking methods based on correlation filter have been showing great advantages and huge potential. However, it will lead to tracking drift when the target is disturbed, such as occlusion, illumination variation, deformation, and others. In the pursuit of tracking accuracy and robustness, a feature integration model and an adaptive update method for robust tracking are proposed. First, the fine-tuned convolutional neural networks medium is employed to obtain the deep features. The deep features are integrated with traditional manual features by combining with the independent component analysis with reference method to obtain more discriminative features. Second, in the model update phase, an adaptive updating strategy based on the center shift Euclidean distance of image patch is proposed to reduce the unnecessary calculation in model training so as to speed up the tracking speed. Finally, the tracker proposed is evaluated on online tracking benchmark (OTB-2015) and visual object tracking benchmark (VOT-2016). The experimental results show that the method that integrates deep features with traditional manual features can distinguish background and target better. Thus the proposed VOT algorithm performs better robustness and accuracy against state-of-the-art methods when dealing with challenging environments.