Skip to main content
SpringerLink
Log in

Non-linear target trajectory prediction for robust visual tracking

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The occlusion of the target in tracking usually causing failure that is a serious issue for long-term sequences. Though the siamese network-based trackers obtained considerable performance, it still suffers from the target missing issue when the target is obscured by the same semantic information interferent. To address this issue, a novel occlusion awareness algorithm is proposed, which can both address the occlusion issue and the same semantic information false identification issues. In addition, a novel generative adversarial training and long short term memory (LSTM) based target trajectory prediction algorithm is proposed to predict the possible direction of the target in the following frames. The proposed trajectory prediction algorithm can deal with complicated tracking situations more robustly than the traditional algorithms, e.g. Kalman filter. To further improve the occlusion awareness ability of the proposed algorithm, an occlusion supervision-based training strategy is proposed, which can improve the robustness of the occlusion awareness ability of the proposed occlusion awareness model. In addition, for accurate estimation of the target bounding box, a distance intersection over union (DIOU) loss for regression training is adopted. A comprehensive evaluation is performed on OTB2015, VOT2016, and VOT2018 to evaluate the effectiveness of the proposed algorithm. The experiment results demonstrate that the proposed algorithms perform well and can largely alleviate the tracking failure issue of the siamese network-based tracker caused by occlusion and the same semantic information target identification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Liu T, Liu H, Li Y, Zhang Z, Liu S (2018) Efficient blind signal reconstruction with wavelet transforms regularization for educational robot infrared vision sensing. IEEE/ASME Trans Mechatron 24(1):384–394

    Article  Google Scholar 

  2. Liu T, Liu H, Li Y-F, Chen Z, Zhang Z, Liu S (2019) Flexible ftir spectral imaging enhancement for industrial robot infrared vision sensing. IEEE Trans Ind Inf 16(1):544–554

    Article  Google Scholar 

  3. Zhang Z, Lai C, Liu H, Li Y-F (2020) Infrared facial expression recognition via gaussian-based label distribution learning in the dark illumination environment for human emotion detection. Neurocomputing 409:341–350

    Article  Google Scholar 

  4. Liu H, Fang S, Zhang Z, Li D, Lin K, Wang J Mfdnet: Collaborative poses perception and matrix fisher distribution for head pose estimation. IEEE Transactions on Multimedia

  5. Cui Z, Lu N (2021) Feature selection accelerated convolutional neural networks for visual tracking. Appl Intell:1–15

  6. Fan H, Lin L, Yang F, Chu P (2019) Etc., Lasot A high-quality benchmark for large-scale single object tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 5374–5383

  7. Kristan M, Leonardis A, Matas J (2016) etc., The visual object tracking VOT2016 challenge results. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), vol. 9914 LNCS. Springer, Cham, pp 777–823

  8. Kristan M, Leonardis A (2019) J. matas, etc., The sixth visual object tracking VOT2018 challenge results. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), vol. 11129 LNCS. Springer, Cham, pp 3–53

  9. Gao L, Liu B, Fu P, Xu M, Li J (2021) Visual tracking via dynamic saliency discriminative correlation filter. Appl Intell:1–15

  10. Danelljan M, Bhat G, Shahbaz Khan F, Felsberg M (2017) Eco: Efficient convolution operators for tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6638–6646

  11. Danelljan M, Hager G, Shahbaz Khan F, Felsberg M (2015) Learning spatially regularized correlation filters for visual tracking. In: Proceedings of the IEEE international conference on computer vision, pp 4310–4318

  12. Nam H, Han B (2016) Learning multi-domain convolutional neural networks for visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4293–4302

  13. Zeng Y, Zeng B, Yin X, Chen G (2021) Siampcf: siamese point regression with coarse-fine classification network for visual tracking. Appl Intell:1–14

  14. Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PH (2016) Fully-convolutional siamese networks for object tracking. In: European conference on computer vision. Springer, pp 850–865

  15. Li B, Yan J, Wu W, Zhu Z, Hu X (2018) High performance visual tracking with siamese region proposal network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8971–8980

  16. Zhang Z, Peng H (2019) Deeper and wider siamese networks for real-time visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 4591–4600

  17. Xu L, Wei Y, Dong C, Xu C, Diao Z (2021) Wasserstein distance-based auto-encoder tracking. Neural Process Lett 53(3):2305–2329

    Article  Google Scholar 

  18. Wu Y, Lim J, Yang M-H (2015) Object tracking benchmark. IEEE Trans Pattern Anal Mach Intell 37(09):1834–1848

    Article  Google Scholar 

  19. Held D, Thrun S, Savarese S (2016) Learning to track at 100 fps with deep regression networks. In: European conference on computer vision. Springer, pp 749–765

  20. Liu H, Nie H, Zhang Z, Li Y. -F. (2021) Anisotropic angle distribution learning for head pose estimation and attention understanding in human-computer interaction. Neurocomputing 433:310–322

    Article  Google Scholar 

  21. Li Z, Liu H, Zhang Z, Liu T, Xiong NN Learning knowledge graph embedding with heterogeneous relation attention networks, IEEE Transactions on Neural Networks and Learning Systems

  22. Zhang Z, Li Z, Liu H, Xiong NN Multi-scale dynamic convolutional network for knowledge graph embedding, IEEE Transactions on Knowledge and Data Engineering

  23. Zhu Z, Wang Q, Li B, Wu W, Yan J, Hu W (2018) Distractor-aware siamese networks for visual object tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 101–117

  24. Li B, Wu W, Wang Q, Zhang F, Xing J, Yan J (2019) Siamrpn++: Evolution of siamese visual tracking with very deep networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 4282–4291

  25. Voigtlaender P, Luiten J, Torr PH, Leibe B (2020) Siam r-cnn: Visual tracking by re-detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 6578–6588

  26. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  27. Wang N, Zhou W, Wang J, Li H (2021) Transformer meets tracker: Exploiting temporal context for robust visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 1571–1580

  28. Yan B, Peng H, Fu J, Wang D, Lu H Learning spatio-temporal transformer for visual tracking, arXiv:2103.17154

  29. Danelljan M, Bhat G, Khan FS, Felsberg M (2019) Atom: Accurate tracking by overlap maximization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 4660–4669

  30. Jiang B, Luo R, Mao J, Xiao T, Jiang Y (2018) Acquisition of localization confidence for accurate object detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 784–799

  31. Zhang Z, Peng H, Fu J, Li B, Hu W (2020) Ocean: Object-aware anchor-free tracking. In: Computer vision–ECCV 2020: 16th european conference, Proceedings, Part XXI 16, Springer, Glasgow, pp 771–787

  32. Xu Y, Wang Z, Li Z, Yuan Y, Yu G (2020) Siamfc++: Towards robust and accurate visual tracking with target estimation guidelines. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 12549–12556

  33. Guo D, Wang J, Cui Y, Wang Z, Chen S (2020) Siamcar: Siamese fully convolutional classification and regression for visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 6269– 6277

  34. Chen Z, Zhong B, Li G, Zhang S, Ji R (2020) Siamese box adaptive network for visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 6668–6677

  35. Cui Y, Jiang C, Wang L, Wu G Target transformed regression for accurate tracking, arXiv:2104.00403

  36. Zheng Z, Wang P, Liu W, Li J, Ye R, Ren D (2020) Distance-iou loss: Faster and better learning for bounding box regression. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 12993–13000

  37. Gupta A, Johnson J, Fei-Fei L, Savarese S, Alahi A (2018) Social gan: Socially acceptable trajectories with generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2255–2264

  38. Wang Q, Teng Z, Xing J, Gao J, Hu W, Maybank S (2018) Learning attentions: residual attentional siamese network for high performance online visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4854–4863

  39. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  40. Huang L, Zhao X, Huang K (2021) Got-10k: a large high-diversity benchmark for generic object tracking in the wild. IEEE Trans Pattern Anal Mach Intell 43(5):1562–1577

    Article  Google Scholar 

  41. Real E, Shlens J, Mazzocchi S, Pan X, Vanhoucke V (2017) Youtube-boundingboxes: a large high-precision human-annotated data set for object detection in video. In: proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5296–5305

  42. Danelljan M, Robinson A, Khan FS, Felsberg M (2016) Beyond correlation filters: Learning continuous convolution operators for visual tracking. In: European conference on computer vision. Springer, pp 472–488

  43. Bhat G, Danelljan M, Gool LV, Timofte R (2019) Learning discriminative model prediction for tracking. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 6182–6191

  44. Danelljan M, Hager G, Shahbaz Khan F, Felsberg M (2015) Convolutional features for correlation filter based visual tracking. In: Proceedings of the IEEE international conference on computer vision workshops, pp 58–66

Download references

Acknowledgements

This work is supported by National Nature Science Foundation of China (grant No.61871106 and No.61370152), Key R & D projects of Liaoning Province, China (grant No. 2020JH2/10100029), and the Open Project Program Foundation of the Key Laboratory of Opto-Electronics Information Processing, Chinese Academy of Sciences (OEIP-O-202002).

Author information

Authors and Affiliations

  1. College of Information Science, Northeastern University, Shenyang, 110179, China

    Long Xu, Zhaofu Diao & Ying Wei

  2. Information Technology R & D Innovation Center of Peking University, Shaoxing, China

    Ying Wei

Authors
  1. Long Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhaofu Diao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ying Wei.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, L., Diao, Z. & Wei, Y. Non-linear target trajectory prediction for robust visual tracking. Appl Intell 52, 8588–8602 (2022). https://doi.org/10.1007/s10489-021-02829-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02829-x

Keywords

Search

Navigation