Skip to main content
SpringerLink
Log in

Minimax Monte Carlo object tracking

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We propose a new approach for visual object tracking based on a combined method of minimax estimator and sequential Monte Carlo filtering. The proposed approach adopts a minimax strategy in the standard particle filtering framework for the problem. Particle filtering is based on probabilistic methodology, while a minimax estimator belongs to deterministic approaches. Experiments show outperforming results of the proposed approach compared to the standard particle filtering in terms of tracking accuracy. We also investigate the computational complexity of the proposed algorithm in terms of elapsed processing time. In this paper, we focus on the particle filtering framework only for the performance comparison between the two methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Notes

  1. Last two video frames employed for experiments in this paper were downloaded from http://www.visual-tracking.net.

References

  1. Agarwal, A., Suryavanshi, S.: Real-time* multiple object tracking (mot) for autonomous navigation. Technical report (2017)

  2. Andrews, T.: Computation time comparison between matlab and c++ using launch windows (2012)

  3. Arulampalam, M., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002)

    Article  Google Scholar 

  4. Avidan, S.: Support vector tracking. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 1064–1072 (2004)

    Article  Google Scholar 

  5. Barina, D.: Gabor wavelets in image processing. arXiv preprint. arXiv:1602.03308 (2016)

  6. Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)

    Article  Google Scholar 

  7. Bhat, P.G., Subudhi, B.N., Veerakumar, T., Laxmi, V., Gaur, M.S.: Multi-feature fusion in particle filter framework for visual tracking. IEEE Sens. J. 20(5), 2405–2415 (2019)

    Article  Google Scholar 

  8. Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: optimal speed and accuracy of object detection. arXiv preprint. arXiv:2004.10934 (2020)

  9. Bu, F., Cai, Y., Yang, Y.: Multiple object tracking based on faster-RCNN detector and KCF tracker. Technical Report [Online]. https://pdfs.semanticscholar.org (2016)

  10. Chahyati, D., Fanany, M.I., Arymurthy, A.M.: Tracking people by detection using CNN features. Procedia Comput. Sci. 124, 167–172 (2017)

    Article  Google Scholar 

  11. Choi, J., Kwon, J., Lee, K.M.: Visual tracking by reinforced decision making (2017)

  12. Comaniciu, D., Ramesh, V., Meer, P.: Real-time tracking of non-rigid objects using mean shift. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 142–149. IEEE (2000)

  13. Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(5), 564–577 (2003)

    Article  Google Scholar 

  14. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 886–893. IEEE (2005)

  15. Danelljan, M., Bhat, G., Shahbaz Khan, F., Felsberg, M.: Eco: efficient convolution operators for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6638–6646 (2017)

  16. Danelljan, M., Hager, G., Shahbaz Khan, F., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4310–4318 (2015)

  17. Danelljan, M., Robinson, A., Khan, F.S., Felsberg, M.: Beyond correlation filters: learning continuous convolution operators for visual tracking. In: European Conference on Computer Vision, pp. 472–488. Springer (2016)

  18. Engineer, P., Velmurugan, R., Patkar, S.: Scalable implementation of particle filter-based visual object tracking on network-on-chip (noc). J. Real-Time Image Proc. 17(5), 1117–1134 (2020)

    Article  Google Scholar 

  19. Fotouhi, M., Gholami, A., Kasaei, S.: Particle filter-based object tracking using adaptive histogram. In: 7th Iranian Conference on Machine Vision and Image Processing, pp. 1–5. IEEE (2011)

  20. Gunjal, P.R., Gunjal, B.R., Shinde, H.A., Vanam, S.M., Aher, S.S.: Moving object tracking using Kalman filter. In: International Conference On Advances in Communication and Computing Technology (ICACCT), pp. 544–547. IEEE (2018)

  21. Ha, N.D., Shimizu, I., et al.: Tracking objects based on multiple particle filters for multipart combined moving directions information. Comput. Intell. Neurosci. (2020). https://doi.org/10.1155/2020/8839725

    Article  Google Scholar 

  22. Hariyono, J., Hoang, V.D., Jo, K.H.: Moving object localization using optical flow for pedestrian detection from a moving vehicle. Sci. World J. (2014). https://doi.org/10.1155/2014/196415

    Article  Google Scholar 

  23. Harmon, M.E., Harmon, S.S.: Reinforcement learning: a tutorial. Technial report, WRIGHT LAB WRIGHT-PATTERSON AFB OH (1997)

  24. Held, D., Thrun, S., Savarese, S.: Learning to track at 100 fps with deep regression networks. In: European Conference on Computer Vision, pp. 749–765. Springer (2016)

  25. Kailath, T.: The divergence and Bhattacharyya distance measures in signal selection. IEEE Trans. Commun. Technol. 15(1), 52–60 (1967)

    Article  Google Scholar 

  26. Kay, S.M.: Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory. Prentice Hall Signal Processing Series, Prentice-Hall Inc, Hoboken (1993)

    MATH  Google Scholar 

  27. Kim, D.S., Kwon, J.: Moving object detection on a vehicle mounted back-up camera. Sensors 16(1), 23 (2016)

    Article  Google Scholar 

  28. Kwame, A.B., Liu, G., Liu, H., Hussain, F.: Real-time recognition and tracing of moving objects in video images using background subtraction, Kalman filter and particle filter. In: 25th International Conference on Automation and Computing (ICAC), pp. 1–5. IEEE (2019)

  29. Li, X., Zheng, N.: Adaptive target color model updating for visual tracking using particle filter. In: IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No. 04CH37583), vol. 4, pp. 3105–3109. IEEE (2004)

  30. Li, Y., Ai, H., Yamashita, T., Lao, S., Kawade, M.: Tracking in low frame rate video: a cascade particle filter with discriminative observers of different life spans. IEEE Trans. Pattern Anal. Mach. Intell. 30(10), 1728–1740 (2008)

    Article  Google Scholar 

  31. Lim, J., Kim, H.S., Park, H.M.: Minimax particle filtering for tracking a highly maneuvering target. Int. J. Robust Nonlinear Control 30(2), 636–651 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  32. Lim, J., Park, H.M.: Minimax particle filtering for a highly maneuvering target tracking. Nonlinear Dynamics (under review and revision)

  33. Luo, W., Sun, P., Zhong, F., Liu, W., Zhang, T., Wang, Y.: End-to-end active object tracking via reinforcement learning. In: International Conference on Machine Learning, PMLR, pp. 3286–3295 (2018)

  34. Malisiewicz, T., Gupta, A., Efros, A.A.: Ensemble of exemplar-svms for object detection and beyond. In: International conference on computer vision, pp. 89–96. IEEE (2011)

  35. Mihaylova, L., Brasnett, P., Canagarajah, N., Bull, D.: Object tracking by particle filtering techniques in video sequences. Advances and challenges in multisensor data and information processing 8, 260–268 (2007)

    Google Scholar 

  36. Najafzadeh, N., Fotouhi, M., Kasaei, S.: Object tracking using Kalman filter with adaptive sampled histogram. In: 23rd Iranian Conference on Electrical Engineering, pp. 781–786. IEEE (2015)

  37. Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4293–4302 (2016)

  38. Nordsjo, A.E.: A constrained extended Kalman filter for target tracking. In: Proceedings of the 2004 IEEE Radar Conference (IEEE Cat. No. 04CH37509), pp. 123–127. IEEE (2004)

  39. Nummiaro, K., Koller-Meier, E., Van Gool, L.: An adaptive color-based particle filter. Image Vis. Comput. 21(1), 99–110 (2003)

    Article  MATH  Google Scholar 

  40. Ojala, T., Pietikainen, M., Harwood, D.: Performance evaluation of texture measures with classification based on kullback discrimination of distributions. In: Proceedings of 12th International Conference on Pattern Recognition, vol. 1, pp. 582–585. IEEE (1994)

  41. Pal, S.K., Pramanik, A., Maiti, J., Mitra, P.: Deep learning in multi-object detection and tracking: state of the art. Appl. Intell. 51, 6400–6429 (2021)

    Article  Google Scholar 

  42. Panda, J., Nanda, P.K.: Video object-tracking using particle filtering and feature fusion. In: Advances in Electrical Control and Signal Systems, pp. 945–957. Springer (2020)

  43. Pérez, P., Hue, C., Vermaak, J., Gangnet, M.: Color-based probabilistic tracking. In: European Conference on Computer Vision, pp. 661–675. Springer (2002)

  44. Poormehdi Ghaemmaghami, M.: Tracking of humans in video stream using LSTM recurrent neural network (2017)

  45. Sun, W., Sun, M., Zhang, X., Li, M.: Moving vehicle detection and tracking based on optical flow method and immune particle filter under complex transportation environments. Complexity (2020). https://doi.org/10.1155/2020/3805320

    Article  Google Scholar 

  46. Tao, R., Gavves, E., Smeulders, A.W.: Siamese instance search for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1420–1429 (2016)

  47. Tian, M., Zhang, W., Liu, F.: On-line ensemble SVM for robust object tracking. In: Asian Conference on Computer Vision, pp. 355–364. Springer (2007)

  48. TrackingNet dataset. https://tracking-net.org

  49. Visual tracker benchmark. http://cvlab.hanyang.ac.kr/tracker_benchmark/datasets.html

  50. Wagenaar, D.A., Kristan, W.B.: Automated video analysis of animal movements using Gabor orientation filters. Neuroinformatics 8(1), 33–42 (2010)

    Article  Google Scholar 

  51. Wan, E.A., Van der Merwe, R.: The unscented Kalman filter for nonlinear estimation. In: IEEE Symposium 2000 (AS-SPCC) (2000)

  52. Wang, N., Shi, J., Yeung, D.Y., Jia, J.: Understanding and diagnosing visual tracking systems. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3101–3109 (2015)

  53. Wang, Y., Ban, X., Wang, H., Li, X., Wang, Z., Wu, D., Yang, Y., Liu, S.: Particle filter vehicles tracking by fusing multiple features. IEEE Access 7, 133694–133706 (2019)

    Article  Google Scholar 

  54. Yang, M., Lin, Y., Huang, D., Kong, L.: Accurate visual tracking via reliable patch. Vis. Comput. 38, 625–638 (2021)

    Article  Google Scholar 

  55. Yun, S., Choi, J., Yoo, Y., Yun, K., Young Choi, J.: Action-decision networks for visual tracking with deep reinforcement learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2711–2720 (2017)

  56. Zhang, S., Yu, X., Sui, Y., Zhao, S., Zhang, L.: Object tracking with multi-view support vector machines. IEEE Trans. Multimed. 17(3), 265–278 (2015)

    Google Scholar 

  57. Zhang, W., Du, Y., Chen, Z., Deng, J., Liu, P.: Robust adaptive learning with Siamese network architecture for visual tracking. Vis. Comput. 37(5), 881–894 (2021)

    Article  Google Scholar 

  58. Zhao, A.: Robust histogram-based object tracking in image sequences. In: 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2007), pp. 45–52. IEEE (2007)

  59. Zhao, H., Yang, G., Wang, D., Lu, H.: Deep mutual learning for visual object tracking. Pattern Recogn. 112, 107796 (2021)

    Article  Google Scholar 

  60. Zhao, Z., Yu, S., Wu, X., Wang, C., Xu, Y.: A multi-target tracking algorithm using texture for real-time surveillance. In: IEEE International Conference on Robotics and Biomimetics, pp. 2150–2155. IEEE (2009)

  61. Zolfaghari, M., Ghanei-Yakhdan, H., Yazdi, M.: Real-time object tracking based on an adaptive transition model and extended Kalman filter to handle full occlusion. Vis. Comput. 36(4), 701–715 (2020)

    Article  Google Scholar 

  62. Zolfaghari, M., Ghanei-Yakhdan, H., Yazdi, M.: Real-time object tracking based on sparse representation and adaptive particle drawing. Vis. Comput. 38, 849–869 (2021)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Basic Science Research Program through the National Research Foundation (NRF) funded by the Korea Government (MSIT) (No. NRF-2019R1I1A1A01058976) and the Korea Agency for Infrastructure Technology Advancement (KAIA) Grant funded by the Ministry of the Interior and Safety (Grant 22PQWO-C153369-04).

Author information

Authors and Affiliations

  1. Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, 48109, USA

    Jaechan Lim

  2. Department of Electronic Engineering, Sogang University, Seoul, 04107, Republic of Korea

    Jaechan Lim, Jin-Young Park & Hyung-Min Park

Authors
  1. Jaechan Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jin-Young Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hyung-Min Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hyung-Min Park.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: The assignment of 2 author portraits was not correct.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lim, J., Park, JY. & Park, HM. Minimax Monte Carlo object tracking. Vis Comput 39, 1853–1868 (2023). https://doi.org/10.1007/s00371-022-02449-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02449-7

Keywords

Search

Navigation