Skip to main content
Log in

A computationally efficient tracker with direct appearance-kinematic measure and adaptive Kalman filter

Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Visual tracking is considered a common procedure in many real-time applications. Such systems are required to track objects under changes in illumination, dynamic viewing angle, image noise and occlusions (to name a few). But to maintain real-time performance despite these challenging conditions, tracking methods should require extremely low computational resources, therefore facing a trade-off between robustness and speed. Emergence of new consumer-level cameras capable of capturing video in 60 fps challenges this tradeoff even further. Unfortunately, state-of-the-art tracking techniques struggle to meet frame rates over 30 VGA-resolution fps with standard desktop power, let alone on typically-weaker mobile devices. In this paper we suggest a significantly cheaper computational method for tracking in colour video clips, that greatly improves tracking performance, in terms of robustness/speed trade-off. The suggested approach employs a novel similarity measure that explicitly combines appearance with object kinematics and a new adaptive Kalman filter extends the basic tracking to provide robustness to occlusions and noise. The linear time complexity of this method is reflected in computational efficiency and high processing rate. Comparisons with two recent trackers show superior tracking robustness at more than 5 times faster operation, all using naïve C/C++ implementation and built-in OpenCV functions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. Predictions such as v P (k) is part of our modified Kalman filtering approach discussed in Sect. 3 and at this point assume it is provided by an external oracle.

  2. The interested reader is referred to [40] for more details on Kalman filter model.

  3. Publicly available at http://www.cvg.rdg.ac.uk/PETS2009/a.html.

  4. Used to be at http://ngsim.fhwa.dot.gov.

  5. Available at http://www.youtube.com/watch?v=omI094GcZcw&feature=related.

  6. Used to be at http://www.vividevaluation.ri.cmu.edu/datasets/datasets.html.

  7. Code used to be at http://www.cs.bilkent.edu.tr/ismaila/MUSCLE/MSTracker.htm.

  8. Code available at http://www.cs.technion.ac.il/~amita/fragtrack/fragtrack.htm.

  9. Publicly available at http://homepages.inf.ed.ac.uk/rbf/CAVIARDATA1/.

  10. Publicly available at http://www.cvg.rdg.ac.uk/PETS2009/a.html.

  11. Publicly available at http://www.robots.ox.ac.uk/ActiveVision/Research/Projects/2009bbenfold_headpose/project.html#datasets.

References

  1. The Mean Shift based Tracker code. http://www.cs.bilkent.edu.tr/ismaila/MUSCLE/MSTracker.htm

  2. The OpenCV library. http://opencv.willowgarage.com/wiki/

  3. The Project Webpage. http://www.cs.bgu.ac.il/rba/Tracking_MPrior_AKF_Results/Project

  4. Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral histogram. In: Proceedings of the CVPR, vol. 1, pp. 798–805 (2006)

  5. Babenko, B., Ming-Hsuan, Y., Belongie, S.: Robust object tracking with online multiple instance learning. IEEE TPAMI 33(8), 1619–1632 (2011)

    Article  Google Scholar 

  6. Ben-Shitrit, H., Berclaz, J., Fleuret, F., Fua, P.: Tracking multiple people under global appearance constraints. In: Proceedings of the ICCV (2011)

  7. Benfold, B., Reid, I.: Stable multi-target tracking in real-time surveillance video. In: Proceedings of the CVPR (2011)

  8. Bernardin, K., Elbs, A., Stiefelhagen, R.: Multiple object tracking performance metrics and evaluation in a smart room environment. In: Sixth IEEE International Workshop on Visual Surveillance, in conjunction with ECCV, vol. 90 (2006)

  9. Beymer, D., Konolige, K.: Real-time tracking of multiple people using continuous detection. In: Proceedings of the ICCV-Frame Rate Workshop (1999)

  10. Boris Babenko Ming-Hsuan Yang, S.B.: Visual tracking with online multiple instance learning. In: Proceedings of the CVPR (2009)

  11. Brodia, T., Chellappa, R.: Estimation of object motion parameters from noisy images. IEEE TPAMI 8(1), 90–99 (1986)

    Article  Google Scholar 

  12. Brown, R.G.: Random Signal Analysis and Kalman Filtering. Wiley, Hoboken (1983)

  13. Bruce, A., Gordon, G.: Better motion prediction for people-tracking. In: Proceedings of the ICRA (2004)

  14. Chu, C.T., Hwang, J.N., Wang, S.Z., Chen, Y.Y.: Human tracking by adaptive Kalman filtering and multiple kernels tracking with projected gradients. In: Proceedings of the Fifth ACM/IEEE International Conference on Distributed Smart Cameras, pp. 1–6 (2011)

  15. Comaniciu, D., Ramesh, V., Meer, P.: Kernel based object tracking. IEEE TPAMI 25, 564–575 (2003)

    Google Scholar 

  16. Daum, F.: Non-particle filters. SPIE—Signal and Data Processing of Small Targets 6326, 614–623 (2006)

  17. Dellaert, F., Thorpe, C.: Robust car tracking using Kalman filtering and bayesian templates. In: Conference on Intelligent Transportation Systems (1997)

  18. Fieguth, P., Terezopoulos, D.: Color-based tracking of heads and other mobile objects at video frame rates. In: Proceedings of the CVPR, pp. 21–27 (1997)

  19. Flávio, B.: Window matching techniques with Kalman filtering for an improved object visual tracking. In: Proceedings of the IEEE Conference on Automation Science and Engineering (2007)

  20. Grabner, H., Leistner, C., Bischof, H.: Semi-supervised on-line boosting for robust tracking. In: Proceedings of the ECCV (2008)

  21. Hii, A.J.H., Hann, C.E., Chase, J.G., Van- Houten, E.W.: Fast normalized cross correlation for motion tracking using basis functions. Comput. Methods Programs Biomed. 82(2), 144–156 (2006)

    Article  Google Scholar 

  22. Jameson, D., Huvich, L.: Complexities of perceived brightness. Science 133, 174–179 (1961).

    Google Scholar 

  23. Li, Y., Ai, H., Yamashita, T., Lao, S., Kawade, M.: Tracking in low frame rate video: A cascade particle filter with discriminative observers of different life spans. IEEE TPAMI 30(10), 1728–1740 (2008)

    Article  Google Scholar 

  24. Liang, D., Huang, Q., Yao, H., Jiang, S., Ji, R., Gao, W.: Novel observation model for probabilistic object tracking. In: Proceedings of the CVPR, pp. 1387–1394 (2010)

  25. Matei, B., Sawhney, H., Samarasekera, S.: Vehicle tracking across nonoverlapping cameras using joint kinematic and appearance features. In: Proceedings of the CVPR (2011)

  26. Matthews, I., Ishikawa, T., Baker, S.: The template update problem. IEEE TPAMI 26(6), 810–815 (2004)

    Article  Google Scholar 

  27. Mauthner, T., Donoser, M., Bischof, H.: Robust tracking of spatial related components. In: Proceedings of the ICPR (2008)

  28. Medrano, C., Martinez, J., Igual, R.: Gaussian approximation for tracking occluding and interacting targets. J. Math. Imaging Vis. 36(2), 241–253 (2010)

    Article  Google Scholar 

  29. Mileva, Y., Bruhn, A., Weickert, J.: Illumination-robust variational optical flow with photometric invariants. In: Proceedings of the DAGM Symposium (2007)

  30. Oussalah, M., Schutter, J.D.: Adaptive Kalman filter for noise identification. In: Proceedings of the International Conference on Noise and Vibration Engineering, pp. 1225–1232 (2000)

  31. Papadakis, N., Bugeau, A.: Tracking with occlusions via graph cuts. IEEE TPAMI 33(1), 144–157 (2011)

    Article  Google Scholar 

  32. Porikli, F.: Achieving real-time object detection and tracking under extreme conditions. J. Real-Time Image Proc. 1, 33–40 (2006)

    Article  Google Scholar 

  33. Sebastian, P., Voon, Y.V.: Tracking using normalized cross correlation and color space. In: Proceedings of the International Conference on Intelligence and Advanced Systems (2007)

  34. Shahrimie, M., Asaari, M., Suandi, S.A.: Hand gesture tracking system using adaptive Kalman filter. In: Proceedings of the International Conference on Intelligent Systems Design and Applications, pp. 166–171 (2010)

  35. Sun, J., Li, Y., Kang, S.B., Shum, H.Y.: Symmetric stereo matching for occlusion handling. In: Proceedings of theInternational Conference on Computer Vision and Pattern Recognition, pp. 399–406 (2005)

  36. Tsai, D.M., Lin, C.T.: Fast normalized cross correlation for defect detection. Pattern Recogn. Lett. 24, 2625–2631 (2003)

    Article  Google Scholar 

  37. Van Leuven, J., Van Leeuwen, M., Groen, F.: Real-time vehicle tracking in image sequences. In: Proceedings of the IEEE Instrumentation and Measurement Technology (2001)

  38. Wang, J., Yagi, Y.: Adaptive mean-shift tracking with auxiliary particles. IEEE Trans. Syst. Man Cybernet. B 39(6), 1578–1589 (2009)

    Article  Google Scholar 

  39. Wang, Y., Teoh, E.K., Shen, D.: Lane detection and tracking using B-snake. Image Vis. Comput. 22(10), 269–280 (2004)

    Article  Google Scholar 

  40. Weltch, G., Bishop, G.: An introduction to the Kalman filter. Technical Report 95-041, Department of Computer Science University of North Carolina at Chapel Hill (2006)

  41. Weng, S.K., Kuo, C.M., Tu, S.K.: Video object tracking using adaptive Kalman filter. J. Vis. Commun. Image Represent. 17, 1190–1208 (2006)

    Article  Google Scholar 

  42. Williams, R.L., Lawrence, D.A.: Linear State-Space Control Systems. Wiley, USA (2007)

  43. Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM Comput. Surv. 38(4) (2006)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rami Ben-Ari.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ben-Ari, R., Ben-Shahar, O. A computationally efficient tracker with direct appearance-kinematic measure and adaptive Kalman filter. J Real-Time Image Proc 11, 271–285 (2016). https://doi.org/10.1007/s11554-013-0329-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-013-0329-2

Keywords

Navigation