Abstract
Visual tracking is considered a common procedure in many real-time applications. Such systems are required to track objects under changes in illumination, dynamic viewing angle, image noise and occlusions (to name a few). But to maintain real-time performance despite these challenging conditions, tracking methods should require extremely low computational resources, therefore facing a trade-off between robustness and speed. Emergence of new consumer-level cameras capable of capturing video in 60 fps challenges this tradeoff even further. Unfortunately, state-of-the-art tracking techniques struggle to meet frame rates over 30 VGA-resolution fps with standard desktop power, let alone on typically-weaker mobile devices. In this paper we suggest a significantly cheaper computational method for tracking in colour video clips, that greatly improves tracking performance, in terms of robustness/speed trade-off. The suggested approach employs a novel similarity measure that explicitly combines appearance with object kinematics and a new adaptive Kalman filter extends the basic tracking to provide robustness to occlusions and noise. The linear time complexity of this method is reflected in computational efficiency and high processing rate. Comparisons with two recent trackers show superior tracking robustness at more than 5 times faster operation, all using naïve C/C++ implementation and built-in OpenCV functions.
Notes
Predictions such as v P (k) is part of our modified Kalman filtering approach discussed in Sect. 3 and at this point assume it is provided by an external oracle.
The interested reader is referred to [40] for more details on Kalman filter model.
Publicly available at http://www.cvg.rdg.ac.uk/PETS2009/a.html.
Used to be at http://ngsim.fhwa.dot.gov.
Used to be at http://www.vividevaluation.ri.cmu.edu/datasets/datasets.html.
Code used to be at http://www.cs.bilkent.edu.tr/ismaila/MUSCLE/MSTracker.htm.
Code available at http://www.cs.technion.ac.il/~amita/fragtrack/fragtrack.htm.
Publicly available at http://homepages.inf.ed.ac.uk/rbf/CAVIARDATA1/.
Publicly available at http://www.cvg.rdg.ac.uk/PETS2009/a.html.
References
The Mean Shift based Tracker code. http://www.cs.bilkent.edu.tr/ismaila/MUSCLE/MSTracker.htm
The OpenCV library. http://opencv.willowgarage.com/wiki/
The Project Webpage. http://www.cs.bgu.ac.il/rba/Tracking_MPrior_AKF_Results/Project
Adam, A., Rivlin, E., Shimshoni, I.: Robust fragments-based tracking using the integral histogram. In: Proceedings of the CVPR, vol. 1, pp. 798–805 (2006)
Babenko, B., Ming-Hsuan, Y., Belongie, S.: Robust object tracking with online multiple instance learning. IEEE TPAMI 33(8), 1619–1632 (2011)
Ben-Shitrit, H., Berclaz, J., Fleuret, F., Fua, P.: Tracking multiple people under global appearance constraints. In: Proceedings of the ICCV (2011)
Benfold, B., Reid, I.: Stable multi-target tracking in real-time surveillance video. In: Proceedings of the CVPR (2011)
Bernardin, K., Elbs, A., Stiefelhagen, R.: Multiple object tracking performance metrics and evaluation in a smart room environment. In: Sixth IEEE International Workshop on Visual Surveillance, in conjunction with ECCV, vol. 90 (2006)
Beymer, D., Konolige, K.: Real-time tracking of multiple people using continuous detection. In: Proceedings of the ICCV-Frame Rate Workshop (1999)
Boris Babenko Ming-Hsuan Yang, S.B.: Visual tracking with online multiple instance learning. In: Proceedings of the CVPR (2009)
Brodia, T., Chellappa, R.: Estimation of object motion parameters from noisy images. IEEE TPAMI 8(1), 90–99 (1986)
Brown, R.G.: Random Signal Analysis and Kalman Filtering. Wiley, Hoboken (1983)
Bruce, A., Gordon, G.: Better motion prediction for people-tracking. In: Proceedings of the ICRA (2004)
Chu, C.T., Hwang, J.N., Wang, S.Z., Chen, Y.Y.: Human tracking by adaptive Kalman filtering and multiple kernels tracking with projected gradients. In: Proceedings of the Fifth ACM/IEEE International Conference on Distributed Smart Cameras, pp. 1–6 (2011)
Comaniciu, D., Ramesh, V., Meer, P.: Kernel based object tracking. IEEE TPAMI 25, 564–575 (2003)
Daum, F.: Non-particle filters. SPIE—Signal and Data Processing of Small Targets 6326, 614–623 (2006)
Dellaert, F., Thorpe, C.: Robust car tracking using Kalman filtering and bayesian templates. In: Conference on Intelligent Transportation Systems (1997)
Fieguth, P., Terezopoulos, D.: Color-based tracking of heads and other mobile objects at video frame rates. In: Proceedings of the CVPR, pp. 21–27 (1997)
Flávio, B.: Window matching techniques with Kalman filtering for an improved object visual tracking. In: Proceedings of the IEEE Conference on Automation Science and Engineering (2007)
Grabner, H., Leistner, C., Bischof, H.: Semi-supervised on-line boosting for robust tracking. In: Proceedings of the ECCV (2008)
Hii, A.J.H., Hann, C.E., Chase, J.G., Van- Houten, E.W.: Fast normalized cross correlation for motion tracking using basis functions. Comput. Methods Programs Biomed. 82(2), 144–156 (2006)
Jameson, D., Huvich, L.: Complexities of perceived brightness. Science 133, 174–179 (1961).
Li, Y., Ai, H., Yamashita, T., Lao, S., Kawade, M.: Tracking in low frame rate video: A cascade particle filter with discriminative observers of different life spans. IEEE TPAMI 30(10), 1728–1740 (2008)
Liang, D., Huang, Q., Yao, H., Jiang, S., Ji, R., Gao, W.: Novel observation model for probabilistic object tracking. In: Proceedings of the CVPR, pp. 1387–1394 (2010)
Matei, B., Sawhney, H., Samarasekera, S.: Vehicle tracking across nonoverlapping cameras using joint kinematic and appearance features. In: Proceedings of the CVPR (2011)
Matthews, I., Ishikawa, T., Baker, S.: The template update problem. IEEE TPAMI 26(6), 810–815 (2004)
Mauthner, T., Donoser, M., Bischof, H.: Robust tracking of spatial related components. In: Proceedings of the ICPR (2008)
Medrano, C., Martinez, J., Igual, R.: Gaussian approximation for tracking occluding and interacting targets. J. Math. Imaging Vis. 36(2), 241–253 (2010)
Mileva, Y., Bruhn, A., Weickert, J.: Illumination-robust variational optical flow with photometric invariants. In: Proceedings of the DAGM Symposium (2007)
Oussalah, M., Schutter, J.D.: Adaptive Kalman filter for noise identification. In: Proceedings of the International Conference on Noise and Vibration Engineering, pp. 1225–1232 (2000)
Papadakis, N., Bugeau, A.: Tracking with occlusions via graph cuts. IEEE TPAMI 33(1), 144–157 (2011)
Porikli, F.: Achieving real-time object detection and tracking under extreme conditions. J. Real-Time Image Proc. 1, 33–40 (2006)
Sebastian, P., Voon, Y.V.: Tracking using normalized cross correlation and color space. In: Proceedings of the International Conference on Intelligence and Advanced Systems (2007)
Shahrimie, M., Asaari, M., Suandi, S.A.: Hand gesture tracking system using adaptive Kalman filter. In: Proceedings of the International Conference on Intelligent Systems Design and Applications, pp. 166–171 (2010)
Sun, J., Li, Y., Kang, S.B., Shum, H.Y.: Symmetric stereo matching for occlusion handling. In: Proceedings of theInternational Conference on Computer Vision and Pattern Recognition, pp. 399–406 (2005)
Tsai, D.M., Lin, C.T.: Fast normalized cross correlation for defect detection. Pattern Recogn. Lett. 24, 2625–2631 (2003)
Van Leuven, J., Van Leeuwen, M., Groen, F.: Real-time vehicle tracking in image sequences. In: Proceedings of the IEEE Instrumentation and Measurement Technology (2001)
Wang, J., Yagi, Y.: Adaptive mean-shift tracking with auxiliary particles. IEEE Trans. Syst. Man Cybernet. B 39(6), 1578–1589 (2009)
Wang, Y., Teoh, E.K., Shen, D.: Lane detection and tracking using B-snake. Image Vis. Comput. 22(10), 269–280 (2004)
Weltch, G., Bishop, G.: An introduction to the Kalman filter. Technical Report 95-041, Department of Computer Science University of North Carolina at Chapel Hill (2006)
Weng, S.K., Kuo, C.M., Tu, S.K.: Video object tracking using adaptive Kalman filter. J. Vis. Commun. Image Represent. 17, 1190–1208 (2006)
Williams, R.L., Lawrence, D.A.: Linear State-Space Control Systems. Wiley, USA (2007)
Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM Comput. Surv. 38(4) (2006)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ben-Ari, R., Ben-Shahar, O. A computationally efficient tracker with direct appearance-kinematic measure and adaptive Kalman filter. J Real-Time Image Proc 11, 271–285 (2016). https://doi.org/10.1007/s11554-013-0329-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-013-0329-2