Skip to main content
SpringerLink
Log in

Fast re-OBJ: real-time object re-identification in rigid scenes

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Re-identifying objects in a rigid scene across varying viewpoints (object Re-ID) is a challenging task, in particular when there are similar, even identical objects coexist in the same environment. Discriminative features play no doubt an essential role in addressing this challenge, while for practical deployment, real-time performance is another desired attribute. We therefore propose a novel framework, named Fast re-OBJ, that is able to improve both Re-ID accuracy and processing speed via tight coupling between the instance segmentation module and embedding generation module. The rich object encoding in the instance segmentation backbone is directly shared to the embedding generation module for training a more discriminative representation via a triplet network. Moreover, we create datasets with the segmentation outputs using real-time object detectors to train and evaluate our object embedding module. With extensive experiments, we prove that our proposed Fast re-OBJ improves the object Re-ID accuracy by 5% and the speed is \(5\times \) faster compared to the state-of-the-art methods. The dataset and code repository are publicly available at: https://tinyurl.com/bdsb53c4.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Ahmed, E., Jones, M., Marks, T.K.: An improved deep learning architecture for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3908–3916 (2015)

  2. Bansal, V., James, S., Del Bue, A.: re-OBJ: Jointly learning the foreground and background for object instance re-identification. In: Proceedings of International Conference on Image Analysis and Processing (ICIAP), pp. 402–413 (2019)

  3. Bazzani, L., Cristani, M., Perina, A., Murino, V.: Multiple-shot person re-identification by chromatic and epitomic analyses. Pattern Recogn. Lett. 33(7), 898–903 (2012)

    Article  Google Scholar 

  4. Bazzani, L., Cristani, M., Murino, V.: Symmetry-driven accumulation of local features for human characterization and re-identification. Comput. Vis. Image Underst. 117(2), 130–144 (2013)

    Article  Google Scholar 

  5. Bedagkar-Gala, A., Shah, S.K.: A survey of approaches and trends in person re-identification. Image Vis. Comput. 32(4), 270–286 (2014)

    Article  Google Scholar 

  6. Bergmann, P., Meinhardt, T., Leal-Taixe, L.: Tracking without bells and whistles. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 941–951 (2019)

  7. Bochinski, E., Eiselein, V., Sikora, T.: High-speed tracking-by-detection without using image information. In: Proceedings of International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6 (2017)

  8. Bolya, D., Zhou, C., Xiao, F., Lee, Y.J.: Yolact: Real-time instance segmentation. In: Proceedings of IEEE International Conference on Computer Vision (ICCV) (2019)

  9. Bolya, D., Zhou, C., Xiao, F., Lee, Y.J.: Yolact++: Better real-time instance segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020)

  10. Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T., Nießner, M.: Scannet: Richly-annotated 3d reconstructions of indoor scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5828–5839 (2017)

  11. Farenzena, M., Bazzani, L., Perina, A., Murino, V., Cristani, M.: Person re-identification by symmetry-driven accumulation of local features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2360–2367 (2010)

  12. Fu, J., Huang, Q., Doherty, K., Wang, Y., Leonard, J.J.: A multi-hypothesis approach to pose ambiguity in object-based slam. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7639–7646 (2021). https://doi.org/10.1109/IROS51168.2021.9635956

  13. Gordo, A., Almazán, J., Revaud, J., Larlus, D.: Deep image retrieval: Learning global representations for image search. In: Proceedings of European Conference on Computer Vision (ECCV), pp. 241–257. Springer (2016)

  14. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

  15. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)

  16. Hermans, A., Beyer, L., Leibe, B.: In defense of the triplet loss for person re-identification. (2017). Preprint arXiv:1703.07737

  17. Kingma, D.P., Ba, J.: Adam (2014), a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR), Preprint arXiv, vol 1412 (2015)

  18. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  19. Le, T., Nguyen, K., Nguyen-Phan, M., Ton, T., Nguyen, T., Trinh, X., Dinh, Q., Nguyen, V., Duong, A., Sugimoto, A., et al.: Instance re-identification flow for video object segmentation. In: CVPR Workshop (2017)

  20. Li, X., Change Loy, C.: Video object segmentation with joint re-identification and attention-aware mask propagation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 90–105 (2018)

  21. Li, X., Qi, Y., Wang, Z., Chen, K., Liu, Z., Shi, J., Luo, P., Loy, C.C., Tang, X., Khoreva, A., et al.: Video object segmentation with re-identification. In: The 2017 DAVIS Challenge on Video Object Segmentation-CVPR Workshops (2017)

  22. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European Conference on Computer Vision, pp. 740–755. Springer (2014)

  23. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

  24. Liu, H., Wang, F., Zhang, X., Sun, F.: Weakly-paired deep dictionary learning for cross-modal retrieval. Pattern Recogn. Lett. 130, 199–206 (2020)

    Article  Google Scholar 

  25. Nicholson, L., Milford, M., Sünderhauf, N.: Quadricslam: dual quadrics from object detections as landmarks in object-oriented slam. IEEE Robot. Autom. Lett. 4(1), 1–8 (2018)

    Article  Google Scholar 

  26. Ok, K., Liu, K., Frey, K., How, J.P., Roy, N.: Robust object-based slam for high-speed autonomous navigation. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 669–675 (2019). https://doi.org/10.1109/ICRA.2019.8794344

  27. Paisitkriangkrai, S., Shen, C., Van Den Hengel, A.: Learning to rank in person re-identification with metric ensembles. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1846–1855 (2015)

  28. Radenović, F., Tolias, G., Chum, O.: Fine-tuning cnn image retrieval with no human annotation. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1655–1668 (2018)

    Article  Google Scholar 

  29. Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. Preprint arXiv:1804.02767 (2018)

  30. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)

  31. Revaud, J., Almazán, J., Rezende, R.S., Souza, C.Rd.: Learning with average precision: Training image retrieval with a listwise loss. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 5107–5116 (2019)

  32. Rubino, C., Crocco, M., Del Bue, A.: 3d object localisation from multi-view image detections. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1281–1294 (2017)

    Google Scholar 

  33. Rubino, C., Crocco, M., Del Bue, A.: 3d object localisation from multi-view image detections. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1281–1294 (2018)

    Google Scholar 

  34. Salvador, A., Giró-i Nieto, X., Marqués, F., Satoh, S.: Faster r-cnn features for instance search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 9–16 (2016)

  35. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)

  36. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)

  37. Tao, R., Gavves, E., Smeulders, A.W.: Siamese instance search for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1420–1429 (2016)

  38. Teichmann, M., Araujo, A., Zhu, M., Sim, J.: Detect-to-retrieve: Efficient regional aggregation for image search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5109–5118 (2019)

  39. Tokmakov, P., Li, J., Burgard, W., Gaidon, A.: Learning to track with object permanence. In: ICCV (2021)

  40. Wang, H., Li, Z., Li, Y., Gupta, B., Choi, C.: Visual saliency guided complex image retrieval. Pattern Recogn. Lett. 130, 64–72 (2020)

    Article  Google Scholar 

  41. Wang, J., Song, Y., Leung, T., Rosenberg, C., Wang, J., Philbin, J., Chen, B., Wu, Y.: Learning fine-grained image similarity with deep ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1386–1393 (2014)

  42. Wei, X.S., Luo, J.H., Wu, J., Zhou, Z.H.: Selective convolutional descriptor aggregation for fine-grained image retrieval. IEEE Trans. Image Process. 26(6), 2868–2881 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  43. Wu, Y., Bourahla, O.E.F., Li, X., Wu, F., Tian, Q., Zhou, X.: Adaptive graph representation learning for video person re-identification. IEEE Transactions on Image Processing (2020)

  44. Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5987–5995 (2017). https://doi.org/10.1109/CVPR.2017.634

  45. Ye, M., Shen, J., Lin, G., Xiang, T., Shao, L., Hoi, S.C.H.: Deep learning for person re-identification: a survey and outlook. Preprint arXiv:2001.04193 (2020)

  46. Ye, M., Shen, J., Lin, G., Xiang, T., Shao, L., Hoi, S.C.: Deep learning for person re-identification: A survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)

  47. Zhou, K., Yang, Y., Cavallaro, A., Xiang, T.: Learning Generalisable Omni-Scale Representations for Person Re-identification. TPAMI (2021)

  48. Zhou, X., Koltun, V., Krähenbühl, P.: Tracking Objects as Points. ECCV (2020)

  49. Zhu, X., Zhu, X., Li, M., Murino, V., Gong, S.: Intra-camera supervised person re-identification: A new benchmark. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (CVPRW) (2019)

Download references

Acknowledgements

This work is partially supported by the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No. 870743.

Author information

Authors and Affiliations

  1. Department of Mechatronics Engineering, Yildiz Technical University, 34349, Besiktas, Istanbul, Turkey

    Ertugrul Bayraktar

  2. Deep Visual Learning (DVL) Lab, Fondazione Bruno Kessler, Trento, Italy

    Yiming Wang

  3. Visual Geometry and Modelling (VGM), Pattern Analysis and Computer Vision (PAVIS), Istituto Italiano di Tecnologia (IIT), Genoa, Italy

    Yiming Wang & Alessio DelBue

Authors
  1. Ertugrul Bayraktar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiming Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alessio DelBue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ertugrul Bayraktar.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bayraktar, E., Wang, Y. & DelBue, A. Fast re-OBJ: real-time object re-identification in rigid scenes. Machine Vision and Applications 33, 97 (2022). https://doi.org/10.1007/s00138-022-01349-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-022-01349-z

Keywords

Search

Navigation