Skip to main content
SpringerLink
Log in

A massively parallel keypoint detection and description (MP-KDD) algorithm for high-speed vision chip

面向高速视觉芯片的大规模并行特征点检测和描述算法

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

This paper proposes a massively parallel keypoint detection and description (MP-KDD) algorithm for the vision chip with parallel array processors. The MP-KDD algorithm largely reduces the computational overhead by removing all floating-point and multiplication operations while preserving the currently popular SIFT and SURF algorithm essence. The MP-KDD algorithm can be directly and effectively mapped onto the pixel-parallel and row-parallel array processors of the vision chip. The vision chip architecture is also enhanced to realize direct memory access (DMA) and random access to array processors so that the MP-KDD algorithm can be executed more effectively. An FPGA-based vision chip prototype is implemented to test and evaluate our MP-KDD algorithm. Its image processing speed reaches 600–760 fps with high accuracy for complex vision applications, such as scene recognition.

摘要

本文提出了一种面向视觉芯片并行图像处理阵列的高效图像特征点提取和描述算法。 该算法基于SIFT特征点检测及SURF特征点描述, 但简化避免了浮点运算以及乘除法操作, 极大地节约了硬件开销和处理时间。 该算法可以直接、 高效地映射到视觉芯片的像素级并行和行并行阵列处理器。 本文实现了基于FPGA的视觉原型, 在其上成功测试了所提出的算法, 达到了600–700帧/秒的较高速度。

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ishikawa M, Ogawa K, Komoro T, et al. A CMOS vision chip with SIMD processing element array for 1 ms image processing. In: Proceedings of IEEE Solid-State Circuits Conference Digest of Technical Papers (ISSCC), San Francisco, 1999. 206–207

    Google Scholar 

  2. Shi C, Yang J, Han Y, et al. A 1000 fps vision chip based on a dynamically reconfigurable hybrid architecture comprising a PE array and self-organizing map neural network. In: Proceedings of IEEE Solid-State Circuits Conference Digest of Technical Papers (ISSCC), San Francisco, 2014. 128–129

    Google Scholar 

  3. Komoro T, Ishii I, Ishikawa M, et al. A digital vision chip specialized for high-speed target tracking. IEEE Trans Electron Dev, 2003, 50: 191–199

    Article  Google Scholar 

  4. Dudek P, Hicks P J. A general-purpose processor-per-pixel analog SIMD vision chip. IEEE Trans Circuits Syst I-Regul Pap, 2005, 52: 13–20

    Article  Google Scholar 

  5. Ishii I, Yamamoto K, Kubozono M. Higher order autocorrelation vision chip. IEEE Trans Electron Dev, 2006, 53: 1797–1804

    Article  Google Scholar 

  6. Miao W, Lin Q Y, Zhang W C, et al. A programmable SIMD vision chip for real-time vision applications. IEEE J Solid-State Circuits, 2008, 43: 1470–1479

    Article  Google Scholar 

  7. Dubois J, Ginhac D, Paindavoine M, et al. A 10 000 fps CMOS sensor with massively parallel image processing. IEEE J Solid-State Circuits, 2008, 43: 706–717

    Article  Google Scholar 

  8. Yamashita H, Sodini C. A CMOS imager with a programmable bit-serial column-parallel SIMD/MIMD processor. IEEE Trans Electron Dev, 2009, 56: 2534–2545

    Article  Google Scholar 

  9. Lin Q Y, Miao W, Zhang W C. A 1000 frame/s programmable vision chip with variable resolution and row-pixel-mixed parallel image processors. Sensors, 2009, 9: 5933–5951

    Article  Google Scholar 

  10. Cheng C C, Lin C H, Li C T, et al. iVisual: an intelligent visual sensor SoC with 2790 fps image sensor and 205 GOPS/W vision processor. IEEE J Solid-State Circuits, 2009, 44: 127–135

    Article  Google Scholar 

  11. Zhang W C, Fu Q Y, Wu N J. A programmable vision chip based on multiple levels of parallel processors. IEEE J Solid-State Circuits, 2011, 46: 2132–2147

    Article  Google Scholar 

  12. Lopich A, Dudek P. A SIMD cellular processor array vision chip with asynchronous processing capabilities. IEEE Trans Circuits Syst I-Regul Pap, 2011, 58: 2420–2431

    Article  MathSciNet  Google Scholar 

  13. Cottini N, Gottardi M, Massari N, et al. A 33 μW 64×64 pixel vision sensor embedding robust dynamic background subtraction for event detection and scene interpretation. IEEE J Solid-State Circuits, 2013, 48: 850–863

    Article  Google Scholar 

  14. Gauglitz S, Höllerer T, Turk M. Evaluation of interest point detectors and feature descriptors for visual tracking. Int J Comput Vis, 2011, 94: 335–360

    Article  MATH  Google Scholar 

  15. Lowe D G. Distinctive image features from scale-invariant keypoints. Int J Comput Vis, 2004, 60: 91–110

    Article  Google Scholar 

  16. Bay H, Ess A, Tuytelaars T, et al. Speeded-up robust features (SURF). Comput Vis Image Underst, 2008, 110: 346–359

    Article  Google Scholar 

  17. Bonato V, Marques E, Constantinides G A. A parallel hardware architecture for scale and rotation invariant feature detection. IEEE Trans Circuits Syst Video Technol, 2008, 18: 1703–1712

    Article  Google Scholar 

  18. Kim K, Lee S, Kim J Y, et al. A configurable heterogeneous multicore architecture with cellular neural network for real-time object recognition. IEEE Trans Circuits Syst Video Technol, 2009, 19: 1612–1622

    Article  Google Scholar 

  19. Zhang W L, Liu L B, Yin S Y, et al. An efficient VLSI architecture of speeded-up robust feature extraction for high resolution and high frame rate video. Sci China Inf Sci, 2013, 56: 072402

    MathSciNet  Google Scholar 

  20. Zhou Y F, Cao Z X, Qin Q, et al. A 1000 FPS high speed CMOS image sensor with low noise global shutter pixels. Sci China Inf Sci, 2014, 57: 042405

    Google Scholar 

  21. Csurka C, Dance C R, Fan L, et al. Visual categorization with bags of keypoints. In: Proceedings of ECCV International Workshop on Statistical Learning in Computer Vision, Prague, 2004. 1–7

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory for Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, 100083, China

    Cong Shi, Jie Yang, LiYuan Liu & NanJian Wu

  2. Department of Electronic Engineering, Tsinghua University, Beijing, 100084, China

    Cong Shi & ZhiHua Wang

  3. Institute of Microelectronics, Tsinghua University, Beijing, 100084, China

    ZhiHua Wang

Authors
  1. Cong Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. LiYuan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. NanJian Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. ZhiHua Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to NanJian Wu.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, C., Yang, J., Liu, L. et al. A massively parallel keypoint detection and description (MP-KDD) algorithm for high-speed vision chip. Sci. China Inf. Sci. 57, 1–12 (2014). https://doi.org/10.1007/s11432-014-5174-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-014-5174-9

Keywords

关键词

Search

Navigation