Skip to main content
SpringerLink
Log in

On-chip semidense representation map for dense visual features driven by attention processes

  • Special Issue
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

We describe an intelligent scheme to condense dense vision features, efficiently reducing the size of representation maps and keeping relevant information for further processing during subsequent stages. We have integrated our condensation algorithm in a low-level-vision system that obtains several vision-features in real-time working on an FPGA. Within this framework, our condensation algorithm allows for the transfer of information from the FPGA device (or processing chip) to any co-processor (from embedded ones to external PCs or DSPs) under technological constraints (such as bandwidth, memory and performance ones). Our condensation core processes 1024 × 1024 resolution images at up to 90 fps. Hence, our condensation module performs this process introducing an insignificant delay in the vision system. A hardware implementation usually implies a simplified version of the vision-feature extractor. Therefore, our condensation process inherently regularizes low-level-vision features, effectively reducing discontinuities and errors. The semidense representation obtained is compatible with mid-/high-level-vision modules, usually implemented as software components. In addition, our versatile semidense map is ready to receive feedback from attention processes, integrating task-driven attention (i.e. top-down information) in real time. Thus, the main advantages of this core are: real-time throughput, versatility, inherent regularization, scalability and feedback from other stages.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Anguita. M., Diaz, J., Ros, E., Fernandez-Baldomero, F. Optimization strategies for high-performance computing of optical-flow in general-purpose processors. IEEE Trans on CSVT 19(10):1475–1488 (2009). doi:10.1109/TCSVT.2009.2026821

    Google Scholar 

  2. Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M., Szeliski, R. A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1):1–31 (2011). doi:10.1007/s11263-010-0390-2

    Google Scholar 

  3. Barranco, F., Tomasi, M., Diaz, J., Vanegas, M., Ros, E. Pipelined architecture for real-time cost-optimized extraction of visual primitives based on fpgas. Digital Signal Process. (2012). doi:10.1016/j.dsp.2012.09.017

  4. Bay, H., Ess, A,, Tuytelaars, T., Van-Gool, L. Speeded-up robust features (surf). Comput. Vis. Image Underst. 110:346–359 (2008). doi:10.1016/j.cviu.2007.09.014

    Google Scholar 

  5. Benoit, A., Caplier, A., Durette, B., Herault, J. Using human visual system modeling for bio-inspired low level image processing. Comput. Vis. Image Underst. 114(7):758–773 (2010). doi:10.1016/j.cviu.2010.01.011

    Google Scholar 

  6. Boahen, K. A burst-mode word-serial address-event linki: Transmitter design. IEEE Trans Circuits Syst I, Reg Papers, pp. 1269–1280 (2004)

  7. Camus, T. Calculating time-to-contact using real-time quantized optical flow. In: National Institute of Standards and Technology NISTIR 5609 (1995)

  8. Canny, J. A computational approach to edge detection. IEEE Trans PAMI 8(6):679–698 (1986). doi:10.1109/TPAMI.1986.4767851

    Google Scholar 

  9. Chessa, M., Solari, F., Sabatini, S.P., Bisio, G.M. Motion interpretation using adjustable linear models. In: BMVC (2008)

  10. core OCJH (2012) http://opencores.org/project, jpeg online

  11. Daz, J., Ros, E., Mota, S., Carrillo, R. Local image phase, energy and orientation extraction using FPGAs. Int. J. Electron. 95:743–760 (2008). doi:10.1080/00207210801941200

    Google Scholar 

  12. Dong, F., Ye, X. Multiscaled texture synthesis using multisized pixel neighborhoods. IEEE Comput. Graph Appl. 27:41–47 (2007). doi:10.1109/MCG.2007.66

    Google Scholar 

  13. Gai, J., Stevenson, R. Optical flow estimation with p-harmonic regularization. In: Image Processing (ICIP), 2010 17th IEEE International Conference on, pp. 1969–1972 (2010)

  14. Galić, I., Weickert, J., Welk, M., Bruhn, A., Belyaev, A., Seidel, H. Image compression with anisotropic diffusion. J. Math. Imag. Vis. 31:255–269 (2008). doi:10.1007/s10851-008-0087-0

    Google Scholar 

  15. Granados, S., Chumerin, N., Mota, S., Diaz, J. Obstacle detection using semidense representation maps. J. Vis. Commun. Image Represent. (2012) (submitted)

  16. Itti, L., Koch, C. Computational modelling of visual attention. Nat. Rev. Neurosci. 2(3):194–203 (2001)

    Google Scholar 

  17. Kolb, H., Lipetz, L. The anatomical basis for colour vision in the vertebrate retina. Vis. Vis. Dysfunct. Percept. colour 6:128–145 (1991)

    Google Scholar 

  18. Kruger, N., Felsberg, M. A continuous formulation of intrinsic dimension. In: Proceedings of the British Machine Vision Conference (2003)

  19. Kruger, N., Potzsch, M., Peters, G. Principles of Cortical Processing Applied to and Motivated by Artificial Object Recognition. Cambridge University Press, Cambridge (2000)

  20. Lowe, D. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60:91–110 (2004). doi:10.1023/B:VISI.0000029664.99615.94

    Google Scholar 

  21. Luettgen, M., Clem-Karl, W., Willsky, A. Efficient multiscale regularization with applications to the computation of optical flow. IEEE Trans. Image Process. 3(1):41–64 (1994). doi:10.1109/83.265979

    Google Scholar 

  22. Mikolajczyk, K., Schmid, C.A performance evaluation of local descriptors. IEEE Trans. PAMI. 27(10), 1615–1630. doi:10.1109/TPAMI.2005.188

  23. Ortigosa, E., Cañas, A., Ros, E., Martínez-Ortigosa, P., Mota, S., Díaz, J. Hardware description of multi-layer perceptrons with different abstraction levels. Microprocess. Microsyst. 30(7):435–444 (2006)

    Google Scholar 

  24. Pauwels, K., Kruger, N., Lappe, M., Worgotter, F., Van-Hulle, M. A cortical architecture on parallel hardware for motion processing in real time. J. Vis. 10(10), Art No 18, 1–21 (2010). doi:10.1167/10.10.18

    Google Scholar 

  25. Pauwels, K., Tomasi, M., Diaz-Alonso, J., Ros, E., Van-Hulle, M. A comparison of fpga and gpu for real-time phase-based optical flow, stereo, and local image features. IEEE Trans. Comput. 61(7):999–1012 (2012). doi:10.1109/TC.2011.120

  26. Pugeault, N., Wörgötter, F., Krüger, N. Visual primitives: local, condensed, semantically rich visual descriptors and their applications in robotics. Int. J. Humanoid Robotics. 7(3):379–405 (2010)

    Google Scholar 

  27. Ralli, J., Díaz, J., Ros, E. A method for sparse disparity densification using voting mask propagation. J. Vis. Comput. Image Rep. 21(1):67–74 (2009). doi:10.1016/j.jvcir.2009.08.005

    Google Scholar 

  28. Sabatini, S., Gastaldi, G., Solari, F., Pauwels, K., Van-Hulle, M., Diaz, J., Ros, E., Pugeault, N., Krüger, N. A compact harmonic code for early vision based on anisotropic frequency channels. Comput. Vis. Image Underst. 114:681–699 (2010). doi: 10.1016/j.cviu.2010.03.008

    Google Scholar 

  29. Sanger, T. Stereo disparity computation using gabor filters. Biol. Cybern. 59:405–418 (1988). doi:10.1007/BF00336114

    Google Scholar 

  30. Sayood, K. Introduction to data compression. Morgan Kaufmann Publishers Inc., San Francisco (2000)

  31. Shi, J., Tomasi, C. Good features to track. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp. 593–600 (1994). doi:10.1109/CVPR.1994.323794

  32. Solutions S (2012) http://www.sevensols.com. online

  33. Tomasi, M., Vanegas, M., Barranco, F., Diaz, J., Ros, E. Massive parallel-hardware architecture for multi-scale stereo, optical flow and image-structure computation. IEEE Trans. CSVT 22, 282–294 (2011a). doi:10.1109/TCSVT.2011.2162260

  34. Tomasi, M., Vanegas, M., Barranco, F., Diaz, J., Ros, E. Real-time architecture for a robust multi-scale stereo engine on fpga. IEEE Trans VLSI Systems 20, 2208–2219 (2011b). doi:10.1109/TVLSI.2011.2172007

  35. Vanegas, M., Tomasi, M., Díaz, J., Vidal, E.R. Multi-port abstraction layer for fpga intensive memory exploitation applications. J. Syst. Architect. Embedded Syst. Des. 56(9):442–451 (2010)

    Google Scholar 

  36. Webpage D (2012) DRIVSCO European Project. http://www.pspc.dibe.unige.it/~drivsco/

  37. Weems, C. Jr. Real-time considerations in the design of the image understanding architecture. Real Time Imag. 2:341–350 (1996). doi:10.1006/rtim.1996.0035

  38. Xilinx (2012) http://www.xilinx.com. online

Download references

Acknowledgments

The authors thank A. L. Tate for revising their English text.

Author information

Authors and Affiliations

  1. Department of Computer Architecture and Computer Technology, CITIC-UGR, University of Granada, 18071, Granada, Spain

    Sara Granados, Francisco Barranco, Javier Díaz & Eduardo Ros

  2. Department of Signal Theory, Telematics and Communications, CITIC-UGR, University of Granada, 18071, Granada, Spain

    Sonia Mota

Authors
  1. Sara Granados
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francisco Barranco
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sonia Mota
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Javier Díaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eduardo Ros
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sara Granados.

Additional information

This work has been partially supported by the Spanish regional grant ITREBA (TIC-5060), the national grant ARC-VISION (TEC2010-15396) and the EU project TOMSY(FP7-270436).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Granados, S., Barranco, F., Mota, S. et al. On-chip semidense representation map for dense visual features driven by attention processes. J Real-Time Image Proc 9, 171–185 (2014). https://doi.org/10.1007/s11554-012-0320-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-012-0320-3

Keywords

Search

Navigation