Skip to main content
SpringerLink
Log in

Bio-inspired visual neural network on spatio-temporal depth rotation perception

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In primates’ cerebral cortex, depth rotation sensitive (DRS) neurons have the property of preferential selectivity for depth rotation motion, whereas such a property is rarely adopted to create computational models for depth rotation motion detection. To fill this gap, a novel feedforward visual neural network is developed to execute depth rotation object detection, based on the recent neurophysiologic achievements on the mammalian vision system. The proposed neural network consists of two parts, i.e., presynaptic and postsynaptic neural networks. The former comprises multiple lateral inhibition neural sub-networks for the capture of visual motion information, and the latter extracts the cues of translational and depth motion and later, synthesizes such clues to perceive the process of depth rotation of an object. Experimentally, the neural network is sufficiently examined by different types of depth rotation under multiple conditions and settings. Numerical experiments show that not only it can effectively detect the spatio-temporal energy change of depth rotation of a moving object, but also its output excitation curve is a quasi-sinusoidal one, which is compatible with the hypothesis suggested by Johansson and Jansson in projective geometry. This research is a critical step toward the construction of artificial vision system for depth rotation object recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

References

  1. Yan C, Xie H, Yang D et al (2018) Supervised hash coding with deep neural network for environment perception of intelligent vehicles. IEEE Trans Intell Transp Syst 19:284–295

    Article  Google Scholar 

  2. Vlasits A, Baden T (2019) Motion vision: a new mechanism in the mammalian retina. Curr Biol 29:R933–R935

    Article  Google Scholar 

  3. Koenderink JJ, van Doorn AJ (1976) Local structure of movement parallax of the plane. J Opt Soc Am 66:717–723

    Article  MathSciNet  Google Scholar 

  4. Verri A, Girosi F, Torre V (1990) Differential techniques for optical flow. J Opt Soc Am A 7:912–922

    Article  Google Scholar 

  5. Maunsell JH, Van Essen DC (1983) Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. J Neurophysiol 49:1127–1147

    Article  Google Scholar 

  6. Rind FC, Simmons PJ (1999) Seeing what is coming: building collision-sensitive neurones. Trends Neurosci 22:215–220

    Article  Google Scholar 

  7. Saito H, Yukie M, Tanaka K et al (1986) Integration of direction signals of image motion in the superior temporal sulcus of the macaque monkey. J Neurosci 6:145–157

    Article  Google Scholar 

  8. Sakata H, Shibutani H, Kawano K, Harrington TL (1985) Neural mechanisms of space vision in the parietal association cortex of the monkey. Vis Res 25:453–463

    Article  Google Scholar 

  9. Sakata H, Shibutani H, Ito Y, Tsurugai K (1986) Parietal cortical neurons responding to rotary movement of visual stimulus in space. Exp Brain Res 61:658–663

    Article  Google Scholar 

  10. Hu B, Yue S, Zhang Z (2017) A rotational motion perception neural network based on asymmetric spatiotemporal visual information processing. IEEE Trans Neural Netw Learn Syst 28:2803–2821

    Article  MathSciNet  Google Scholar 

  11. Sakata H, Shibutani H, Ito Y et al (1994) Functional properties of rotation-sensitive neurons in the posterior parietal association cortex of the monkey. Exp Brain Res 101:183–202

    Article  Google Scholar 

  12. Sakata H, Taira M, Kusunoki M et al (1997) The parietal association cortex in depth perception and visual control of hand action. Trends Neurosci 20:350–357

    Article  Google Scholar 

  13. Wang H, Peng J, Zheng X, Yue S (2020) A robust visual system for small target motion detection against cluttered moving backgrounds. IEEE Trans Neural Networks Learn Syst 31:839–853

    Article  Google Scholar 

  14. Shojaei K (2019) Three-dimensional neural network tracking control of a moving target by underactuated autonomous underwater vehicles. Neural Comput Appl 31:509–521

    Article  Google Scholar 

  15. Li L, Zhang Z, Lu J (2021) Artificial fly visual joint perception neural network inspired by multiple-regional collision detection. Neural Netw 135:13–28

    Article  Google Scholar 

  16. Fu Q, Hu C, Peng J et al (2020) A robust collision perception visual neural network with specific selectivity to darker objects. IEEE Trans Cybern 50:5074–5088

    Article  Google Scholar 

  17. Maheshan MS, Harish BS, Nagadarshan N (2019) A convolution neural network engine for sclera recognition. Int J Interact Multimed Artif Intell 6:78–83

    Google Scholar 

  18. Liu D, Bellotto N, Yue S (2020) Deep spiking neural network for video-based disguise face recognition based on dynamic facial movements. IEEE Trans Neural Netw Learn Syst 31:1843–1855

    Article  Google Scholar 

  19. Jha S, Dey A, Kumar R, Kumar-Solanki V (2019) A novel approach on visual question answering by parameter prediction using faster region based convolutional neural network. Int J Interact Multimed Artif Intell 5:30–37

    Google Scholar 

  20. Hu B, Zhang Z, Li L (2019) LGMD-based visual neural network for detecting crowd escape behavior. In: Proceedings 2018 5th IEEE international conference cloud computing intelligent systems, CCIS 2018, vol 6, pp 772–778

  21. Chen J, Su W, Wang Z (2020) Crowd counting with crowd attention convolutional neural network. Neurocomputing 382:210–220

    Article  Google Scholar 

  22. Braunstein ML (1972) Perception of rotation in depth: a process model. Psychol Rev 79:510–524

    Article  Google Scholar 

  23. Hershberger WA, Stewart MR, Laughlin NK (1976) Conflicting motion perspective simulating simultaneous clockwise and counterclockwise rotation in depth. J Exp Psychol Hum Percept Perform 2:174–178

    Article  Google Scholar 

  24. Braunstein ML (1984) Perception of rotation in depth: the psychophysical evidence. ACM SIGGRAPH Comput Graph 18:25–26

    Article  Google Scholar 

  25. Shulman GL (1991) Attentional modulation of mechanisms that analyze rotation in depth. J Exp Psychol Hum Percept Perform 17:726–737

    Article  Google Scholar 

  26. Braunstein ML (1976) Depth perception through motion. Academic Press, London

    Google Scholar 

  27. Petersik JT (1980) Rotation judgments and depth judgments: separate or dependent processes? Percept Psychophys 27:588–590

    Article  Google Scholar 

  28. Andersen GJ, Braunstein ML (1983) Dynamic occlusion in the perception of rotation in depth. Percept Psychophys 34:356–362

    Article  Google Scholar 

  29. Johansson G, Jansson G (1968) Perceived rotary motion from changes in a straight line. Percept Psychophys 4:165–170

    Article  Google Scholar 

  30. Carpenter DL, Dugan MP (1983) Motion parallax information for direction of rotation in depth: order and direction components. Perception 12:559–569

    Article  Google Scholar 

  31. Miles FA (1998) The neural processing of 3-D visual information: evidence from eye movements. Eur J Neurosci 10:811–822

    Article  Google Scholar 

  32. Schaafsma SJ, Duysens J, Gielen CCAM (1997) Responses in ventral intraparietal area of awake macaque monkey to optic flow patterns corresponding to rotation of planes in depth can be explained by translation and expansion effects. Vis Neurosci 14:633–646

    Article  Google Scholar 

  33. Simmons PJ, Rind FC, Santer RD (2010) Escapes with and without preparation: the neuroethology of visual startle in locusts. J Insect Physiol 56:876–883

    Article  Google Scholar 

  34. Rind FC, Bramwell DI (1996) Neural network based on the input organization of an identified neuron signaling impending collision. J Neurophysiol 75:967–985

    Article  Google Scholar 

  35. Yue S, Rind FC (2006) Collision detection in complex dynamic scenes using an LGMD-based visual neural network with feature enhancement. IEEE Trans Neural Netw 17:705–716

    Article  Google Scholar 

  36. Yue S, Rind FC (2006) Visual motion pattern extraction and fusion for collision detection in complex dynamic scenes. Comput Vis Image Underst 104:48–60

    Article  Google Scholar 

  37. Yue S, Rind FC (2013) Postsynaptic organisations of directional selective visual neural networks for collision detection. Neurocomputing 103:50–62

    Article  Google Scholar 

  38. Gabriel JP, Trivedi CA, Maurer CM et al (2012) Layer-specific targeting of direction-selective neurons in the zebrafish optic tectum. Neuron 76:1147–1160

    Article  Google Scholar 

  39. Bereshpolova Y, Stoelzel CR, Su C et al (2019) Activation of a visual cortical column by a directionally selective thalamocortical neuron. Cell Rep 27:3733–3740

    Article  Google Scholar 

  40. Fried SI, Münch TA, Werblin FS (2002) Mechanisms and circuitry underlying directional selectivity in the retina. Nature 420:411–414

    Article  Google Scholar 

  41. Huang X, Rangel M, Briggman KL, Wei W (2019) Neural mechanisms of contextual modulation in the retinal direction selective circuit. Nat Commun 10:1–15

    Google Scholar 

  42. Fu Q, Yue S (2017) Modeling direction selective visual neural network with ON and OFF pathways for extracting motion cues from cluttered background. In: 2017 International joint conference on neural networks (IJCNN) IEEE 831–838

  43. Fu Q, Yue S (2020) Modelling Drosophila motion vision pathways for decoding the direction of translating objects against cluttered moving backgrounds. Biol Cybern 114:443–460

    Article  MATH  Google Scholar 

  44. Wei W (2018) Neural mechanisms of motion processing in the mammalian retina. Annu Rev Vis Sci 4:165–192

    Article  Google Scholar 

  45. Vlasits AL, Euler T, Franke K (2019) Function first: classifying cell types and circuits of the retina. Curr Opin Neurobiol 56:8–15

    Article  Google Scholar 

  46. Morrone MC, Burr DC, Vaina LM (1995) Two stages of visual processing for radial and circular motion. Nature 376:507–509

    Article  Google Scholar 

  47. Fu Q, Wang H, Hu C, Yue S (2019) Towards computational models and applications of insect visual systems for motion perception: a review. Artif Life 25:263–311

    Article  Google Scholar 

  48. Grünert U, Martin PR (2020) Cell types and cell circuits in human and non-human primate retina. Prog Retin Eye Res 78:1–33

    Article  Google Scholar 

  49. Field GD, Rieke F (2002) Nonlinear signal transfer from mouse rods to bipolar cells and implications for visual sensitivity. Neuron 34:773–785

    Article  Google Scholar 

  50. Gollisch T, Meister M (2010) Eye smarter than scientists believed: neural computations in circuits of the retina. Neuron 65:150–164

    Article  Google Scholar 

  51. Yang X, Wu SM (1991) Feedforward lateral inhibition in retinal bipolar cells: Input-output relation of the horizontal cell-depolarizing bipolar cell synapse. Proc Natl Acad Sci 88:3310–3313

    Article  Google Scholar 

  52. Thoreson WB, Mangel SC (2012) Lateral interactions in the outer retina. Prog Retin Eye Res 31:407–441

    Article  Google Scholar 

  53. Rind FC, Wernitznig S, Pölt P et al (2016) Two identified looming detectors in the locust: ubiquitous lateral connections among their inputs contribute to selective responses to looming objects. Sci Rep 6:1–16

    Article  Google Scholar 

  54. Hu B, Zhang Z (2018) Bio-plausible visual neural network for spatio-temporally spiral motion perception. Neurocomputing 310:96–114

    Article  Google Scholar 

  55. Yue S, Rind FC (2013) Redundant neural vision systems-competing for collision recognition roles. IEEE Trans Auton Ment Dev 5:173–186

    Article  Google Scholar 

  56. Albright TD, Desimone R, Gross CG (1984) Columnar organization of directionally selective cells in visual area MT of the macaque. J Neurophysiol 51:16–31

    Article  Google Scholar 

  57. Schneider M, Kemper VG, Emmerling TC et al (2019) Columnar clusters in the human motion complex reflect consciously perceived motion axis. Proc Natl Acad Sci U S A 116:5096–5101

    Article  Google Scholar 

  58. Beardsley SA, Ward RL, Vaina LM (2003) A neural network model of spiral-planar motion tuning in MSTd. Vision Res 43:577–595

    Article  Google Scholar 

Download references

Acknowledgements

The authors sincerely thank the anonymous reviewers for their valuable comments. We also thank the editors of this work for their support. The work is supported by National Natural Science Foundation of China (Nos. 62063002, 61563009, 62066006).

Author information

Authors and Affiliations

  1. Department of Computer Science, College of Computer Science and Technology, Guizhou University, Guiyang, 550025, China

    Bin Hu

  2. Department of Big Data Science and Engineering, College of Big Data and Information Engineering, Guizhou University, Guiyang, 550025, China

    Zhuhong Zhang

Authors
  1. Bin Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhuhong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhuhong Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, B., Zhang, Z. Bio-inspired visual neural network on spatio-temporal depth rotation perception. Neural Comput & Applic 33, 10351–10370 (2021). https://doi.org/10.1007/s00521-021-05796-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-05796-z

Keywords

Search

Navigation