Skip to main content
SpringerLink
Log in

Adaptive sparse coding based on memristive neural network with applications

  • Research Article
  • Published:
Cognitive Neurodynamics Aims and scope Submit manuscript

Abstract

Memristor is a nanoscale circuit element with nonvolatile, binary, multilevel and analog states. Its conductance (resistance) plasticity is similar to biological synapses. Information sparse coding is considered as the key mechanism of biological neural systems to process mass complex perception data, which is applied in the fields of signal processing, computer vision and so on. This paper proposes a soft-threshold adaptive sparse coding algorithm named MMN-SLCA based on the memristor, neural network and sparse coding theory. Specifically, the memristor crossbar array is used to realize the dictionary set. And by leveraging its unique vector–matrix operation advantages and biological synaptic characteristic, two key compositions of the sparse coding, namely, pattern matching and lateral neuronal inhibition are realized conveniently and efficiently. Besides, threshold variability further enhances the adaptive ability of the intelligent sparse coding. Furthermore, a hardware implementation framework of the sparse coding algorithm is designed to provide feasible solutions for hardware acceleration, real-time processing and embedded applications. Finally, the application of MMN-SLCA in image super-resolution reconstruction is discussed. Experimental simulations and result analysis verify the effectiveness of the proposed scheme and show its superior potentials in large-scale low-power intelligent information coding and processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  • Adhikari SP, Yang C, Kim H (2012) Memristor bridge synapse-based neural network and its learning. IEEE Trans Neural Netw Learn Syst 23(9):1426–1435

    Article  PubMed  Google Scholar 

  • Bao B, Jiang T, Wang G (2017) Two-memristor-based Chua’s hyperchaotic circuit with plane equilibrium and its extreme multistability. Nonlinear Dyn 89(2):1157–1171

    Article  Google Scholar 

  • Bao H, Wang N, Bao B (2018) Initial condition-dependent dynamics and transient period in memristor-based hypogenetic jerk system with four line equilibria. Commun Nonlinear Sci Numer Simul 57:264–275

    Article  Google Scholar 

  • Bao H, Liu W, Hu A (2019) Coexisting multiple firing patterns in two adjacent neurons coupled by memristive electromagnetic induction. Nonlinear Dyn 95(1):43–56

    Article  Google Scholar 

  • Candès EJ, Wakin MB (2008) An introduction to compressive sampling. IEEE Signal Process Mag 25(2):21–30

    Article  Google Scholar 

  • Chen L, Li C, Huang T (2013) A synapse memristor model with forgetting effect. Phys Lett A 377(45–48):3260–3265

    Article  CAS  Google Scholar 

  • Chua L (1971) Memristor-the missing circuit element. IEEE Trans Circuit Theory 18(5):507–519

    Article  Google Scholar 

  • Donoho DL, Elad M (2003) Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization. Proc Natl Acad Sci 100(5):2197–2202

    Article  CAS  PubMed  Google Scholar 

  • Field DJ (1987) Relations between the statistics of natural images and the response properties of cortical cells. J Opt Soc 4(12):2379–2394

    Article  CAS  Google Scholar 

  • Field DJ (1989) What the statistics of natural images tell us about visual coding. Hum Vis Vis Process Digit Disp 1077:269–277

    Article  Google Scholar 

  • Hu X, Feng G, Duan S (2017) A memristive multilayer cellular neural network with applications to image processing. IEEE Trans Neural Netw Learn Syst 28(8):1889–1901

    Article  PubMed  Google Scholar 

  • Itoh M, Chua L (2014) Memristor cellular automata and memristor discrete-time cellular neural networks. In: Memristor networks. Springer, Cham, pp 649–713

  • Jo SH, Chang T (2010) Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett 10(4):1297–1301

    Article  CAS  PubMed  Google Scholar 

  • Kawahara A, Azuma R, Ikeda Y (2013) An 8 Mb multi-layered cross-point ReRAM macro with 443 MB/s write throughput. IEEE J Solid-State Circuits 48(1):178–185

    Article  Google Scholar 

  • Kvatinsky S, Friedman EG, Kolodny A (2013) TEAM: threshold adaptive memristor model. IEEE Trans Circuits Syst I Regul Pap 60(1):211–221

    Article  Google Scholar 

  • Lazzaro J, Ryckebusch S, Mahowald M A (1989) Winner-take-all networks of O(n) complexity. In: Advances in neural information processing systems, pp 703–711

  • Li Y, Cichocki A, Amari S (2004) Analysis of sparse representation and blind source separation. Neural Comput 16(6):1193–1234

    Article  PubMed  Google Scholar 

  • Long S, Perniola L, Cagli C (2013) Voltage and power-controlled regimes in the progressive unipolar RESET transition of HfO 2-based RRAM. Sci Rep 3:2929

    Article  PubMed  PubMed Central  Google Scholar 

  • Muenstermann R, Menke T, Dittmann R (2010) Coexistence of filamentary and homogeneous resistive switching in Fe-doped SrTiO3 thin-film memristive devices. Adv Mater 22(43):4819–4822

    Article  CAS  PubMed  Google Scholar 

  • Olshausen BA, Field DJ (1997) Sparse coding with an overcomplete basis set: A strategy employed by V1? Vis Res 37(23):3311–3325

    Article  CAS  PubMed  Google Scholar 

  • Pershin YV, Di Ventra M (2010) Practical approach to programmable analog circuits with memristors. IEEE Trans Circuits Syst I Regul Pap 57(8):1857–1864

    Article  Google Scholar 

  • Rozell CJ, Johnson DH, Baraniuk RG (2008) Sparse coding via thresholding and local competition in neural circuits. Neural Comput 20(10):2526–2563

    Article  PubMed  Google Scholar 

  • Sheridan PM, Cai F, Du C (2017) Sparse coding with memristor networks. Nat Nanotechnol 12(8):784

    Article  CAS  PubMed  Google Scholar 

  • Snider G, Amerson R, Carter D (2011) From synapses to circuitry: using memristive memory to explore the electronic brain. Computer 44(2):21–28

    Article  Google Scholar 

  • Strukov DB, Snider GS, Stewart DR (2008) The missing memristor found. Nature 453(7191):80–83

    Article  CAS  PubMed  Google Scholar 

  • Wang X, Chen Y, Xi H (2009) Spintronic memristor through spin-torque-induced magnetization motion. IEEE Electron Device Lett 30(3):294–297

    Article  Google Scholar 

  • Wang M, Cai S, Pan C (2018) Robust memristors based on layered two-dimensional materials. Nat Electron 1(2):130

    Article  CAS  Google Scholar 

  • Wright J, Ma Y, Mairal J (2010) Sparse representation for computer vision and pattern recognition. Proc IEEE 98(6):1031–1044

    Article  Google Scholar 

  • Yan B, Chen Y, Li H (2018) Challenges of memristor based neuromorphic computing system. Sci China Inf Sci 61(6):060425

  • Yang JJ, Strukov DB, Stewart DR (2013) Memristive devices for computing. Nat Nanotechnol 8(1):13

    Article  CAS  PubMed  Google Scholar 

  • Yang Y, Chang T, Lu W (2014) Memristive devices: switching effects, modeling, and applications. In: Memristors and memristive systems. Springer, New York, NY, pp 195–221

  • Yesil A (2018) A new grounded memristor emulator based on MOSFET-C. AEU-Int J Electron Commun 91:143–149

    Article  Google Scholar 

  • Zakhidov AA, Jung B, Slinker JD (2010) A light-emitting memristor. Org Electron 11(1):150–153

    Article  CAS  Google Scholar 

  • Zhang F, Duan S, Wang L (2017) Route searching based on neural networks and heuristic reinforcement learning. Cogn Neurodyn 11(3):245–258

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant No. 61601376, 61672436), Fundamental Research Funds for the Central Universities (XDJK2019C034), Fundamental Science and Advanced Technology Research Foundation of Chongqing (cstc2016jcyjA0547), China Postdoctoral Science Foundation Special Funded (2018T110937), Chongqing Postdoctoral Science Foundation Special Funded (Xm2017039), Student’s Platform for Innovation and Entrepreneurship Training Program (201810635017).

Author information

Authors and Affiliations

  1. College of Computer and Information Science, Southwest University, Chongqing, 400715, China

    Xun Ji

  2. College of Artificial Intelligence, Southwest University, Chongqing, 400715, China

    Xiaofang Hu, Yue Zhou & Shukai Duan

  3. Brain-Inspired Computing and Intelligent Control of Chongqing Key Lab, Southwest University, Chongqing, 400715, China

    Xiaofang Hu, Yue Zhou & Shukai Duan

  4. College of Electrical Engineering, Zhejiang University, Hangzhou, 310027, China

    Zhekang Dong

Authors
  1. Xun Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaofang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yue Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhekang Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shukai Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaofang Hu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, X., Hu, X., Zhou, Y. et al. Adaptive sparse coding based on memristive neural network with applications. Cogn Neurodyn 13, 475–488 (2019). https://doi.org/10.1007/s11571-019-09537-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11571-019-09537-w

Keywords

Search

Navigation