logo

SCIENCE CHINA Information Sciences, Volume 61 , Issue 6 : 060423(2018) https://doi.org/10.1007/s11432-018-9425-1

Artificial neural networks based on memristive devices

More info
  • ReceivedFeb 22, 2018
  • AcceptedMar 27, 2018
  • PublishedMay 15, 2018

Abstract


References

[1] Mack C A. Fifty years of Moore's law. IEEE Trans Semicond Manufact, 2011, 24: 202-207 CrossRef Google Scholar

[2] Schulz M. The end of the road for silicon? Nature, 1999, 399: 729--730. Google Scholar

[3] Godfrey M D, Hendry D F. The computer as von Neumann planned it. IEEE Ann Hist Comput, 1993, 15: 11-21 CrossRef Google Scholar

[4] Anderson H C. Neural network machines. IEEE Potentials, 1989, 8: 13--16. Google Scholar

[5] Squire L R, Berg D, Bloom F, et al. Fundamental neuroscience. Curr Opin Neurobiol, 2008, 10: 649--654. Google Scholar

[6] Pedretti G, Milo V, Ambrogio S. Memristive neural network for on-line learning and tracking with brain-inspired spike timing dependent plasticity. Sci Rep, 2017, 7: 5288 CrossRef PubMed ADS Google Scholar

[7] Zamarre no-Ramos C, Camu nas-Mesa L A, Pérez-Carrasco J A, et al. On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front Neurosci-Switz, 2011, 5: 1--22. Google Scholar

[8] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521: 436-444 CrossRef PubMed ADS Google Scholar

[9] Li C, Hu M, Li Y. Analogue signal and image processing with large memristor crossbars. Nat Electron, 2018, 1: 52-59 CrossRef Google Scholar

[10] Hebb D O. The first stage of perception: growth of the assembly BT — the organization of behavior. Organ Behav, 1949, 4: 60--78. Google Scholar

[11] Wang Z, Joshi S, Savel'ev S E. Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing. Nat Mater, 2017, 16: 101-108 CrossRef PubMed ADS Google Scholar

[12] Hu M, Strachan J P, Li Z, et al. Dot-product engine for neuromorphic computing: programming 1T1M crossbar to accelerate matrix-vector multiplication. In: Proceedings of the 53rd ACM/EDAC/IEEE Design Automation Conference (DAC), Austin, 2016. 1--6. Google Scholar

[13] Shafiee A, Nag A, Muralimanohar N, et al. ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars. In: Proceedings of ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), Seoul, 2016. 14--26. Google Scholar

[14] Chi P, Li S, Xu C, et al. PRIME: a novel processing-in-memory architecture for neural network computation in reRAM-based main memory. In: Proceedings of ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), Seoul, 2016. 27--39. Google Scholar

[15] Tang S, Yin S, Zheng S, et al. AEPE: an area and power efficient RRAM crossbar-based accelerator for deep CNNs. In: Proceedings of IEEE 6th Non-Volatile Memory Systems and Applications Symposium (NVMSA), Hsinchu, 2017. 1--6. Google Scholar

[16] Yao P, Wu H, Gao B. Face classification using electronic synapses. Nat Commun, 2017, 8: 15199 CrossRef PubMed ADS Google Scholar

[17] Wang Z, Joshi S, Savel'ev S. Fully memristive neural networks for pattern classification with unsupervised learning. Nat Electron, 2018, 1: 137-145 CrossRef Google Scholar

[18] Sheridan P M, Cai F, Du C. Sparse coding with memristor networks. Nat Nanotech, 2017, 12: 784-789 CrossRef PubMed Google Scholar

[19] Hu M, Graves C E, Li C. Memristor-based analog computation and neural network classification with a dot product engine. Adv Mater, 2018, 30: 1705914 CrossRef PubMed Google Scholar

[20] Li C, Belkin D, Li Y, et al. Efficient and self-adaptive in-situ learning in multilayer memristor neural networks. Nat Commun, in press, 2018. Google Scholar

[21] Zarudnyi K, Mehonic A, Montesi L. Spike-timing dependent plasticity in unipolar silicon oxide RRAM devices. Front Neurosci, 2018, 12: 57 CrossRef Google Scholar

[22] Yu S M, Li Z W, Chen P-Y, et al. Binary neural network with 16 Mb RRAM macro chip for classification and online training. In: Proceedings of IEEE International Electron Devices Meeting (IEDM), San Francisco, 2016. 1--4. Google Scholar

[23] Prezioso M, Merrikh-Bayat F, Hoskins B D. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature, 2015, 521: 61-64 CrossRef PubMed ADS arXiv Google Scholar

[24] Yu S. Neuro-inspired computing with emerging nonvolatile memorys. Proc IEEE, 2018, 106: 260-285 CrossRef Google Scholar

[25] Alibart F, Zamanidoost E, Strukov D B, et al. Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat Commun, 2013, 2013: 2072. Google Scholar

[26] Merrikh B F, Prezioso M, Chakrabarti B, et al. Advancing memristive analog neuromorphic networks: increasing complexity, and coping with imperfect hardware components,. arXiv Google Scholar

[27] Du C, Cai F, Zidan M A. Reservoir computing using dynamic memristors for temporal information processing. Nat Commun, 2017, 8: 2204 CrossRef PubMed ADS Google Scholar

[28] Mehonic A, Kenyon A J. Emulating the electrical activity of the neuron using a silicon oxide RRAM cell. Front Neurosci-Switz, 2016, 10: 57. Google Scholar

[29] Chen Y J, Luo T, Liu S L, et al. DaDianNao: a machine-learning supercomputer. In: Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture, Cambridge, 2014. 609--622. Google Scholar

[30] Merolla P A, Arthur J V, Alvarez-Icaza R. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 2014, 345: 668-673 CrossRef PubMed ADS Google Scholar

[31] Jouppi N P, Young C, Patil N, et al. In-Datacenter performance analysis of a tensor processing unit. In: Proceedings of the 44th Annual International Symposium on Computer Architecture, Toronto, 2017. 1--12. Google Scholar

[32] Gawande N A, Landwehr J B, Daily J A, et al. Scaling deep learning workloads: NVIDIA DGX-1/Pascal and intel knights landing. In: Proceedings of 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Lake Buena Vista, 2017. 399--408. Google Scholar

[33] Chua L O. Memristor — the missing circuit element. IEEE Trans Circuits Syst, 1971, 18: 507--519. Google Scholar

[34] Strukov D B, Snider G S, Stewart D R. The missing memristor found. Nature, 2008, 453: 80-83 CrossRef PubMed ADS Google Scholar

[35] Yang J J, Pickett M D, Li X. Memristive switching mechanism for metal/oxide/metal nanodevices. Nat Nanotech, 2008, 3: 429-433 CrossRef PubMed Google Scholar

[36] Joshua Yang J, Miao F, Pickett M D. The mechanism of electroforming of metal oxide memristive switches. Nanotechnology, 2009, 20: 215201 CrossRef PubMed ADS Google Scholar

[37] Chua L. Resistance switching memories are memristors. Appl Phys A, 2011, 102: 765-783 CrossRef ADS Google Scholar

[38] Yang J J, Strukov D B, Stewart D R. Memristive devices for computing. Nat Nanotech, 2013, 8: 13-24 CrossRef PubMed ADS Google Scholar

[39] Yang J J, Williams R S. Memristive devices in computing system: promises and challenges. ACM J Emerg Tech Com, 2013, 9: 1--20. Google Scholar

[40] Pickett M D, Strukov D B, Borghetti J L. Switching dynamics in titanium dioxide memristive devices. J Appl Phys, 2009, 106: 074508 CrossRef ADS Google Scholar

[41] Alibart F, Gao L, Hoskins B D. High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm. Nanotechnology, 2012, 23: 075201 CrossRef PubMed ADS arXiv Google Scholar

[42] Choi S, Sheridan P, Lu W D. Data clustering using memristor networks. Sci Rep, 2015, 5: 10492 CrossRef PubMed ADS Google Scholar

[43] Yang J J, Zhang M X, Strachan J P. High switching endurance in TaO$_{x}$ memristive devices. Appl Phys Lett, 2010, 97: 232102 CrossRef ADS Google Scholar

[44] Choi B J, Torrezan A C, Strachan J P. High-speed and low-energy nitride memristors. Adv Funct Mater, 2016, 26: 5290-5296 CrossRef Google Scholar

[45] Yoon J H, Zhang J, Ren X. Truly Electroforming-Free and Low-Energy Memristors with Preconditioned Conductive Tunneling Paths. Adv Funct Mater, 2017, 27: 1702010 CrossRef Google Scholar

[46] Li C, Han L, Jiang H. Three-dimensional crossbar arrays of self-rectifying Si/SiO$_{2}$/Si memristors. Nat Commun, 2017, 8: 15666 CrossRef PubMed ADS Google Scholar

[47] Shulaker M M, Hills G, Park R S. Three-dimensional integration of nanotechnologies for computing and data storage on a single chip. Nature, 2017, 547: 74-78 CrossRef PubMed ADS Google Scholar

[48] Yu S, Wu Y, Jeyasingh R. An Electronic Synapse Device Based on Metal Oxide Resistive Switching Memory for Neuromorphic Computation. IEEE Trans Electron Devices, 2011, 58: 2729-2737 CrossRef ADS Google Scholar

[49] Chang T, Yang Y, Lu W. Building Neuromorphic Circuits with Memristive Devices. IEEE Circuits Syst Mag, 2013, 13: 56-73 CrossRef Google Scholar

[50] Chang T, Jo S H, Kim K H. Synaptic behaviors and modeling of a metal oxide memristive device. Appl Phys A, 2011, 102: 857-863 CrossRef ADS Google Scholar

[51] Gaba S, Sheridan P, Zhou J. Stochastic memristive devices for computing and neuromorphic applications. Nanoscale, 2013, 5: 5872-5878 CrossRef PubMed ADS arXiv Google Scholar

  • Figure 1

    (Color online) Neuron and synapse structure. (a) Simplified biological structure of pre-synaptic and post-synaptic neurons showing the axon, synapses, and dendrites [6]@Copyright 2017 Scientific Reports; (b) biological synaptic structure and function [7]@Copyright 2011 Frontiers in Neuroscience.

  • Figure 2

    (Color online) (a) Memristor crossbar for vector matrix multiplication (VMM); (b) current (mA) output was experimentally measured for a 6464 crossbar for VMM and the resulting current from column 1 and column 64 were compared with the expected values using software VMM. Non-idealities in the devices and circuitry result in a drastic difference between the ideal output and the actual output [9]@Copyright 2018 Nature Electronics.

  • Figure 3

    (a) ISAAC architecture using tiles [13]. (b) Simplified design of PRIME [14]. External I/O data is fed into RRAM crossbar based banks. Memory subarrays only store data. Full function subarrays can store data or perform computation, using a CMOS controller to enable configuration. The buffer subarrays serve as data buffers for the full function subarrays. (c) AEPE Implementation with efficient tile architecture [15]. The register buffer feeds into M rows of crossbar based PE arrays through an assigned DAC for each row.

  • Figure 4

    (Color online) (a) A single-layer is mapped onto the one transistor one memristor (1T1R) array; (b) micrograph of the 1024 cell 1T1R array fabricated using CMOS compatible processes [16]@Copyright 2017 Nature Communications.

  • Figure 5

    (Color online) (a) Photograph of fabricated 1T1R crossbars. Two dies are shown, each one containing various array sizes from 4$\times$4 to 128$\times$64 cells. (b) Micrograph of four cells in a 1T1R array (scale bar, 10 mm). (c) Relatively linear I-V curves for all the devices over the chosen conductance range. (d) Device state retention and read disturbance (1000 cycles of 0.2 V read pulses) at room temperature show no discernible drift [9]@Copyright 2017 Nature Electronics.

  • Figure 6

    (Color online) (a) Synapse array with peripheral circuitry; (b) a pre-synaptic device controls the FET gate of the 1T1R structure, while the post-synaptic device receives the input current and controls the synapse top electrode to induce synaptic current and stimulate synaptic potentiation or depression during the fire [6]@Copyright 2017 Scientific Reports.

  • Figure 7

    (Color online) Diffusive memristor neuron schematic. The diffusive memristor behaves in a fundamentally similar manner as a biological ion channel on the soma membrane of a neuron [17]@Copyright 2018 Nature Electronics.

  • Figure 8

    (Color online) (a) All memristor network fabricated using (b) and (c) drift memristors for synapses and (d) and (e) diffusive memristors for neurons [17]@Copyright 2018 Nature Electronics.

  • Table 1   Summary of experimental memristor based network implementations presented in literature$^{\rm~a)}$
    ImplementationComputing efficiencyAccuracyPower consumptionMemristor used/Dielectric thicknessCrossbar cell countReference
    1T1R face classifying perceptron networkConverges within 10 iterations88% recognition accuracy within 10 iterations$\sim$30 nJ per epoch for classification taskMemristor-8 nm HfAl$_{y}$O$_{x}$128$\times$8 cells[16]
    1T1R Ta/HfO$_{2}$ memristor network119.7 trillion operations per second per watt91.71% recognition accuracy on MNIST data set13.7 mW for image compressionMemristor-5 nm HfO$_{2}$128$\times$64 cells[9,19,20]
    1T1R STDP networkMemristor-10 nm HfO$_{2}$4$\times$4 pre-cells and 1 post-cell[6]
    Diffusive memristor artificial neuron networkDiffusive memristor-10 nm Ag/SiO$_{2}$ par Drift par Memristor-5 nm HfO$_{2}$8$\times$8 synaptic crossbar with 8 neurons[17]

    a