SCIENCE CHINA Information Sciences, Volume 63 , Issue 6 : 160401(2020) https://doi.org/10.1007/s11432-020-2863-y

Towards an intelligent photonic system

More info
  • ReceivedJan 21, 2020
  • AcceptedApr 1, 2020
  • PublishedMay 9, 2020



This work was supported by National Key RD Program of China (Grant No. 2019YFB2203700) and National Natural Science Foundation of China (Grant No. 61822508).


[1] Kikuchi K. Fundamentals of Coherent Optical Fiber Communications. J Lightw Technol, 2016, 34: 157-179 CrossRef ADS Google Scholar

[2] Yao J. Microwave Photonics. J Lightw Technol, 2009, 27: 314-335 CrossRef ADS Google Scholar

[3] Liang J, Wang L V. Single-shot ultrafast optical imaging. Optica, 2018, 5: 1113-1127 CrossRef ADS Google Scholar

[4] Chen J H, Li D R, Xu F. Optical Microfiber Sensors: Sensing Mechanisms, and Recent Advances. J Lightwave Technol, 2019, 37: 2577-2589 CrossRef ADS Google Scholar

[5] Capmany J, Novak D. Microwave photonics combines two worlds. Nat Photon, 2007, 1: 319-330 CrossRef ADS Google Scholar

[6] Sun C, Wade M T, Lee Y. Single-chip microprocessor that communicates directly using light. Nature, 2015, 528: 534-538 CrossRef PubMed ADS Google Scholar

[7] Khan M H, Shen H, Xuan Y. Ultrabroad-bandwidth arbitrary radiofrequency waveform generation with a silicon photonic chip-based spectral shaper. Nat Photon, 2010, 4: 117-122 CrossRef ADS Google Scholar

[8] Zhuang L, Roeloffzen C G H, Hoekman M. Programmable photonic signal processor chip for radiofrequency applications. Optica, 2015, 2: 854-859 CrossRef ADS arXiv Google Scholar

[9] Miller D A B. Self-configuring universal linear optical component [Invited]. Photon Res, 2013, 1: 1 CrossRef Google Scholar

[10] Pérez D, Gasulla I, Crudgington L. Multipurpose silicon photonics signal processor core. Nat Commun, 2017, 8: 636 CrossRef PubMed ADS Google Scholar

[11] Perez D, Gasulla I, Capmany J. Toward Programmable Microwave Photonics Processors. J Lightwave Technol, 2018, 36: 519-532 CrossRef ADS Google Scholar

[12] Zhang J, Yao J. A Microwave Photonic Signal Processor for Arbitrary Microwave Waveform Generation and Pulse Compression. J Lightwave Technol, 2016, 34: 5610-5615 CrossRef ADS Google Scholar

[13] García-Meca C, Lechago S, Brimont A. On-chip wireless silicon photonics: from reconfigurable interconnects to lab-on-chip devices. Light Sci Appl, 2017, 6: e17053-e17053 CrossRef PubMed ADS Google Scholar

[14] Silver D, Huang A, Maddison C J. Mastering the game of Go with deep neural networks and tree search. Nature, 2016, 529: 484-489 CrossRef PubMed ADS Google Scholar

[15] Silver D, Schrittwieser J, Simonyan K. Mastering the game of Go without human knowledge. Nature, 2017, 550: 354-359 CrossRef PubMed ADS Google Scholar

[16] Topol E J. High-performance medicine: the convergence of human and artificial intelligence.. Nat Med, 2019, 25: 44-56 CrossRef PubMed Google Scholar

[17] Abdel-Hamid O, Mohamed A, Jiang H. Convolutional Neural Networks for Speech Recognition. IEEE/ACM Trans Audio Speech Lang Process, 2014, 22: 1533-1545 CrossRef Google Scholar

[18] Brown N, Sandholm T. Superhuman AI for multiplayer poker. Science, 2019, 365: 885-890 CrossRef PubMed ADS Google Scholar

[19] Winfield A. Ethical standards in robotics and AI. Nat Electron, 2019, 2: 46-48 CrossRef Google Scholar

[20] Wang J G, Zhou L B. Traffic Light Recognition With High Dynamic Range Imaging and Deep Learning. IEEE Trans Intell Transp Syst, 2019, 20: 1341-1352 CrossRef Google Scholar

[21] Minasian R A. Photonic signal processing of microwave signals. IEEE Trans Microwave Theor Techn, 2006, 54: 832-846 CrossRef ADS Google Scholar

[22] Miller D A B. Perfect Optics With Imperfect Components. Optica, 2015, 2: 747-750 CrossRef ADS Google Scholar

[23] Yang G, Zou W, Yu L. Compensation of multi-channel mismatches in high-speed high-resolution photonic analog-to-digital converter. Opt Express, 2016, 24: 24061-24074 CrossRef PubMed ADS Google Scholar

[24] Minzioni P, Alberti F, Schiffini A. Techniques for Nonlinearity Cancellation Into Embedded Links by Optical Phase Conjugation. J Lightwave Technol, 2005, 23: 2364-2370 CrossRef ADS Google Scholar

[25] Park S W, Park J Y, Bong K, et al. An energy-efficient and scalable deep learning/inference processor with tetra-parallel MIMD architecture for big data applications. IEEE Trans Biomed Circ Syst, 2015, 9: 838--848. Google Scholar

[26] Waldrop M M. The chips are down for Moore's law.. Nature, 2016, 530: 144-147 CrossRef PubMed ADS Google Scholar

[27] Tait A N, Ferreira de Lima T, Nahmias M A. Silicon Photonic Modulator Neuron. Phys Rev Appl, 2019, 11: 064043 CrossRef ADS arXiv Google Scholar

[28] Denève S, Alemi A, Bourdoukan R. The Brain as an Efficient and Robust Adaptive Learner.. Neuron, 2017, 94: 969-977 CrossRef PubMed Google Scholar

[29] Roy K, Jaiswal A, Panda P. Towards spike-based machine intelligence with neuromorphic computing. Nature, 2019, 575: 607-617 CrossRef PubMed ADS Google Scholar

[30] Maass W, Natschl?ger T, Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations.. Neural Computation, 2002, 14: 2531-2560 CrossRef PubMed Google Scholar

[31] Ma W, Zidan M A, Lu W D. Neuromorphic computing with memristive devices. Sci China Inf Sci, 2018, 61: 060422. Google Scholar

[32] Wu N J. Neuromorphic vision chips. Sci China Inf Sci, 2018, 61: 060421. Google Scholar

[33] Yan B N, Chen Y R, Li H. Challenges of memristor based neuromorphic computing system. Sci China Inf Sci, 2018, 61: 060425. Google Scholar

[34] Cully A, Clune J, Tarapore D. Robots that can adapt like animals. Nature, 2015, 521: 503-507 CrossRef PubMed ADS arXiv Google Scholar

[35] Barbastathis G, Ozcan A, Situ G. On the use of deep learning for computational imaging. Optica, 2019, 6: 921-943 CrossRef ADS Google Scholar

[36] Rivenson Y, G?r?cs Z, Günaydin H. Deep learning microscopy. Optica, 2017, 4: 1437-1443 CrossRef ADS arXiv Google Scholar

[37] Sinha A, Lee J, Li S. Lensless computational imaging through deep learning. Optica, 2017, 4: 1117-1125 CrossRef ADS arXiv Google Scholar

[38] Wu Y, Rivenson Y, Zhang Y. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery. Optica, 2018, 5: 704-710 CrossRef ADS arXiv Google Scholar

[39] Zhang X, Chen Y, Ning K. Deep learning optical-sectioning method. Opt Express, 2018, 26: 30762-30772 CrossRef PubMed ADS Google Scholar

[40] Manifold B, Thomas E, Francis A T. Denoising of stimulated Raman scattering microscopy images via deep learning.. Biomed Opt Express, 2019, 10: 3860-3874 CrossRef PubMed Google Scholar

[41] Esman D J, Ataie V, Kuo B P P. Comb-Assisted Cyclostationary Analysis of Wideband RF Signals. J Lightwave Technol, 2017, 35: 3705-3712 CrossRef ADS Google Scholar

[42] Ma M, Adams R, Chen L R. Integrated Photonic Chip Enabled Simultaneous Multichannel Wideband Radio Frequency Spectrum Analyzer. J Lightwave Technol, 2017, 35: 2622-2628 CrossRef ADS Google Scholar

[43] Fortier T, Baumann E. 20 years of developments in optical frequency comb technology and applications. Commun Phys, 2019, 2: 153 CrossRef ADS arXiv Google Scholar

[44] Hammond A M, Camacho R M. Designing integrated photonic devices using artificial neural networks. Opt Express, 2019, 27: 29620-29638 CrossRef PubMed ADS arXiv Google Scholar

[45] Malkiel I, Mrejen M, Nagler A. Plasmonic nanostructure design and characterization via Deep Learning. Light Sci Appl, 2018, 7: 60 CrossRef PubMed ADS Google Scholar

[46] Laporte F, Dambre J, Bienstman P. Highly parallel simulation and optimization of photonic circuits in time and frequency domain based on the deep-learning framework PyTorch. Sci Rep, 2019, 9: 5918 CrossRef PubMed ADS Google Scholar

[47] Zahavy T, Dikopoltsev A, Moss D. Deep learning reconstruction of ultrashort pulses. Optica, 2018, 5: 666-673 CrossRef ADS arXiv Google Scholar

[48] Xu S, Zou X, Ma B. Deep-learning-powered photonic analog-to-digital conversion. Light Sci Appl, 2019, 8: 66 CrossRef PubMed ADS Google Scholar

[49] Zou X, Xu S, Li S. Optimization of the Brillouin instantaneous frequency measurement using convolutional neural networks. Opt Lett, 2019, 44: 5723-5726 CrossRef PubMed ADS Google Scholar

[50] Shen Y, Harris N C, Skirlo S. Deep learning with coherent nanophotonic circuits. Nat Photon, 2017, 11: 441-446 CrossRef ADS arXiv Google Scholar

[51] Lin X, Rivenson Y, Yardimci N T. All-optical machine learning using diffractive deep neural networks. Science, 2018, 361: 1004-1008 CrossRef PubMed ADS arXiv Google Scholar

[52] Hamerly R, Bernstein L, Sludds A. Large-Scale Optical Neural Networks Based on Photoelectric Multiplication. Phys Rev X, 2019, 9: 021032 CrossRef ADS arXiv Google Scholar

[53] Bangari V, Marquez B A, Miller H. Digital Electronics and Analog Photonics for Convolutional Neural Networks (DEAP-CNNs). IEEE J Sel Top Quantum Electron, 2020, 26: 1-13 CrossRef ADS arXiv Google Scholar

[54] Williamson I A D, Hughes T W, Minkov M. Reprogrammable Electro-Optic Nonlinear Activation Functions for Optical Neural Networks. IEEE J Sel Top Quantum Electron, 2020, 26: 1-12 CrossRef ADS arXiv Google Scholar

[55] George J K, Mehrabian A, Amin R. Neuromorphic photonics with electro-absorption modulators. Opt Express, 2019, 27: 5181-5191 CrossRef PubMed ADS arXiv Google Scholar

[56] Zuo Y, Li B, Zhao Y. All-optical neural network with nonlinear activation functions. Optica, 2019, 6: 1132-1137 CrossRef ADS arXiv Google Scholar

[57] Mourgias-Alexandris G, Tsakyridis A, Passalis N. An all-optical neuron with sigmoid activation function. Opt Express, 2019, 27: 9620-9630 CrossRef PubMed ADS Google Scholar

[58] Miscuglio M, Mehrabian A, Hu Z. All-optical nonlinear activation function for photonic neural networks [Invited]. Opt Mater Express, 2018, 8: 3851-3863 CrossRef ADS arXiv Google Scholar

[59] Hughes T W, Minkov M, Shi Y. Training of photonic neural networks through in situ backpropagation and gradient measurement. Optica, 2018, 5: 864-871 CrossRef ADS arXiv Google Scholar

[60] Xu S, Wang J, Wang R. High-accuracy optical convolution unit architecture for convolutional neural networks by cascaded acousto-optical modulator arrays. Opt Express, 2019, 27: 19778 CrossRef PubMed ADS Google Scholar

[61] Xu S F, Wang J, Zou W W. High-energy-efficiency integrated photonic convolutional neural networks,. arXiv Google Scholar

[62] Prucnal P R, Shastri B J. Neuromorphic Photonics. Boca Raton: CRC Press, 2017. Google Scholar

[63] Nahmias M A, Shastri B J, Tait A N. A Leaky Integrate-and-Fire Laser Neuron for Ultrafast Cognitive Computing. IEEE J Sel Top Quantum Electron, 2013, 19: 1-12 CrossRef ADS Google Scholar

[64] Robertson J, Wade E, Kopp Y. Toward Neuromorphic Photonic Networks of Ultrafast Spiking Laser Neurons. IEEE J Sel Top Quantum Electron, 2020, 26: 1-15 CrossRef ADS Google Scholar

[65] Xiang S Y, Zhang H, Guo X X. Cascadable Neuron-Like Spiking Dynamics in Coupled VCSELs Subject to Orthogonally Polarized Optical Pulse Injection. IEEE J Sel Top Quantum Electron, 2017, 23: 1-7 CrossRef ADS Google Scholar

[66] Prucnal P R, Shastri B J, Ferreira de Lima T. Recent progress in semiconductor excitable lasers for photonic spike processing. Adv Opt Photon, 2016, 8: 228-299 CrossRef ADS Google Scholar

[67] Chakraborty I, Saha G, Roy K. Photonic In-Memory Computing Primitive for Spiking Neural Networks Using Phase-Change Materials. Phys Rev Appl, 2019, 11: 014063 CrossRef ADS Google Scholar

[68] Xiang S, Ren Z, Zhang Y. All-optical neuromorphic XOR operation with inhibitory dynamics of a single photonic spiking neuron based on a VCSEL-SA.. Opt Lett, 2020, 45: 1104-1107 CrossRef PubMed Google Scholar

[69] Cheng Z, Ríos C, Pernice W H P. On-chip photonic synapse. Sci Adv, 2017, 3: e1700160 CrossRef PubMed ADS Google Scholar

[70] Tait A N, Nahmias M A, Shastri B J. Broadcast and Weight: An Integrated Network For Scalable Photonic Spike Processing. J Lightwave Technol, 2014, 32: 4029-4041 CrossRef ADS Google Scholar

[71] Feldmann J, Youngblood N, Wright C D. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature, 2019, 569: 208-214 CrossRef PubMed ADS Google Scholar

[72] Xiang S, Zhang Y, Gong J. STDP-Based Unsupervised Spike Pattern Learning in a Photonic Spiking Neural Network With VCSELs and VCSOAs. IEEE J Sel Top Quantum Electron, 2019, 25: 1-9 CrossRef ADS Google Scholar

[73] Ren Q, Zhang Y, Wang R. Optical spike-timing-dependent plasticity with weight-dependent learning window and reward modulation. Opt Express, 2015, 23: 25247-25258 CrossRef PubMed ADS Google Scholar

[74] Toole R, Tait A N, Ferreira de Lima T. Photonic Implementation of Spike-Timing-Dependent Plasticity and Learning Algorithms of Biological Neural Systems. J Lightwave Technol, 2016, 34: 470-476 CrossRef ADS Google Scholar

[75] Fok M P, Tian Y, Rosenbluth D. Pulse lead/lag timing detection for adaptive feedback and control based on optical spike-timing-dependent plasticity. Opt Lett, 2013, 38: 419-421 CrossRef PubMed ADS Google Scholar

[76] Ma B W, Chen J P, Zou W W. A DFB-LD-based photonic neuromorphic network for spatiotemporal pattern recognition. In: Proceedings of Optical Fiber Communication Conference, San Diego, 2020. M2K.2. Google Scholar

[77] Smit M, Leijtens X. Integration of passive and active components in InP-Based PICs In: Proceedings of Advances in Optical Sciences Congress, Honolulu, 2009. ITuB2. Google Scholar

[78] van Emmerik C I, Dijkstra M, de Goede M. Single-layer active-passive Al2O3 photonic integration platform. Opt Mater Express, 2018, 8: 3049-3054 CrossRef ADS Google Scholar

[79] de Valicourt G, Chang C M, Eggleston M S. Photonic Integrated Circuit Based on Hybrid III-V/Silicon Integration. J Lightwave Technol, 2018, 36: 265-273 CrossRef ADS Google Scholar

[80] Yoo S J B, Guan B, Scott R P. Heterogeneous 2D/3D Photonic Integrated Microsystems. Microsyst Nanoeng, 2016, 2: 16030 CrossRef PubMed ADS Google Scholar

[81] Hill M, Smit M, Crombez P, et al. Digital vs. Analog photonic integration In: Proceedings of Integrated Photonics and Nanophotonics Research and Applications, Boston, 2008. IWC1. Google Scholar

[82] Atabaki A H, Moazeni S, Pavanello F. Integrating photonics with silicon nanoelectronics for the next generation of systems on a chip. Nature, 2018, 556: 349-354 CrossRef PubMed ADS Google Scholar

[83] Sengupta K, Nagatsuma T, Mittleman D M. Terahertz integrated electronic and hybrid electronic-photonic systems. Nat Electron, 2018, 1: 622-635 CrossRef Google Scholar

  • Figure 1

    (Color online) The hierarchy of the IPS concept. DEC: digital electronic circuit.

  • Figure 2

    (Color online) The architecture of an AI-powered IPS.

  • Figure 3

    (Color online) (a) The training process of the DNN for microscopic imaging; (b) after training, the output of the DNN shows improved performance [35]@Copyright 2017 The Optical Society; (c) schematic of the deep learning microscopy.

  • Figure 4

    (Color online) Schematic of the DL-PADC architecture [44]@Copyright 2019 Springer Nature.

  • Figure 5

    (Color online) Optimized results of different signal formats using CNN-based method in BIFM, including linear frequency modulation (LFM) (up-chirp) (a), LFM (down-chirp) (b), nonlinear frequency modulation (NLFM) (c), binary a frequency-shift keying (BFSK) (d), and Costas frequency modulation (e) [45]@Copyright 2019 The Optical Society.

  • Figure 6

    (Color online) The architecture of an IPS with OANN-facilitated AI.

  • Figure 7

    (Color online) (a) Operation process of a two-layer OANN. (b) Feedback loop introduced in the experiment. protectłinebreak (c) The architecture of MZI-based OANN, which is tunable by the accompanied phase shifters as shown in (d) [50]@Copyright 2017 Springer Nature.

  • Figure 8

    (Color online) (a) Schematic of the diffraction-based OANN with multiple diffractive layers. The OANN implemented in experiments as a classifier (b) and an imager (c) [51]@Copyright 2018 The AAAS.

  • Figure 9

    (Color online) (a) Multi-layer schematic of the neural network; (b) the architecture of a single-layer OANN with coherent detection [52]@Copyright 2019 American Physical Society.

  • Figure 10

    (Color online) (a) A convolution using DEAP; (b) two convolutional units to perform a convolution [53]@Copyright 2020 IEEE.

  • Figure 11

    (Color online) (a) The optical convolution unit architecture; (b) the transmission rate versus the modulation voltage of the used modulators; (c) an illustration of the serialization method [60]@Copyright 2019 The Optical Society.

  • Figure 12

    (Color online) (a) Schematic of the CNN implementation with optical delay lines and the WDM; (b) detailed structure of the vector-matrix multiplication core [61].

  • Figure 13

    (Color online) The architecture of an IPS with OBNN-facilitated AI.

  • Figure 14

    (Color online) Simulation results of the laser with an SA featuring excitability. An optical pulse is released when the input perturbations accumulate to a threshold [63]@Copyright 2013 IEEE.

  • Figure 15

    (Color online) (a) The schematic of the photonic neuron with PCM (top) and the TE mode distribution (bottom); (b) the photonic synapse resembles the function of the biology synapse; (c) experimental setup of STDP realization by the photonic synapse [69]@Copyright 2017 The AAAS.

  • Figure 16

    (Color online) Broadcast-and-weight scheme, including a laser array, the WDM, spectral filters, and the waveguide loop [70]@Copyright 2014 IEEE.

  • Figure 17

    (Color online) (a) and (b) Schematic of the PCM-based spiking neural network; (c) the diagram of an integrated optical neuron; (d) optical micrograph of three optical neurons with four input ports respectively [71]@Copyright 2019 Springer Nature.

  • Figure 18

    (Color online) The experimental results of the sound azimuth measurement with DFB-LDs. The outputs when the delay is 3 $\mu$s (a) and 4 $\mu$s (b). The 2nd peak difference dependent on the delay (c) and the sound azimuth (d).

  • Figure 19

    (Color online) The DFB-LD-based spatiotemporal pattern recognition network with STDP learning module [72]@Copyright 2020 The Authors.

  • Table 1  

    Table 1Progress in large-scale hybrid integration

    Hybrid typeReferenceHighlights
    Active/Passive[77]Passive building blocks in the generic integration technology
    [78]First one-layer active-passive Al$_2$O$_{3}$ photonic integration
    Heterogenous[79]Systematical reviews on III-V/Silicon integration
    [80]Heterogeneous 2D/3D photonic integration
    Digital/Analog[81]Discussion on digital photonic integrated circuits
    [6]70 million transistors and 850 photonic components on a chip
    Photonic/Electronic[82]A way to integrate photonics with state-of-the-art nanoelectronics
    [83]Terahertz integrated hybrid electronic–photonic systems