logo

SCIENTIA SINICA Informationis, Volume 50 , Issue 9 : 1281(2020) https://doi.org/10.1360/SSI-2020-0204

Toward the third generation of artificial intelligence

More info
  • ReceivedJul 6, 2020
  • AcceptedAug 12, 2020
  • PublishedSep 22, 2020

Abstract


Funded by

国家自然科学基金重点国际合作项目(61620106010)


References

[1] Simon H A. Models of Man. New York: Wiley & Sons, 1957. Google Scholar

[2] Newell A, Simon H A. Computer science as empirical inquiry: symbols and search. Commun ACM, 1976, 19: 113-126 CrossRef Google Scholar

[3] Newell A. Physical Symbol Systems*. Cognitive Sci, 1980, 4: 135-183 CrossRef Google Scholar

[4] Fodor J A. Methodological solipsism considered as a research strategy in cognitive psychology. Behav Brain Sci, 1980, 3: 63-73 CrossRef Google Scholar

[5] McCarthy J, Minsky M L, Rochester N, et al. A proposal for the Dartmouth summer research project on artificial intelligence. 1955, 27: 12 https://doi.org/10.1609/aimag.v27i4.1904. Google Scholar

[6] Lindsay R K, Buchanan B G, Feigenbaum E A, et al. Applications of Artificial Intelligence for Organic Chemistry: The Dendral Project. New York: McGraw-Hill Book Company, 1980. Google Scholar

[7] Buchanan B G, Shortliffe E H. Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project. Boston: Addison Wesley, 1984. Google Scholar

[8] Muggleton S, de Raedt L. Inductive Logic Programming: Theory and methods. J Logic Programming, 1994, 19-20: 629-679 CrossRef Google Scholar

[9] Fabrizio R, Elena B, Riccardo Z, et al. A history of probabilistic inductive logic programming. Frontiers in Robotics and AI, 2014, 1: 1-6. Google Scholar

[10] Yang, Qiang, et al. Transfer Learning. Cambridge: Cambridge Press, 2020. Google Scholar

[11] Ehrlinger L, Wöß W. Towards a definition of knowledge graphs. In: Proceedings of Joint Proceedings of the Posters and Demos Track of 12th International Conference on Semantic Systems — SEMANTiCS2016 and 1st International Workshop on Semantic Change & Evolving Semantics (SuCCESS16), At Leipzig, 2016. 48: 1--4. Google Scholar

[12] Russel S J, Norvig P. Artificial Intelligence-A modern Approach. 2nd ed. Upper Saddle River: Prentice Hall, Inc., 2003. Google Scholar

[13] Rosenblatt F. The perceptron: A probabilistic model for information storage and organization in the brain.. Psychological Rev, 1958, 65: 386-408 CrossRef Google Scholar

[14] Rosenblatt F. Principles of Neurodynamics. Arch Gen Psychiatry. 1962, 7(3): 218-219. Google Scholar

[15] McCulloch W S, Pitts W. A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys, 1943, 5: 115-133 CrossRef Google Scholar

[16] Hebb D O. The Organization of Behavior-A Neuropsychological Theory. Abingdon: Psychology Press, 1949. Google Scholar

[17] Minsky M, Papert S A. Perceptrons: An Introduction to Computational Geometry. Cambridge: The MIT Press, 1969. Google Scholar

[18] Cauchy A. Methode generale pour la resolution des systemes d'equations simultanees. Comptes Rendus Hebd Seances Acad Sci, 1847, 25: 536--538. Google Scholar

[19] Nesterov Y E. A Method for Solving the Convex Programming Problem with Convergence Rate O(1/k2). Soviet Mathematics Doklady, 1983, 27: 372-376. Google Scholar

[20] Rumelhart D E, Hinton G E, Williams R J. Learning representations by back-propagating errors. Nature, 1986, 323: 533-536 CrossRef ADS Google Scholar

[21] Janocha K, Czarnecki W M. On Loss Functions for Deep Neural Networks in Classification. Schedae Informaticae, 2016, 25: 49-59. Google Scholar

[22] Wan L, Zeiler M, Zhang S X, et al. Regularization of neural networks using dropconnect. In: Proceedings of the 30th International Conference on Machine Learning (ICML), Atlanta, 2013. 1058--1066. Google Scholar

[23] Fukushima K. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybernetics, 1980, 36: 193-202 CrossRef Google Scholar

[24] Deep convolutional networks for quality assessment of protein folds. Bioinformatics, 2018, 34: 4046-4053 CrossRef Google Scholar

[25] Lipton J C, Berkowitz J. A Critical Review of Recurrent Neural Networks for Sequence Learning. 2015,. arXiv Google Scholar

[26] Hochreiter S, Schmidhuber J. Long Short-Term Memory. Neural Computation, 1997, 9: 1735-1780 CrossRef Google Scholar

[27] Hinton G E, Osindero S, Teh Y W. A Fast Learning Algorithm for Deep Belief Nets. Neural Computation, 2006, 18: 1527-1554 CrossRef Google Scholar

[28] Lecun Y, Bengio Y, and Hinton G E. Deep Learning. Nature, 2015, 521: 436-444. Google Scholar

[29] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, 2016. 770--778. Google Scholar

[30] Coulom R. Efficient selectivity and backup operators in Monte-Carlo tree search. In: Proceedings of the International Conference on Computers and Games, Berlin, 2006. 72--83. Google Scholar

[31] Sutton R S, Barto A G. Reinforcement Learning: An Introduction. Cambridge: The MIT Press, 2015. Google Scholar

[32] Dong Y P, Liao F Z, Pang T Y, et al. Boosting adversarial attacks with momentum. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, 2018. 9185--9193. Google Scholar

[33] Dong Y P, Su H, Zhu J, et al. Towards interpretable deep neural networks by leveraging adversarial examples. In: Proceedings of the IJCAI workshop on AISC, Sydney, 2019. 1--6. Google Scholar

[34] Ferrucci D, Brown E, Chu-Carroll J. Building Watson: An Overview of the DeepQA Project. AIMag, 2010, 31: 59-79 CrossRef Google Scholar

[35] Harnad S. The symbol grounding problem. Physica D-NOnlinear Phenomena, 1990, 42: 335-346 CrossRef Google Scholar

[36] Chen X, Duan Y, Houthooft R, et al. InfoGAN, interpretable representation learning by information maximizing generative adversarial nets. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Barcelona SPAIN, 2016. 2172--2180. Google Scholar

[37] Liu Y, Wei F Y, Shao J, et al. Exploring disentangled feature representation beyond face identificatio. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, 2018. 2080--2089. Google Scholar

[38] Higgins I, Matthey L, Pal A, et al. Beta-VAE: learning basic visual concepts with a constrained variational framework. In: Proceedings of the 32nd International Conference on Logic Programming (ICLP), New York City, 2016. 1--12. Google Scholar

[39] Siddharth N, Brooks P, Alban D, et al. Inducing interpretable representations with variational autoencoders. 2016,. arXiv Google Scholar

[40] Li C, Xu K, Zhu J, et al. Triple generative adversarial nets. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Long Beach, 2017. 4088--4098. Google Scholar

[41] Sparsity-Regularized HMAX for Visual Recognition. PLoS ONE, 2014, 9: e81813 CrossRef ADS Google Scholar

[42] Mnih V, Kavukcuoglu K, Silver D. Human-level control through deep reinforcement learning. Nature, 2015, 518: 529-533 CrossRef ADS Google Scholar

[43] Vinyals O, Babuschkin I, Czarnecki W M. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 2019, 575: 350-354 CrossRef ADS Google Scholar

[44] Silver D, Huang A, Maddison C J. Mastering the game of Go with deep neural networks and tree search. Nature, 2016, 529: 484-489 CrossRef ADS Google Scholar

[45] Silver D, Schrittwieser J, Simonyan K. Mastering the game of Go without human knowledge. Nature, 2017, 550: 354-359 CrossRef ADS Google Scholar

[46] Duan Y, Chen X, Houthooft R, et al. Benchmarking deep reinforcement learning for continuous control. In: Proceedings of the International Conference on Machine Learning (ICML), New York, 2016. 1329--1338. Google Scholar

[47] Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms. 2017,. arXiv Google Scholar

[48] Wiering, Marco, and Martijn Van Otterlo. Reinforcement learning. Springer, 2012, 12. Google Scholar

[49] Fran?ois-Lavet V, et al. An Introduction to Deep Reinforcement Learning. Foundations and Trends in Machine Learning, 2018, 11: 3-4. Google Scholar

[50] Huang S Y, Su H, Zhu J, et al. Combo-action: training agent for FPS game with auxiliary tasks. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI), Honolulu, 2019. 954--961. Google Scholar

[51] Song S H, Weng J Y, Su H, et al. Playing FPS games with environment-aware hierarchical reinforcement learning. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI), Macau, 2019. 3475--3482. Google Scholar

[52] Zhou Y C, Li J L, Zhu J. Posterior sampling for multi-agent reinforcement learning: solving extensive games with imperfect information. In: Proceedings of the International Conference on Learning Representations (ICLR), Addis Ababa, 2020. Google Scholar

[53] Mikolov T, Chen K, Corrado G, et al. Efficient estimation of word representations in vector space. In: Proceedings of the International Conference on Learning Representations (ICLR), Scottsdale, 2013. 1--6. Google Scholar

[54] Pennington J, Socher R, Manning C D. Glove, global vectors for word representation. In: Proceedings of the Conference On Empirical Methods in Natural Language Processing (EMNLP), Doha, 2014. 1532--1543. Google Scholar

[55] Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Lake Tahoe, 2013. 3111--3119. Google Scholar

[56] Kusner M J, Sun Y, Kolkin N I, et al. From word embeddings to document distances. In: Proceedings of the 32nd International Conference on Machine Learning (ICML), Lille, 2015. 957--966. Google Scholar

[57] Zhang J, Liu Y, Luan H B, et al. Prior knowledge integration for neural machine translation using posterior regularization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), 2017. 1514--1523. Google Scholar

[58] Ding Y Z, Liu Y, Luan H B, et al. Visualizing and understanding neural machine translation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), 2017. 1150--1159. Google Scholar

[59] 张钹. 后深度学习时代的人工智能. 中国人工智能通讯学会通讯, 2017, 7: 3--5. Google Scholar

[60] Liu M, Shi J, Li Z, et al. Towards Better Analysis of Deep Convolutional Neural Networks. In: Proceedings of the IEEE Conference on Visual Analytics Science and Technology, 2016. 91--100. Google Scholar

[61] Wang Y L, Su H, Hu X L. Interpret neural networks by identifying critical data routing paths. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, 2018. 8906--8914. Google Scholar

[62] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Montreal, 2014. 1--6. Google Scholar

[63] Dong Y P, Su H, Zhu J, et al. Improving interpretability of deep neural networks with semantic information. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 2017. 4306--4314. Google Scholar

[64] Dong Y P, Fu Q-A, Yang X, et al. Benchmarking adversarial robustness on image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 2020. 321--331. Google Scholar

[65] Liao F Z, Liang M, Dong Y P, et al. Defense against adversarial attacks using high-level representation guided denoiser. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, 2018. 1778--1787. Google Scholar

[66] Pang T Y, Xu K, Du C, et al. Improving adversarial robustness via promoting ensemble diversity. In: Proceedings of the International Conference on Machine Learning (ICML), Long Beach, 2019. 4970--4979. Google Scholar

[67] Castelvecchi D. Can we open the black box of AI? Nature News, 2016, 7623: 20. Google Scholar

[68] Ghahramani Z. Probabilistic machine learning and artificial intelligence. Nature, 2015, 521: 452-459 CrossRef ADS Google Scholar

[69] Zhu J, Chen J, Hu W, et al. Big Learning with Bayesian Methods, National Science Review, 2017, 4: 627-651. Google Scholar

[70] Dempster A, Laird N, Rubin D. Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B (Methodological), 1977, 39: 1-38. Google Scholar

[71] Zhu J, Chen N, Xing E P. Bayesian Inference with Posterior Regularization and applications to Infinite Latent SVMs. Journal of Machine Learning Research (JMLR), 2014, 15: 1799-1847. Google Scholar

[72] Williams P M. Bayesian Conditionalisation and the Principle of Minimum Information. Br J Philos Sci, 1980, 31: 131-144 CrossRef Google Scholar

[73] Mei S, Zhu J, Zhu X. Robust RegBayes: Selectively Incorporating First-Order Logic Domain Knowledge into Bayesian Models. In: Proceedings of the International Conference on Machine Learning (ICML), Beijing, 2014. 253--261. Google Scholar

[74] Zhu J, Ahmed A, Xing E P. MedLDA: maximum margin supervised topic models. Journal of Machine Learning Research (JMLR), 2012, 13: 2237--2278. Google Scholar

[75] Mackay D, Bridle J, Cheeseman P C. Bayesian methods for adaptive models. Dissertation for Ph.D. Degree. Pasadena: California Institute of Technology, 1992. Google Scholar

[76] Neal R M, Hinton G. Bayesian learning for neural networksm. Dissertation for Ph.D. Degree. Toronto: University of Toronto, 1995. Google Scholar

[77] Shi J, Sun S, Zhu J. A spectral approach to gradient estimation for implicit distributions. In: Proceedings of the 35th International Conference on Machine Learning (ICML), Stockholm, 2018. 4644--4653. Google Scholar

[78] Zhou Y, Shi J, Zhu J. Nonparametric score estimators. In: Proceedings of the International Conference on Machine Learning (ICML), Vienna, 2020. Google Scholar

[79] Wang Z, Ren T, Zhu J, et al. Function Space Particle Optimization for Bayesian Neural Networks. In: Proceedings of the International Conference on Learning Representations (ICLR), New Orleans, 2019. 1--12. Google Scholar

[80] Xu K, Li C, Zhu J, et al. Understanding and stabilizing GANs' training dynamics using control theory. In: Proceedings of the International Conference on Machine Learning (ICML), Vienna, 2020. Google Scholar

[81] Chen J, Lu C, Chenli B, et al. VFlow: more expressive generative flows with variational data augmentation. In: Proceedings of the International Conference on Machine Learning (ICML), Vienna, 2020. Google Scholar

[82] Shi J, Chen J, Zhu J, et al. ZhuSuan: a library for bayesian deep learning. 2017,. arXiv Google Scholar

[83] Ren M, Kiros R, Zemel R S. Exploring models and data for image question answering. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Montréal, 2015. 2953--2961. Google Scholar

[84] Zhu Y, Groth O, Bernstein M S, et al. Visual7w: grounded question answering in images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016. 4995--5004. Google Scholar

[85] Li J Z, Su H, Zhu J, et al. Essay-anchor attentive multi-modal bilinear pooling for textbook question answering. In: Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), San Diego, 2018. 1--6. Google Scholar