logo

SCIENCE CHINA Information Sciences, Volume 63 , Issue 11 : 212207(2020) https://doi.org/10.1007/s11432-019-2761-y

Motion trajectory prediction based on a CNN-LSTM sequential model

More info
  • ReceivedSep 11, 2019
  • AcceptedNov 29, 2019
  • PublishedOct 15, 2020

Abstract


Acknowledgment

This work was supported by National Key RD Program of China (Grant No. 2018YFB1201500), National Natural Science Foundation of China (Grant Nos. 61873201, 61773313, U1734210), Key Research and Development Program of Shaanxi Province (Grant No. 2018GY-139), Natural Science Foundation of Shaanxi Provincial Department of Education (Grant No. 19JS051), CERNET Innovation Project (Grant No. NGII20161201), and Scientific and Technological Planning Project of Beilin District of Xi'an (Grant No. GX1819).


References

[1] Cao Y, Ma L, Zhang Y. Application of fuzzy predictive control technology in automatic train operation. Cluster Comput, 2019, 22: 14135-14144 CrossRef Google Scholar

[2] Xie G, Peng X, Li X. Remaining useful life prediction of lithium?ıon battery based on an improved particle filter algorithm. Can J Chem Eng, 2019, 41: cjce.23675 CrossRef Google Scholar

[3] Xie G, Li X, Peng X. Estimating the Probability Density Function of Remaining Useful Life for Wiener Degradation Process with Uncertain Parameters. Int J Control Autom Syst, 2019, 17: 2734-2745 CrossRef Google Scholar

[4] Yu W, Zhao C. Online Fault Diagnosis for Industrial Processes With Bayesian Network-Based Probabilistic Ensemble Learning Strategy. IEEE Trans Automat Sci Eng, 2019, 16: 1922-1932 CrossRef Google Scholar

[5] Chai Z, Zhao C. A Fine-Grained Adversarial Network Method for Cross-Domain Industrial Fault Diagnosis. IEEE Trans Automat Sci Eng, 2020, : 1-11 CrossRef Google Scholar

[6] Xie G, Jin Y Z, Hei X H, et al. Adaptive Identification of Time-varying Environmental Parameters in Train Dynamics Model. Acta Automatica Sinica, 2020. doi: 10.16383/j.aas.c190215. Google Scholar

[7] Cao Y, Zhang Y, Wen T. Research on dynamic nonlinear input prediction of fault diagnosis based on fractional differential operator equation in high-speed train control system. Chaos, 2019, 29: 013130 CrossRef PubMed ADS Google Scholar

[8] He W, Zhang S. Control Design for Nonlinear Flexible Wings of a Robotic Aircraft. IEEE Trans Contr Syst Technol, 2017, 25: 351-357 CrossRef Google Scholar

[9] Qiao S J, Jin K, Han N, et al. Trajectory prediction algorithm based on gaussian mixture model. J Softw, 2015, 26: 1048--1063. Google Scholar

[10] Qiao S, Shen D, Wang X. A Self-Adaptive Parameter Selection Trajectory Prediction Approach via Hidden Markov Models. IEEE Trans Intell Transp Syst, 2015, 16: 284-296 CrossRef Google Scholar

[11] Wu P J, Yang W T, Yu C, et al. Trajectory prediction method for high precision servo control system (in Chinese). Electric Mach Control, 2014, 18: 1--5. Google Scholar

[12] Houenou A, Bonnifait P, Cherfaoui V, et al. Vehicle trajectory prediction based on motion model and maneuver recognition. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, 2013. 4363--4369. Google Scholar

[13] A motion simulation model for road network based crowdsourced map datum. IFS, 2020, 38: 391-407 CrossRef Google Scholar

[14] Xie G, Sun L, Wen T. Adaptive Transition Probability Matrix-Based Parallel IMM Algorithm. IEEE Trans Syst Man Cybern Syst, 2019, : 1-10 CrossRef Google Scholar

[15] Lecun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521: 436. Google Scholar

[16] Li D, Liu M, Zhao F. Challenges and countermeasures of interaction in autonomous vehicles. Sci China Inf Sci, 2019, 62: 50201 CrossRef Google Scholar

[17] Deo N, Trivedi M M. Multi-modal trajectory prediction of surrounding vehicles with maneuver based LSTMs. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), Changshu, 2018. Google Scholar

[18] Park S H, Kim B D, Kang C M, et al. Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), Changshu, 2018. Google Scholar

[19] Altché F, Arnaud D L F. An LSTM network for highway trajectory prediction. In: Proceedings of IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017. 353--359. Google Scholar

[20] Zhang P, Yang T, Liu Y N, et al. QAR data feature extraction and prediction based on CNN-LSTM (in Chinese). Appl Res Comput, 2019, 36:. Google Scholar

[21] Kim B D, Kang C M, Lee S H, et al. Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network. In: Proceedings of IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017. 399--404. Google Scholar

[22] Yang J, Xie G, Yang Y. Deep model integrated with data correlation analysis for multiple intermittent faults diagnosis.. ISA Trans, 2019, 95: 306-319 CrossRef PubMed Google Scholar

[23] Yang J, Xie G, Yang Y. An improved deep network for intelligent diagnosis of machinery faults with similar features. IEEJ Trans Elec Electron Eng, 2019, 14: 1851-1864 CrossRef Google Scholar

[24] Cao Y, Sun Y, Xie G. Fault Diagnosis of Train Plug Door Based on a Hybrid Criterion for IMFs Selection and Fractional Wavelet Package Energy Entropy. IEEE Trans Veh Technol, 2019, 68: 7544-7551 CrossRef Google Scholar

[25] Zhang S, Dong Y, Ouyang Y. Adaptive Neural Control for Robotic Manipulators With Output Constraints and Uncertainties.. IEEE Trans Neural Netw Learning Syst, 2018, 29: 5554-5564 CrossRef PubMed Google Scholar

[26] He W, Dong Y. Adaptive Fuzzy Neural Network Control for a Constrained Robot Using Impedance Learning.. IEEE Trans Neural Netw Learning Syst, 2018, 29: 1174-1186 CrossRef PubMed Google Scholar

[27] Xue Z, Liu J, Wu Z. Development and path planning of a novel unmanned surface vehicle system and its application to exploitation of Qarhan Salt Lake. Sci China Inf Sci, 2019, 62: 084202 CrossRef Google Scholar

[28] Thiemann C, Treiber M, Kesting A. Estimating acceleration and lane-changing dynamics based on NGSIM trajectory Data. Transport Res Record J Transport Res Board, 2008, 2088: 90--101. Google Scholar

[29] Li P, Dargaville R, Cao Y. Storage Aided System Property Enhancing and Hybrid Robust Smoothing for Large-Scale PV Systems. IEEE Trans Smart Grid, 2017, 8: 2871-2879 CrossRef Google Scholar

[30] Hinton G E, Osindero S, Teh Y W. A fast learning algorithm for deep belief nets.. Neural Computation, 2006, 18: 1527-1554 CrossRef PubMed Google Scholar

[31] Lecun Y, Bottou L, Bengio Y. Gradient-based learning applied to document recognition. Proc IEEE, 1998, 86: 2278-2324 CrossRef Google Scholar

[32] Chan T A, Hermeking H, Lengauer C. 14-3-3Sigma is required to prevent mitotic catastrophe after DNA damage.. Nature, 1999, 401: 616-620 CrossRef PubMed ADS Google Scholar

[33] Gers F A, Schmidhuber J, Cummins F. Learning to forget: continual prediction with LSTM.. Neural Computation, 2000, 12: 2451-2471 CrossRef PubMed Google Scholar

[34] Hochreiter S, Schmidhuber J. Long short-term memory.. Neural Computation, 1997, 9: 1735-1780 CrossRef PubMed Google Scholar

[35] Wang Y, Liu J, Misic J. Assessing Optimizer Impact on DNN Model Sensitivity to Adversarial Examples. IEEE Access, 2019, 7: 152766-152776 CrossRef Google Scholar

  • Figure 1

    (Color online) A driving scenario of a self-driving vehicle.

  • Figure 2

    (Color online) The framework of vehicle trajectory prediction.

  • Figure 5

    (Color online) 1D convolution for time series data.

  • Figure 6

    (Color online) Pooling for time series data.

  • Figure 7

    (Color online) LSTM structure.

  • Figure 8

    (Color online) CNN-LSTM sequential model framework.

  • Figure 9

    (Color online) Loss value for different optimizer.

  • Figure 10

    (Color online) Loss value for different convolution kernels. (a) Training dataset; (b) test dataset.

  • Figure 11

    (Color online) Prediction error for different kernel size.

  • Figure 12

    (Color online) Loss value for different kernel size. (a) Training dataset; (b) test dataset.

  • Figure 13

    (Color online) Loss value for different number of LSTM neurons. (a) Training dataset; (b) test dataset.

  • Figure 14

    (Color online) Loss value for different number of LSTM layers. (a) Training dataset; (b) test dataset.

  • Figure 15

    (Color online) Loss value for different number of neurons in the FC layer. (a) Training dataset; (b) test dataset.

  • Figure 18

    (Color online) Network consumption time for different batch sizes.

  • Figure 19

    (Color online) Predicted lateral trajectories of surrounding vehicles. (a) V1; (b) V2; (c) V3; (d) V4; (e) V5; (f) V6; (g) V7; (h) V8.

  • Figure 20

    (Color online) Prediction relative error. (a) V1; (b) V2; (c) V3; (d) V4; (e) V5; (f) V6; (g) V7; (h) V8.

  • Figure 21

    (Color online) Result of outlier value detection via input data at annular crossing.

  • Figure 22

    (Color online) Predicted trajectories of surrounding vehicles. (a) V1; (b) V2; (c) V3; (d) V4.

  • Figure 23

    (Color online) Relative errors of predicted trajectories. (a) V1; (b) V2; (c) V3; (d) V4.

  • Table 1  

    Table 1Useful information in the NGSIM

    Name Description Unit
    $x$ Lateral offset Feet
    $y$ Longitudinal offset Feet
    $v$ Vehicle speed Feet/s
    $l$ Left lane distance Feet
    $r$ Right lane distance Feet
  • Table 2  

    Table 2Prediction error of different filters

    KERNEL 5 KERNEL 10 KERNEL 15 KERNEL 20
    RMSE 0.04731 0.02724 0.03417 0.03368
    MAE 0.03504 0.01555 0.02079 0.02034
    Deviation 0.00617 0.00268 0.00361 0.00353
  • Table 3  

    Table 3Prediction error for different number of LSTM neurons

    3 neurons 6 neurons 9 neurons 12 neurons
    RMSE 0.02923 0.02248 0.02656 0.04312
    MAE 0.01746 0.01188 0.01844 0.03181
    Deviation 0.00304 0.00205 0.00323 0.00556
  • Table 4  

    Table 4Prediction error for different number of LSTM layers

    One LSTM layer Two LSTM layers with 3 neurons Two LSTM layers with 6 neurons
    RMSE 0.03853 0.06607 0.35303
    MAE 0.02473 0.04929 0.28778
    Deviation 0.00427 0.00861 0.05022
  • Table 5  

    Table 5Prediction error for different number of neurons in the FC layer

    3 neurons 6 neurons 9 neurons 12 neurons
    RMSE 0.05429 0.03896 0.05138 0.04488
    MAE 0.04028 0.02492 0.03751 0.03257
    Deviation 0.00703 0.00431 0.00654 0.00568
  • Table 6  

    Table 6Vehicles in four lanes

    LANE 1 LANE 2 LANE 3 LANE 4
    Lane-keeping V1 V3 V5 V7
    Lane-changing V2 V4 V6 V8
  • Table 7  

    Table 7Various error indicators for trajectory prediction using different models

    CNN-LSTM LSTM GRU CNN
    V1RMSE 0.06933 0.14911 0.13252 0.09680
    MAE 0.02830 0.06519 0.07954 0.03618
    Deviation 0.01095 0.02487 0.03183 0.01357
    Time 4.8037 10.3154 6.4992 8.5562
    V2RMSE 0.12953 1.52990 1.67531 0.99138
    MAE 0.07426 0.94095 1.02386 0.42756
    Deviation 0.01936 0.22333 0.24207 0.07478
    Time 8.7319 13.3231 9.0863 11.5380
    V3RMSE 0.02734 0.08685 0.12410 0.03316
    MAE 0.01086 0.04670 0.06239 0.01496
    Deviation 0.00178 0.00752 0.00995 0.00238
    Time 5.0013 9.1223 5.2013 6.9874
    V4RMSE 0.04316 0.14060 0.16167 0.09295
    MAE 0.02214 0.10137 0.11464 0.05610
    Deviation 0.01442 0.06222 0.07053 0.03613
    Time 5.5436 8.1660 8.4666 6.9444
    V5RMSE 0.01382 0.04065 0.02898 0.02418
    MAE 0.010430.02655 0.01620 0.02050
    Deviation 0.00111 0.002810.00172 0.00219
    Time 6.2157 7.8658 6.2764 6.7486
    V6RMSE 0.20344 0.52590 0.51133 0.40998
    MAE 0.067060.27635 0.175530.37104
    Deviation 0.02041 0.070260.05287 0.07420
    Time 5.37007.1364 7.3454 6.1584
    V7RMSE 0.00690 0.08620 0.01849 0.03323
    MAE 0.005460.05337 0.015410.02315
    Deviation 0.00041 0.004230.00122 0.00191
    Time 5.93499.0136 7.7046 7.2841
    V8RMSE 0.05230 0.15493 0.10782 0.06727
    MAE 0.019210.13760 0.096770.05844
    Deviation 0.00213 0.014850.01040 0.00630
    Time 6.223910.2061 9.0357 8.7624
  • Table 8  

    Table 8Measured indicators of prediction results

    CNN-LSTM LSTM GRU CNN
    V1RMSE 147.5097 212.7747 161.5150210.6115
    MAE 131.4657 175.5494 145.9455 188.2853
    Deviation 0.1372 0.2318 0.1540 0.1723
    Time 6.6897 9.0634 8.7445 6.7868
    V2RMSE 36.8245 103.9171 108.3020 55.6265
    MAE 23.698874.672246.178335.7754
    Deviation 0.0236 0.0801 0.0377 0.0338
    Time 7.111410.40419.70679.8593
    V3RMSE 144.3097 172.3119 153.5478148.0862
    MAE 101.5878 144.9382 119.0696 116.0209
    Deviation 0.0663 0.1004 0.0772 0.0800
    Time 9.728810.19619.6121 10.0884
    V4RMSE 10.2299 12.9799 27.7174 196.76836
    MAE 7.9365 9.9685 24.9205 124.1512
    Deviation 0.0060 0.0069 0.0218 0.08541
    Time 9.513510.895912.1631 11.8550