logo

SCIENCE CHINA Information Sciences, Volume 64 , Issue 7 : 172207(2021) https://doi.org/10.1007/s11432-020-3071-8

Trajectory prediction of cyclist based on dynamic Bayesian network and long short-term memory model at unsignalized intersections

More info
  • ReceivedApr 18, 2020
  • AcceptedSep 1, 2020
  • PublishedMay 18, 2021

Abstract


Acknowledgment

This work was supported in part by National Natural Science Foundation of China (Grant Nos. U1804161, U2013601, U20A20225), Key Research and Development Plan of Anhui Province (Grant No. 202004a05020058), Fundamental Research Funds for the Central Universities, Science and Technology Innovation Planning Project of Ministry of Education of China, NVIDIA NVAIL program, and Key Laboratory of Advanced Perception and Intelligent Control of High-end Equipment of Ministry of Education (Anhui Polytechnic University, Wuhu, China, 241000) (Grant No. GDSC202007). And experiments are conducted on NVIDIA DGX-2.


References

[1] The Annual Statistical Report for Road Traffic Accidents in China (2017). The Ministry of Public Security Traffic Management Bureau. 2018. Google Scholar

[2] Endsley M R. Toward a Theory of Situation Awareness in Dynamic Systems. Hum Factors, 1995, 37: 32-64 CrossRef Google Scholar

[3] Duan J, Li R, Hou L. Driver braking behavior analysis to improve autonomous emergency braking systems in typical Chinese vehicle-bicycle conflicts. Accident Anal Prevention, 2017, 108: 74-82 CrossRef Google Scholar

[4] Li D, Gao H. A Hardware Platform Framework for an Intelligent Vehicle Based on a Driving Brain. Engineering, 2018, 4: 464-470 CrossRef Google Scholar

[5] Ma X, Luo D. Modeling cyclist acceleration process for bicycle traffic simulation using naturalistic data. Transpation Res Part F-Traffic Psychology Behaviour, 2016, 40: 130-144 CrossRef Google Scholar

[6] Wu X, Li Z, Kan Z. Reference Trajectory Reshaping Optimization and Control of Robotic Exoskeletons for Human-Robot Co-Manipulation. IEEE Trans Cybern, 2020, 50: 3740-3751 CrossRef Google Scholar

[7] Goldhammer M, Kohler S, Doll K, Sick B. Camera based pedestrian path prediction by means of polynomial least-squares approximation and multilayer perceptron neural networks. In: Proceedings of 2015 SAI Intelligent Systems Conference (IntelliSys), London, 2015. 390--399. Google Scholar

[8] Gao H, Luo L, Pi M. EEG-Based Volitional Control of Prosthetic Legs for Walking in Different Terrains. IEEE Trans Automat Sci Eng, 2020, : 1-11 CrossRef Google Scholar

[9] Bieshaar M, Zernetsch S, Depping M, Sick B, Doll K. Cooperative starting intention detection of cyclists based on smart devices and infrastructure. In: Proceedings of 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, 2017. 1--8. Google Scholar

[10] Alahi A, Goel K, Ramanathan V, et al. Social LSTM: human trajectory prediction in crowded spaces. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016. 961--971. Google Scholar

[11] Xue H, Huynh D Q, Reynolds M. SS-LSTM: a hierarchical LSTM model for pedestrian trajectory prediction. In: Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, 2018. 1186--1194. Google Scholar

[12] Khaled S, Hossny M, Nahavandi S. Cyclist trajectory prediction using bidirectional recurrent neural networks. In: Proceedings of the 31st Australasian Joint Conference on Advances in Artificial Intelligence, Wellington, 2018. 11--14. Google Scholar

[13] PoolE A I, Kooij J F P, Gavrila D M. Using road topology to improve cyclist path prediction. In: Proceedings of 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, 2017. 289--296. Google Scholar

[14] Kooij J F P, Schneider N, Gavrila D M. Analysis of pedestrian dynamics from a vehicle perspective. In: Proceedings of 2014 IEEE Intelligent Vehicles Symposium Proceedings, 2014. 1445--1450. Google Scholar

[15] Koehler S, Goldhammer M, Bauer S. Stationary Detection of the Pedestrian?s Intention at Intersections. IEEE Intell Transp Syst Mag, 2013, 5: 87-99 CrossRef Google Scholar

[16] Bieshaar M, Zernetsch S, Depping M, Sick B, Doll K. Cooperative starting intention detection of cyclists based on smart devices and infrastructure. In: Proceedings of 2017 IEEE 20th International Conference on Intelligent Transportation Systems, Yokohama, 2017. 1--8. Google Scholar

[17] Xie G, Gao H, Qian L. Vehicle Trajectory Prediction by Integrating Physics- and Maneuver-Based Approaches Using Interactive Multiple Models. IEEE Trans Ind Electron, 2018, 65: 5999-6008 CrossRef Google Scholar

[18] Gao H, Cheng B, Wang J. Object Classification Using CNN-Based Fusion of Vision and LIDAR in Autonomous Vehicle Environment. IEEE Trans Ind Inf, 2018, 14: 4224-4231 CrossRef Google Scholar

[19] Kooij J F P, Schneider N, D. Gavrila M. Analysis of pedestrian dynamics from a vehicle perspective. In: Proceedings of 2014 IEEE Intelligent Vehicles Symposium Proceedings, Dearborn, 2014. 1445--1450. Google Scholar

[20] Zhang X, Gao H, Li G. Multi-view clustering based on graph-regularized nonnegative matrix factorization for object recognition. Inf Sci, 2018, 432: 463-478 CrossRef Google Scholar

[21] Li Z, Huang B, Ye Z. Physical Human-Robot Interaction of a Robotic Exoskeleton By Admittance Control. IEEE Trans Ind Electron, 2018, 65: 9614-9624 CrossRef Google Scholar

[22] Kooij J F P, Flohr F, Pool E A I. Context-Based Path Prediction for Targets with Switching Dynamics. Int J Comput Vis, 2019, 127: 239-262 CrossRef Google Scholar

[23] Gao H, Zhu J, Zhang T, et al. Situational assessment for intelligent vehicles based on stochastic model and Gaussian distributions in typical traffic scenarios. IEEE Trans Syst Man Cy-S, 2020. doi: 10.1109/TSMC.2020.3019512. Google Scholar

[24] Wakim C F, Capperon S, Oksman J. A Markovian model of pedestrian behavior. In: Proceedings of 2004 IEEE International Conference on Systems, Man and Cybernetics, The Hague, 2004. 4028--4033. Google Scholar

[25] Levinson J, Thrun S. Robust vehicle localization in urban environments using probabilistic maps. In: Proceedings of 2010 IEEE International Conference on Robotics and Automation, Anchorage, 2010. 4372--4378. Google Scholar

[26] Koehler S, Goldhammer M, Bauer S. Stationary Detection of the Pedestrian?s Intention at Intersections. IEEE Intell Transp Syst Mag, 2013, 5: 87-99 CrossRef Google Scholar

[27] Huang L, Wu J. Study on the cyclist behavior at signalized intersections. In: Proceedings of the 2003 IEEE International Conference on Intelligent Transportation Systems, Shanghai, 2003. 317--322. Google Scholar

[28] Ling H, Wu J. A Study on Cyclist Behavior at Signalized Intersections. IEEE Trans Intell Transp Syst, 2004, 5: 293-299 CrossRef Google Scholar

  • Figure 1

    (Color online) Block diagram of the proposed method.

  • Figure 2

    (Color online) The DBN-based model for cyclist intention inference from time step $t-1$ to $t$. The observed variables (circle) correspond to the hidden variables. Intentions (shaded rectangle) are inferred via given hidden variables.

  • Figure 3

    (Color online) Inferring process steps of DBN.

  • Figure 4

    (Color online) Framework of trajectory prediction.

  • Figure 5

    (Color online) Scenarios for data collection.

  • Figure 6

    (Color online) Comparison of intention inference methods. (a) ACCR; (b) PRE; (c) RECR.

  • Figure 7

    (Color online) The probability of cyclist intentions changes over TTE in three cases. (a) Going straight; (b) turning left; (c) turning right.

  • Figure 8

    (Color online) Error comparison of trajectory prediction methods. (a) 1.0 s-AEE; (b) 1.5 s-AEE; (c) 2.5 s-AEE.

  • Figure 9

    (Color online) Prediction error over TTE in three cases for prediction time horizon 1.0 s. (a) Going straight; (b) turning left; (c) turning right.

  • Table 1  

    Table 1Number of frames in the cyclist dataset

    Longitudinal From left From right Oncoming Total
    Going straight 750 750 1126 657 3283
    Turning left 735 571 898 490 2694
    Turning right 425 372 372 212 1381
    Total 1910 1693 2396 1359 7358
  •   

    Algorithm 1 Intention inference model

    $\textbf{Initialize:}~P(H_t^O|H_{t-1}^O),~P(H_t^V|H_{t-1}^V~),~P(H_t^R|H_{t-1}^R),~P(O^O|H^O),~P(O^V~|H^V),~P(O^R~|H^R),~P({\rm~Int}_t|{\rm~Int}_{t-1},H_t)$;

    $\textbf{for}~~\mbox{episode}~=~1,~M~~\textbf{do}$

    $~~\mbox{Initialize}~~~\hat{P}({\rm~Int}_{t-1},H_{t-1});$

    $~\mbox{Obtain~the~temporal~transitions~for~hidden~variables~with~time~step:}~~~P(H_t~|H_{t-1});$

    $~\mbox{Obtain~the~relationship~between~the~observed~variables~and~the~hidden~variables:}~~~P(O_t~|H_t);$

    $~\mbox{Obtain~joint~prior~distribution~of}~~{\rm~Int}~~~\mbox{and}~~~H~~~\mbox{at~time~step}~~t~~\mbox{and}~~t-1:~~\overline{P}~({\rm~Int}_t,{\rm~Int}_{t-1},H_t,H_{t-1})$;

    $~\mbox{Obtain~joint~prior~distribution~of}~~{\rm~Int}~~\mbox{and}~~H~~\mbox{at~time~step}~~t:~~\overline{P}({\rm~Int}_t,H)$;

    $~\mbox{Obtain~and~output~the~posterior~probability~of~crossing~function}~~\hat{P}({\rm~Int}_t);$

    $~\textbf{end~for}$

  • Table 2  

    Table 2Confusion matrix definition

    Inference Reality
    Yes No
    Yes True positive (TP) False positive (FP)
    No True negative (TN) False negative (FN)
  •   

    Algorithm 2 Trajectory prediction model

    $\textbf{Initialize:}~\mbox{Encoder~network}~~Q_e(X_t,~w),~\mbox{and~decoder~network}~Q_d(C_t,~{\rm~BOS},~v)~(w~~\mbox{and}~v~~\mbox{are~the~network~parameters})$;

    $\textbf{for}~~\mbox{episode}~=~1,~M~~\textbf{do}$

    $~~\mbox{Initialize~encoder~input}~~X_t;$

    $~~\mbox{Initialize~starting~condition~BOS};$

    $~~\textbf{for}~~{t=h,~H}~~\textbf{do}$

    $~~~~\mbox{Obtain~the~output~of~encoder}~~C_t;$

    $~~~~\mbox{Obtain~the~output~of~decoder}~~Y_{t+1};$

    $~~~~\mbox{Minimize~the~loss~function~to~update~networks:}~~~{\rm~MSE}=\frac{\sum_{i=1}^n(Y_i-Y_{ip})^2}{n};$

    $~~~\textbf{end~for}$

    $~\textbf{end~for}$

qqqq

Contact and support