logo

Journals Books
Advanced Search
Sign in Register

中文

SCIENCE CHINA Information Sciences
Download PDF
SCIENCE CHINA Information Sciences, Volume 63 , Issue 10 : 202301(2020) https://doi.org/10.1007/s11432-019-2734-y

Measuring quality of experience for 360-degree videos in virtual reality

null Muhammad Shahid ANWAR 1, Jing WANG 1,*, Asad ULLAH 1, Wahab KHAN 1, Sadique AHMAD 2, Zesong FEI 1
More info
  • ReceivedJul 18, 2019
  • AcceptedNov 15, 2019
  • PublishedAug 19, 2020
Previous Table of Contents Next
Share
Rating

Abstract


References

[1] Le Callet P, Möller S, Perkis A. Qualinet white paper on definitions of quality of experience (2012). European Network on Quality of Experience in Multimedia Systems and Services (COST Action IC 1003). version 1.2. 2013. Google Scholar

[2] Liu Y T, Yun S, Mao Y N, et al. A study on quality of experience for adaptive streaming service. In: Proceedings of IEEE International Conference on Communications Workshops (ICC), 2013. 682--686. Google Scholar

[3] Lewcio B, Belmudez B, Enghardt T, et al. On the way to high-quality video calls in future mobile networks. In: Proceedings of 2011 3rd International Workshop on Quality of Multimedia Experience (QoMEX), 2011. 43--48. Google Scholar

[4] Zare A, Aminlou A, Hannuksela M M, et al. HEVC-compliant tile-based streaming of panoramic video for virtual reality applications. In: Proceedings of Proceedings of the 24th ACM International Conference on Multimedia, 2016. 601--605. Google Scholar

[5] Qian F, Ji L, Han B, et al. Optimizing 360 video delivery over cellular networks. In: Proceedings of the 5th Workshop on All Things Cellular: Operations, Applications and Challenges, 2016. 1--6. Google Scholar

[6] Anwar M S, Wang J, Ullah A, Khan W, Li Z, and Ahmad S. User profile analysis for enhancing QoE of 360 panoramic video in virtual reality environment. In: Proceedings of 2018 International Conference on Virtual Reality and Visualization (ICVRV), 2018. 106--111. Google Scholar

[7] Zhang B, Zhao J, Yang S, et al. Subjective and objective quality assessment of panoramic videos in virtual reality environments. In: Proceedings of IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2017. 163--168. Google Scholar

[8] Li C, Xu M, Du X Z, et al. Bridge the gap between VQA and human behavior on omnidirectional video. 2018. 932--940. Google Scholar

[9] Schatz R, Sackl A, Timmerer C, et al. Towards subjective quality of experience assessment for omnidirectional video streaming. In: Proceedings of 2017 9th International Conference on Quality of Multimedia Experience (QoMEX), 2017. 1--6. Google Scholar

[10] Zou W, Yang F. Measuring quality of experience of novel 360-degree streaming video during stalling. In: Proceedings of International Conference on Communicatins and Networking in China, 2017. 417--424. Google Scholar

[11] Zhang W, Zou W, Yang F. The impact of stalling on the perceptual quality of http-based omnidirectional video streaming. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019. 4060--4064. Google Scholar

[12] Chao Chen , Lark Kwon Choi , de Veciana G. Modeling the time--varying subjective quality of HTTP video streams with rate adaptations.. IEEE Trans Image Process, 2014, 23: 2206-2221 CrossRef PubMed Google Scholar

[13] Singla A, Fremerey S, Robitza W, et al. Measuring and comparing qoe and simulator sickness of omnidirectional videos in different head mounted displays. In: Proceedings of 2017 9th International Conference on Quality of Multimedia Experience (QoMEX), 2017. 1--6. Google Scholar

[14] Huyen T, Ngoc N P, Pham C T, et al. A subjective study on QoE of 360 video for VR communication. In: Proceedings of 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP), 2017. 1--6. Google Scholar

[15] Tasaka S. Bayesian Hierarchical Regression Models for QoE Estimation and Prediction in Audiovisual Communications. IEEE Trans Multimedia, 2017, 19: 1195-1208 CrossRef Google Scholar

[16] Ahmad S, Li K, Amin A. A Multilayer Prediction Approach for the Student Cognitive Skills Measurement. IEEE Access, 2018, 6: 57470-57484 CrossRef Google Scholar

[17] Zhang Y, Wang Y, Liu F. Subjective Panoramic Video Quality Assessment Database for Coding Applications. IEEE Trans Broadcast, 2018, 64: 461-473 CrossRef Google Scholar

[18] Corbillon X, de Simone F, Simon G. 360-degree video head movement dataset. In: Proceedings of the 8th ACM on Multimedia Systems Conference, 2017. 199--204. Google Scholar

[19] Lo W-C, Fan C-L, Lee J, et al. 360 video viewing dataset in head-mounted virtual reality. In: Proceedings of the 8th ACM on Multimedia Systems Conference, 2017. 211--216. Google Scholar

[20] Wu C, Tan Z, Wang Z, et al. A dataset for exploring user behaviors in vr spherical video streaming. In: Proceedings of the 8th ACM on Multimedia Systems Conference, 2017. 193--198. Google Scholar

[21] Staelens N, Moens S, Van den Broeck W. Assessing Quality of Experience of IPTV and Video on Demand Services in Real-Life Environments. IEEE Trans Broadcast, 2010, 56: 458-466 CrossRef Google Scholar

[22] ITU-T P.910, subjective video quality assessment methods for multimedia applications. 2008. Google Scholar

[23] ITU-R BT, 500-12, recommendation: Methodology for the subjective assessment of the quality of television pictures. 1993. Google Scholar

[24] Yao S-H, Fan C-L, Hsu C-H. Towards quality-of-experience models for watching 360 videos in head-mounted virtual reality. In: Proceedings of 2019 11th International Conference on Quality of Multimedia Experience (QoMEX), 2019. 1--3. Google Scholar

  • Click to view Download high-quality image

    Figure 1

    (Color online) Source sequence thumbnails. (a) Idaho Boat; (b) Animation 1; (c) Science Fiction HELP; protectłinebreak (d) Gyrocopter; (e) Snow driving; (f) Animation 2;. (g) Skiing; (h) Military parade; (i) Beach volleyball; (j) Rio Olympics; (k) Cockpit view; (l) Undersea; (m) Roller coaster; (n) Animation 3; (o) Surrounded by Elephants; (p) Project Soul.

  • Click to view Download high-quality image

    Figure 2

    (Color online) PLCC and SRCC performance evaluation of individual subjects.

  • Click to view Download high-quality image

    Figure 3

    (Color online) MOS of all compressed videos in the absence of stalling.

  • Click to view Download high-quality image

    Figure 4

    (Color online) The posterior QoE with three different video bitrates in the absence of stalling (a), and in the presence of initial stalling (b), mid stalling (c), multiple stalling (d).

  • Click to view Download high-quality image

    Figure 5

    (Color online) Prediction of overall QoE.

  • Click to view Download high-quality image

    Figure 6

    (Color online) PLCC and SRCC performance evaluation of the average MOS and predicted values.

  • Table 1  

    Table 1The details of source 360-degree videos

    Index360-degree videos nameDescriptionFrame rate (fps)SI/TI
    (a) Idaho Dinghy BoatHuman, high motion3059.9/66.8
    (b) Animation 1Outdoor scene2448.3/38.4
    (c) Science Fiction HELPHuman, indoor3028.5/18.4
    (d) Skyhub Dubai, GyrocopterHuman, architecture 3042.9/14.7
    (e) Snow drivingHuman, nature2554.7/22.4
    (f) Animation 2Snake in the forest2449.1/0.9
    (g) SkiingHuman, high motion2554.1/1.5
    (h) Military paradeHuman, outdoor3050.1/16.1
    (i) Beach volleyballHuman, high motion2430.6/7.4
    (j) Rio OlympicsHuman, outdoor2456.8/15.5
    (k) Cockpit viewHuman, indoor2559.5/21.7
    (l) UnderseaHuman, nature3032.1/7.1
    (m) Roller coasterHuman, high motion3078.3/48.1
    (n) Animation 3Outdoor scene2464.7/56.2
    (o) Surrounded by ElephantsWild, nature3036.2/2.8
    (p) Project SoulHuman, indoor3027.5/11.9
  •   

    Algorithm 1 Bitrate posterior QoE estimation

    Require:sets QoE, bitrates.

    Output:sets ${\rm~BR}_m$ $(m=1,2,3)$. Initialization:

    Iterations bitrate:

    for each $i$ in bitrates

    for each $j$ in QoE

    Calculate posterior of $j$ with respect to $i$;

    Add posterior to ${\rm~BR}_m$;

    end for

    return ${\rm~BR}_m$;

    Increment ${\rm~BR}_m$ to ${\rm~BR}_{m+1}$;

    end for

  •   

    Algorithm 2 Stalling experience posterior QoE estimation under different bitrate levels

    Require:sets ${\rm~BR}_1$, ${\rm~BR}_2$, ${\rm~BR}_3$, Stalling.

    Output:sets ${\rm~ST}_p$ $(p=1,2,3)$, ${\rm~ST}_q$ $(q=1,2,3)$, and ${\rm~ST}_r$ $(r=1,2,3)$. Initialization:

    Iterations of Stalling:

    for each $a$ in Stalling

    for each $b$ in ${\rm~BR}_1$

    Calculate posterior of $a$ with respect to $b$;

    Add posterior to ${\rm~ST}_p$;

    end for

    return ${\rm~ST}_p$;

    Increment ${\rm~ST}_p$ to ${\rm~ST}_{p+1}$;

    for each $c$ in ${\rm~BR}_2$

    Calculate posterior of $a$ with respect to $c$;

    Add posterior to ${\rm~ST}_q$;

    end for

    return ${\rm~ST}_q$;

    Increment ${\rm~ST}_q$ to ${\rm~ST}_{q+1}$;

    for each $d$ in ${\rm~BR}_3$

    calculate posterior of $a$ with respect to $d$;

    add posterior to ${\rm~ST}_r$;

    end for

    return ${\rm~ST}_r$;

    Increment ${\rm~ST}_r$ to ${\rm~ST}_{r+1}$;

    end for

  • Table 2  

    Table 2Average MOS values for various stalling events under different bitrate levels
    Stalling event Bitrate (1 Mbps) Bitrate (5 Mbps) Bitrate (15 Mbps)
    No stalling 3.94 6.3 8.67
    Initial stalling 3.49 5.66 6.94
    Middle stalling 3.3 5.04 6.22
    Multiple stalling 2.79 4.38 5.44
    No stalling – initial stalling 0.45 0.64 1.73
    No stalling – middle stalling 0.64 1.26 2.45
    No stalling – multiple stalling 1.15 1.92 3.23
  • Table 3  

    Table 3Performance comparsion of the proposed technique
    Model PLCC SRCC
    Model 1 0.6769 0.6861
    Model 2 0.7190 0.7158
    Model 3 0.7476 0.7205
    Model 4 0.7723 0.7205
    Model 5 0.7905 0.7430
    Model 6 0.7395 0.6414
    Model 7 0.7455 0.6560
    Proposed technique 0.8095 0.7954