logo

SCIENTIA SINICA Informationis, Volume 48 , Issue 3 : 261-273(2018) https://doi.org/10.1360/N112017-00243

A luminance compensation method for fisheye video panorama stitching

More info
  • ReceivedJan 7, 2018
  • AcceptedJan 31, 2018
  • PublishedMar 16, 2018

Abstract


Funded by

国家高技术研究发展计划(2015AA015903)


References

[1] Deng X M, Wu F C, Wu Y H, et al. Automatic spherical panorama generation with two fisheye images. In: Proceedings of the 7th World Congress on Intelligent Control and Automation, Chongqing, 2008. 5955--5959. Google Scholar

[2] Ho T, Budagavi M. Dual-fisheye lens stitching for 360-degree imaging. In: Proceedings of International Conference on Acoustics, Speech and Signal Processing, New Orleans, 2017. Google Scholar

[3] Liao W S, Hsieh T J, Liang W Y, et al. Real-time spherical panorama image stitching using OpenCL. In: Proceedings of International Conference on Computer Graphics and Virtual Reality, Las Vegas, 2011. 113--119. Google Scholar

[4] Xu Y, Zhou Q, Gong L. High-Speed Simultaneous Image Distortion Correction Transformations for a Multicamera Cylindrical Panorama Real-time Video System Using FPGA. IEEE Trans Circuits Syst Video Technol, 2014, 24: 1061-1069 CrossRef Google Scholar

[5] Reinhard E, Adhikhmin M, Gooch B, et al. Color transfer between images. IEEE Comput Graph Appl, 2001, 21: 34--41. Google Scholar

[6] Tian G Y, Gledhill D, Taylor D, et al. Colour correction for panoramic imaging. In: Proceedings of the 6th International Conference on Information Visualisation, London, 2002. 483--488. Google Scholar

[7] Brown M, Lowe D G. Automatic Panoramic Image Stitching using Invariant Features. Int J Comput Vision, 2007, 74: 59-73 CrossRef Google Scholar

[8] Kim Y N, Sim D G. Vignetting and illumination compensation for omni-directional image generation on spherical coordinate. In: Proceedings of the 16th International Conference on Artificial Reality and Telexistence--Workshops, Hangzhou, 2006. 413--418. Google Scholar

[9] Fecker U, Barkowsky M, Kaup A. Histogram-Based Prefiltering for Luminance and Chrominance Compensation of Multiview Video. IEEE Trans Circuits Syst Video Technol, 2008, 18: 1258-1267 CrossRef Google Scholar

[10] Yamamoto K, Oi R. Color correction for multi-view video using energy minimization of view networks. Int J Autom Comput, 2008, 5: 234-245 CrossRef Google Scholar

[11] Basu A, Licardie S. Alternative models for fish-eye lenses. Pattern Recognition Lett, 1995, 16: 433-441 CrossRef Google Scholar

[12] Salomon D. Transformations and Projections in Computer Graphics. Berlin: Springer, 2007. Google Scholar

[13] Ye Y, Alshina E, Boyce J. Algorithm descriptions of projection format conversion and video quality metrics in 360Lib version 54. In: Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Document: JVET-H1004, Macau, 2017. Google Scholar

[14] Szeliski R, Shum H Y. Creating full view panoramic image mosaics and environment maps. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM, 1997. 251--258. Google Scholar

[15] Sun Y, Lu A, Yu L. Ahg8: WS-PSNR for 360 video objective quality evaluation. In: Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Document: JVET-D0040, Chengdu, 2016. Google Scholar

[16] Chen J L, Liu J L, Ye J H, et al. Luminance and chrominance correction for multi-view video using overlapped local histogram matching. Chinese J Image Graph, 2007, 12: 1992--1999. Google Scholar

[17] Ruderman D L, Cronin T W, Chiao C C. Statistics of cone responses to natural images: implications for visual coding. J Opt Soc Am A, 1998, 15: 2036-2045 CrossRef ADS Google Scholar

  • Figure 1

    (Color online) Fisheye-based video caption and stitching system. (a) Frame for 6 GoPro cameras; (b) fisheye image; (c) equirectangular projection

  • Figure 2

    (Color online) Overlap region before and after equirectangular projection. (a) Origin image; (b) equirectangular projection result and the overlap region; (c) overlap region in the origin image; (d) histogram of the two regions

  • Figure 3

    Preprocessing of the fisheye image

  • Figure 4

    (Color online) (a) Weight distribution map of WS-PSNR; (b) density distribution of an equirectangular projection panorama image

  • Figure 6

    (Color online) Real-time panorama video stitching system

  • Figure 7

    (Color online) The luminance compensation results based on histogram statistics with (a) original images,protectłinebreak (b) corrected images

  • Figure 8

    Histograms of three input images. From left to right are img0, img1, img5. (a) Histogram of overlap region with previous image; (b) histogram of overlap region with next image

  • Figure 9

    (Color online) Different results of different reference images. (a) Selecting img5 (score=0.8527) as reference image; (b) Selecting img0 (score=1.5063) as reference image

  • Figure 10

    (Color online) Experiment results of referenced method and proposal method. (a) Ref. [5], YUV color space;protect łinebreak (b) proposal method

  • Table 1   Effectiveness and efficiency of luminance compensation with different min/max sampling intervals
    Index (min, max) Time (ms) PSNR WS-PSNR Index (min, max) Time (ms) PSNR WS-PSNR
    1 (1, 1024) 9.08 31.4385 31.5229 9 (2, 256) 10.05 34.2231 34.1798
    2 (1, 512) 9.93 31.9630 32.0129 10 (4, 256) 9.11 34.4783 34.4546
    3 (1, 256) 11.57 32.8647 32.8514 11 (8, 256) 8.24 35.7685 35.7989
    4 (1, 128) 13.30 33.9658 33.8869 12 (16, 256) 7.51 38.0210 37.9278
    5 (1, 64) 14.99 34.5430 34.4906 13 (32, 256) 6.97 37.1566 37.1456
    6 (1, 32) 17.54 35.7429 35.6996 14 (64, 256) 6.58 34.8895 35.1714
    7 (1, 16) 20.52 37.0278 36.8767 15 (128, 256) 6.38 34.4281 34.0717
    8 (1, 8) 24.31 39.6580 39.5388
  • Table 2   Experiment results for efficiency (100 frames)
    Method Time (ms)
    [5], YUV 2173
    Proposal method (without adaptive reference image) 1115
    Proposal method (with adaptive reference image) 1143