Video saliency incorporating spatiotemporal cues and uncertainty weighting

IEEE Trans Image Process. 2014 Sep;23(9):3910-21. doi: 10.1109/TIP.2014.2336549. Epub 2014 Jul 16.

Abstract

We propose a novel algorithm to detect visual saliency from video signals by combining both spatial and temporal information and statistical uncertainty measures. The main novelty of the proposed method is twofold. First, separate spatial and temporal saliency maps are generated, where the computation of temporal saliency incorporates a recent psychological study of human visual speed perception. Second, the spatial and temporal saliency maps are merged into one using a spatiotemporally adaptive entropy-based uncertainty weighting approach. The spatial uncertainty weighing incorporates the characteristics of proximity and continuity of spatial saliency, while the temporal uncertainty weighting takes into account the variations of background motion and local contrast. Experimental results show that the proposed spatiotemporal uncertainty weighting algorithm significantly outperforms state-of-the-art video saliency detection models.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Algorithms
  • Artificial Intelligence
  • Biomimetics / methods*
  • Computer Simulation
  • Humans
  • Image Enhancement / methods
  • Image Interpretation, Computer-Assisted / methods*
  • Models, Statistical
  • Pattern Recognition, Automated / methods*
  • Photography / methods*
  • Reproducibility of Results
  • Sensitivity and Specificity
  • Spatio-Temporal Analysis
  • Subtraction Technique*
  • Video Recording / methods*
  • Visual Perception / physiology