Video Prediction refers to predicting future frames of a video given a few past frames. Video Prediction has found applications in video representation learning, robotics, compression, and many others. While researchers have focused on this problem of prediction, there has been very little work on the complementary problem of evaluating the predicted videos. To bridge this gap, we proposed video naturalness as an evaluation measure. In this regard, we have developed a new database of videos predicted by various models and conducted a human study to understand how humans perceive naturalness, which we call as the IISc Video Naturalness Evaluation (IISc-VINE) database. Using our database, we have developed a new naturalness measure based on deep features of videos. We show that, while popular quality assessment (QA) measures such as mean squared error (MSE) and structural similarity (SSIM) do not correlate adequately well with human perception (in this context), our algorithm achieves the state of the art performance w.r.t. correlation with human scores.
Nagabhushan S N
Nagabhushan S N is a PhD Student in Visual Information Processing lab in ECE Dept, working under the guidance of Prof. Rajiv Soundararajan. He obtained his B.E. degree in Electronics and Communications from P.E.S. Institute of Technology in 2016, with a gold medal. He worked as a Software Engineer in Cisco Systems India Pvt. Ltd. for 2 years (2016-18). His current research interests are in the areas of Image and Video Signal Processing, Machine Learning and Computer Vision. Nagabhushan’s personal webpage can be found at https://sites.google.com/site/nagabhushansn95.