Recurrent Fully Convolutional Networks Based on Optical Flow for Video Eyes Fixation Prediction

Abstract

Although the research on eye fixation prediction has been activated in recent years, most of the methods are on images and cannot be directly applied to videos. In this paper, a recurrent fully convolutional neural network (RFCN) based on the optical flow framework was proposed for predicting the eye fixation. Seven frames and their optical flow feature were used as the input. We used the RFCN without deconvolution layers to extract the frames and their optical flow of the video to obtain the convolution features. Adjacent convolution features were merged as the next input. A Gaussian blob emphasized the center tendency of human eyes fixation when calculating the convolutional features. Finally, VAGBA and CRCNS public video databases were used to evaluate the model. The experimental results show that the prediction performance is better than the contrastive algorithms.