Overview

Unsupervised Learning of Video Representations using LSTMs

Long-term Future Prediction

A test set for evaluating sequence prediction/reconstruction

Moving MNIST [782Mb] contains 10,000 sequences each of length 20 showing 2 digits moving in a 64 x 64 frame.

The results in the updated arxiv paper use this test set to report numbers. For future prediction, the metric is cross entropy loss for predicting the last 10 frames for each sequence conditioned on the first 10 frames.

Citation

Please use the following citation when referencing the dataset:

@inproceedings{srivastava2015unsupervised,
  title={Unsupervised learning of video representations using lstms},
  author={Srivastava, Nitish and Mansimov, Elman and Salakhudinov, Ruslan},
  booktitle={International conference on machine learning},
  pages={843--852},
  year={2015}
}