Learning Visual Anticipation: A Top-Down Approach

Vempati Anurag Sai
Supervisor: Dr. Amitabha Mukerjee

Abstract:

This work aims at developing an anticipatory gaze model, in a top-down approach. In the past there have been many models that can track relevant objects from a video making use of computer vision. But one crucial difference is that, a human eye can learn to anticipate the trajectory of an object very quickly in an unsupervised fashion. A sportsperson knows the importance of trajectory estimation better than anyone else. For example, in cricket it is very crucial that the batsman estimates length, bounce and type (swing, spin etc.) of the delivery in a fraction of seconds. In a study by Land & McLeod, 2000 three skilled batsmen facing medium paced deliveries looked at the ball for the first 100-150ms of flight and then made a rapid glance (or saccade) at approximately 50-80% of the ball flight duration to the predicted ball bounce location. This kind of anticipation comes with prolonged training. We aim to build a computational model that could imitate the way humans learn to anticipate the trajectory of fast moving objects.

We start of building a dataset for the training phase. The dataset consists of a ball bouncing off the walls and floor as viewed from different viewpoints. The ball's speed and release direction are randomly chosen with a swing/spin components incorporated in few of the videos. Then, for each video, the ball is segmented out and all relevant data is collected. The objective to learn the Transision matrix can then be framed as a Linear Regression problem. Now for a given test video, conditioned on the data collected till then, a Kalman Filter predicts the future location of the ball well in advance.

A sample video from the dataset:

Predicted Trajectory:

Predticted Trajectory in 'BLUE' and size in 'RED'. Shown 7X slower



Documentation:

PROPOSAL    PRESENTATION    POSTER    FINAL REPORT    CODES

REFERENCES:

  1. Land, Michael F., and Peter McLeod. "From eye movements to actions: how batsmen hit the ball." Nature neuroscience 3.12 (2000): 1340- 1345.
  2. Sciutti, Alessandra, et al. "Anticipatory gaze in human-robot interactions." Gaze in HRI from modeling to communication" workshop at the 7th ACM/IEEE international conference on human-robot interaction, Boston, Massachusetts, USA. 2012.
  3. Perse, Matej, et al. "Physics-based modelling of human motion using kalman filter and collision avoidance algorithm." International Symposium on Image and Signal Processing and Analysis, ISPA05, Zagreb, Croatia. 2005
  4. http://en.wikipedia.org/wiki/Kalman_filter
  5. Thrun, Sebastian, et al. "Stanley: The robot that won the DARPA Grand Challenge." Journal of field Robotics 23.9 (2006): 661-692.
  6. Horak, Fay B. "Postural orientation and equilibrium: what do we need to know about neural control of balance to prevent falls?." Age and ageing 35.suppl 2 (2006): ii7-ii11.