Useful Links

Here are a few links and brief summaries for some of the papers we'll be working from:

Behavior Recognition via Sparse Spatio-Temporal Features by Dollár et al.
The main paper we'll be using for this project. This paper introduces the use of a response function based on a quadrature pair of gabor filters applied temporally and a 2d Gaussian applied along the spatial dimension. Cuboids (small spatio-temporal video clips) are then extracted at each local maxima given by the response function applied to a clip of video. A transformation is then applied to the cuboid (the paper tests several) and a feature vector is then created. Since the amount of cuboids possible is large, but only a few types are possible, similar cuboids are then clustered together to form cuboid prototypes. Each behavior is then described as a video clip in which a given set of cuboid prototypes is present.








Learning to Detect Objects in Images via a Sparse, Part-Based Representation by Agarwal et al.
One of the first papers to use sparse features for object detection. Agarwal et al. use the Föstner corner detector to find interest points. They then use 2d windows around the interest points to create a vocabulary of parts. Objects are described by the presence and relative positioning of each part wrt other parts. SNoW is used to train a classifier based on those features.

Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words by Niebles et al.
Based on Dollár et al. this paper uses the same response function, however, they use a probabilistic Latent Semantic Analysis (not quite sure how this works yet) model to determine behavior.


Some useful tutorials and manuals:
Gabor Filters
Fairly complex tutorial, still having a hard time completely understanding Gabor filters.
SNoW (Sparse Network of Winnows)
Described by Roth as a "multi-class classifier", the executable is available on their website. We might use this to include relative cuboid positioning in our project.

Presentations:
Object Recognition using sparse features
Presentation for Agarwal et al.'s paper. Includes a short demo on SNoW.
Behavior Recognition using Cuboids
Presentation for Dollár et al.'s paper.

Datasets:
Mouse behavior dataset
sets of clips obtained from the smart vivarium