Our Behavior Descriptor

E-mailed Piotr last week asking for his implementation of cuboids. Since we'll be going through his code, we've decided to start working on our behavior descriptor. Specifically, how we'll represent the spatial relationships between cuboids.
Agarwal et al. keep track of spatial relationships between detected parts by dividing the angle between each pair into bins of 45 degrees and measuring the distance between parts by window size. They represent this information in the feature vector of each training image. Their feature vector is set up as a series of binary features, indicating whether or not a part or relationship is present.
Our task is to extend this into the spatio-temporal domain.
Possible ways to do this are:

1. Calculate distance and angle between each pair of cuboids in x, y coordinates, store time difference in a separate field.
   
2. Calculate euclidean distance between each pair of cuboids in 3d using x, y, and t coordinates.
   

Once we have the relationships between parts, we include them in our final behavior descriptor.
We'll most likely be using a histogram of the cuboid types present and the relationships between these.