Detecting Movement cont.

We tried to use blob detection in order to approximate the movement of the mouse. We took the centroid of the blobs to represent the center mass of the mouse, and the change in position of the centroid would approximate the change in the mouse position. However, using blobs this way turned out to be a chaos, we detected more noise than the actual movement of the mouse.

To resolve this issue, we tried using the cuboids. We computed a binary image of the cuboids that appear, and then calculate a centroid from the image. The centroid serves the same purpose as before: to approximate the center mass of the mouse. Here is the unfiltered graph of the mouse movement:

Unfiltered x and y displacement for exploring

Unfiltered x and y displacement for grooming

We noted that there is a distinctive pattern between exploring and grooming. The x and y displacement while the mouse is exploring is greater than when the mouse is grooming. Furthermore, the x and y displacement when the mouse is grooming tend to be around zero. This is intuitive because the mouse does not move as much when it is grooming than when it is exploring. We want to add this result as a feature to Piotr's cuboids, but we need to filter out more noise. We used median and average filtering to obtain the following results:

Filtered x displacement for exploring

Filtered y displacement for exploring

Filtered x displacement for grooming

Filtered y displacement for grooming

Next step:
- Create displacement graphs for all training set
- Scale the graphs to 100 frames for consistency

Detecting movement

After last week's class, we've implemented a way of detecting movement suggested by Serge. By subtracting the average background for each frame and binarizing the image, we can get a representation of movement.
We still need to work on the threshold used and clean up the blobs a bit, but it looks promising.

Here is the clip of blobs obtained from explore001 from set00:


Here are the blobs from groom001 from set00:


Current work:
1. Clean up our binary images, adjust the threshold to keep the number of blobs small.
2. Use each blob's centroid to create a displacement graph with respect to time in X and Y directions.
3. Create displacement prototypes based on these graphs by clustering them together.
4. Add this information to our behavior descriptor.

Most commonly mislabeled behavior

The memory problem was solved by saving and clearing the workspace after work on a specific clip is done, unfortunately this makes the code run a bit slower (5+ hrs). For now, we'll only be using a subset of the whole data set; set00-set003. One of the most commonly mislabeled behaviors using the cuboids code is grooming, it's most often labeled as exploring. Grooming is characterized by the movement of the mouse's paws across its face or its face across its legs while the mouse stays in the same place. Drinking is also commonly mislabeled as exploring. We believe the main difference between these two behaviors is the movement of the mouse from one place to another. By keeping track of where cuboids are detected and incorporating that data into the behavior descriptor, we believe we can increase the accuracy of these behaviors.

The following videos show the cuboids as they're detected by the response function.
(still waiting for processing by google video)

Here we have a sample clip for grooming where the mouse stays in the same spot and mostly just moves his paws and face:


A grooming clip where the mouse moves around a bit:



A sample clip for drinking:


A clip for exploring: