Object Tracking, Multiple Object Tracking, Behavior Analysis

CS585 HW4

Shuzhe Luan

Long Guo

04/03/2020

Problem Definition

In this assignment, we need to segement the image use track the objects. We assume that the velocity of objects don't change between frames and we use that assumption in our prediction.

Method and Implementation

Bats : First we use a greedy algorithm to find all bats from the processed image. Then we create the vector of bat object. Each bat have three values : id, state and mass. And state is a vector that is composed by {coordinate_x, coordinate_y, velocity_x, velocity_y}; After that, we use each bat's relative movement to predict their location on the next frame and compute the probability by prob = w1 * ratio of distance + w2 * ratio of weight Finally, we use KM algorithm to find a optimal match and update bats' states. Cells: First we compute the average mat of the first 50 frames to find the background Then we subtract the average mat from each frame to perform the background subtraction. After subtracting the background, we perform some morphological operations to remove the noise and fill the broken cells. And what's happening next is all the same.

Some code : code_1

produceAveVector : Compute the average mat. getBats : Find all objects. compareTwoFramesBat : Do the prediction, compute the probability, find the match and update the state. addTrajectories : Add trajectories that have been found. Perform the KM algorithm and find an optimal match.

Results

$\log \frac{\lVert next_j-prev_i\rVert}{\lVert V_i\cos(next_j-prev_i,V_i)\rVert}$ $2.0$ tuned to positive infinity.

bat_dscs_3_3000

We also tried on simple distance functions, i.e. calculating the distance between nodes and "predicted location".

bat_dst_3000

It clearly showed that the first one featured with less zigzagging, therefore a more stable tracking.

Below is the results on Cell dataset.

cell_200_1000

We spent most effort on improving the segmentation, as the main reason for unstable lines is the unstable centroid obtained from segmentation.

Discussions

In Bats dataset, since the pictures are already segmented and we can use the processed centroid, the tracking is relatively more accurate than Cells, which adds another layer of noise. The Bats centroids, however, still has some wobbling on traces, this however wouldn't be too troublesome for tracking. On the bottom of the image, there are many small bats presenting in this region, and our algorithm can keep the traces relatively intact, with few disruptions and switching back-and-forths. However, when two Bats overlapped, it is difficult to keep both traces intact, as there may be only one centroid presenting. In the Cells dataset, the tracking on computed centroids were rather easy, however, the centroids were difficult to obtain and keep, therefore making it difficult to preserve an intact trace.

When we found a matching pair with distance greater than a threshold, that implies a track terminated while a new object entered.
With two objects touch and occlude each other, if two centroids can be obtained, then there is nothing different while handling this case; however if only one centroid was found, then one of them may break, as we can't tell the difference between object vanishing and objects overlapping, and we have no way to tell the future.
Assume it would be a new addition to the image.
If the bats were moving in a rather constant speed, like on this occasion, calculating the velocity does not give much advantage comparing to just consider the distances between objects in subsequent frames, and it it much easier and quicker to just compute the distance. However, if it is moving in different speed, like accelerating, calculating speed creates a more precise prediction on the location on next frame.

Conclusion