CS585 Assignment 4


E Chengyuan
teammate: Li Jun
Apr 2 2020

Programming Assignment: Multiple Object Tracking

Problem Definition

The goal of this part of the programming assignment is for you to learn more about the practical issues that arise when designing a tracking system. You are asked to track moving objects in video sequences, i.e., identifying the same object from frame to frame:

You may consider two frames at a time (or, more ambitiously, use multiple hypothesis tracking (MHT) with more than two frames). You may use a greedy, suboptimal bipartite matching algorithm (or, more ambitiously, implement an optimal data association algorithm). To estimate the state of each tracked object, you may use an alpha-beta filter (or, more ambitiously, a Kalman filter).

Database discription

1) The bat dataset shows bats in flight, where the bats appear bright against a dark sky. We included both grayscale and false-color images from this thermal image sequence; you may use whichever images you prefer.

2) The cell dataset shows mouse muscle stem cells moving in hydrogel microwells, the brightness of the pixels within the cells are very similar to the values of the background.

Method and Implementation

Part 1 - Tracking bats

  1. Pre-processing

    2. Load the image file and localization file , save them as panda dataframe.

  2. Data association using greedy algorithm

    3. First compute cost matrix for two sets of points, and pass into a greedy algorithm where first predicted_value (object tracked for the longest time) gets highest priority. Then Formulate the 2D assignment problem and obtain a global optimal solution as the best assignment hypothesis. Return the index of measurements associated to the predict value.

  3. Kalman Filter

    4. Implement a Kalman filter. Input image frames and measurements(loc_data), predict new loc based on previous prediction and current measurements. Return a list of tracks of objects between frames, from k-1 to k. A iteration process is shown in the picture below.

  4. Alpha-beta Filter

    5. Implement a Kalman filter, which is a general and simpler form of Kalman filter. The alpha-beta filter becomes a steady-state Kalman filter if filter parameters are calculated from the sampling interval T, the process variance sigma _{w}^{2} and the noise variance sigma _{v}^{2}.
    The parameters are set to: alpha = 0.85 beta = 0.005

  5. Visualization

    6. Based on a list of sequences related to time(traces for the objects), use cv2 to visualize the trace of each object. Then, output the frames as a video (shown in the result part).


Part 2 - Tracking cells

  1. Pre-processing

    2. Load the image file and localization file , save them as panda dataframe.

  2. Segmentation

    3. Segmentate cells in each frame to get individual cells for tracing purpose. Use binary thresholding and cv2.findContours to try to have a clear segmentaion. However, the segmentation is not easy and clean althrough some parameter modification is made, so it affects the tracking result.

  3. Same as in part 1

    4. Same as in part 1

  4. Kalman Filter

    5. Same as in part 1

  5. Alpha-beta Filter

    6. Same as in part 1

  6. Visualization

    7. Same as in part 1

Results

Tracking bats using Alphabeta filter

Tracking bats using Kalman filter

Tracking cells using Alphabeta filter

Tracking cells using Kalman filter

Discussion

For tracking bats

Alphabeta filter implemented as a baseline approach which works very well.

Kalman Filter is an optimal estimator - ie infers parameters of interest from indirect, inaccurate and uncertain observations. It is recursive so that new measurements can be processed as they arrive.

Kalman Filter implemented as a advanced approach which is still limited.
For data association use greedy method instead of validation gate(to reject outlier measurements). We use the ground truth position of the objects for bat tracking , so set measurement error to be trivial.

For tracking cells

The result is not perfect, but is resonable . The main problem is on the segmentation part. It is not easy and direct to segemnet each cells because of the lightness, color, boundries are not very different between cells and background.

Conclusions

The implementation of Alpha-beta filter and Kalman filter to track bats and cells is a success. Some improvemnet can be achived for segmenting cells for a better result.

Credits and Bibliography

http://bilgin.esme.org/BitsAndBytes/KalmanFilterforDummies
CS 585 Lab
https://www.cs.cmu.edu/~motionplanning/papers/sbp_papers/kalman/kleeman_understanding_kalman.pdf
https://en.wikipedia.org/wiki/Kalman_filter https://en.wikipedia.org/wiki/Alpha_beta_filter
http://www.cs.bu.edu/fac/betke/cs585/restricted/lectures/2020_Kalman_Filter_CS585.pdf

Collabrated with my teammate: Li Jun.