Give a concise description of current problem. What
needs to be solved? Why is the result useful? Do you make any assumptions?
What are the anticipated difficulties?
The current problems are:
Multiple Object Tracking
1. Implement a matching algorithm.
2. Implement a filter algorithm to estimate the state of each tracked object.
3. For the cell data set, implement a segmentation algorithm.
Method and Implementation
Give a concise description of the implemented method. For example, you might
describe the motivation of your idea, the algorithmic steps of your methods, or
the mathematical formulation of your method.
We implemented the greedy matching algorithm that matches the nearest measurement, and an alpha-beta filter algorithm to make predictions, which gives us results with good performance. We tried to use 3rd party Kalman filter implementations along with MHT, but they turned out to not have better performance than the simpler implementations possibly because of the parameters of Kalman filter needs to be tuned while we did not do well on that.
As for the cell data set, we apply background differencing, connected components sequential labeling, and then remove components upper in the image that do not represent cells. We then perform the algorithms earlier, alpha-beta filter and greedy matching algorithm, and then draw lines to show the tracking results. The image frames are output together as a video in the form of avi.
List your experimental results. Provide examples of input images and output images. If relevant, you may provide images showing any intermediate steps. If your work involves videos, do not submit the videos but only links to them.
result sample frame of bat tracking
result sample frame of cell tracking
result gif of bat dataset
result gif of cell dataset
Discuss your method and results:
- What are the strengths and weaknesses of your method?
- Do your results show that your method is generally successful or are there limitations? Describe what you expected to find in your experiments, and how that differed or was confirmed by your results.
- Potential future work. How could your method be improved? What would you try (if you had more time) to overcome the failures/limitations of your work?
The strengths are that we successfully track bats in the video, and display the results of the tracking algorithm on top of the original images with different colors. Using lines to show the history of the bats flying across the sky in a dark background
The weaknesses are that when bats get close to each other the algorithm may fail to distinguish them. And for the cell data set, it may lose track when the cell moves fast and chaotically.
Our method is generally successful.
We'll try Kalman Filter to replace the Alpha Beta Filter to improve the result.
Based on your discussion, what are your conclusions? What is your main message?
Our method is generally successful in tracking multiple objects, but it may fail when the objects overlap with each
other or move chaotically.
In response to
1. Show your tracking results on some portion of the sequence. In addition to showing your tracking results on an easy portion of the data, identify a challenging situation where your tracker succeeds, and a challenging situation where your tracker fails.
Answer: The results are shown above in images/gifs. It succeeds even when cells having different velocities in two consecutive frames. It fails when multiple objects touch each other, mostly due to direct application of connected components labeling algorithm.
2. How do you decide to begin new tracks and terminate old tracks as the objects enter and leave the field of view?
Answer:We create new tracks when objects enter the field of view and terminate old tracks at the frame where old track leaves field of view. We think Kalman filter could probably do better on this part.
3. What happens with your algorithm when objects touch and occlude each other, and how could you handle this so you do not break track?
Answer:In the bat frames result, in the upper part when two bats fly together to merge, our connected components labeling algorithm identify them as one single component, thus ruining the tracking. If there is anything can be done to it, we would modify our components labeling algorithm along with our tracking functions to identify components that merges. Their different velocities would greatly help to identify them.
4. What happens when there are spurious detections that do not connect with other measurements in subsequent frames?
Answer:I think the math in matching algorithm will determine the results, i.e. parameters of the model.
5. What are the advantages and drawbacks of different kinematic models: Do you need to model the velocity of the objects, or is it sufficient to just consider the distances between the objects in subsequent frames?
Answer:Just consider the distances is definitely not enough, the reason why our model has decent results is because we modeled the velocities.
Credits and Bibliography
Cite any papers or other references you consulted while developing your solution. Citations to papers should include the authors, the year of publication, the title of the work, and the publication information (e.g., book name and publisher; conference proceedings and location; journal name, volume and pages; technical report and institution).
Material on the web should include the url and date of access.
Credit any joint work or discussions with your classmates.
I complete this homework with my teammates Yuqi Guo and Siyang Li.