Problem Definition
1. In part 1, use exist segmentaion and location information to track bats on sky.
2. In part 2, use exist contour image to track fish in tank.
Method and Implementation
Part 1:
In this bat tracking, the location and segmentation and labels of objects are given. So the main problem is to associate next measurement to existing tracking.
- If point can be associated to exist tracks, then use this new point as measurement, get prediction from kalman filter and use this measurement to correct it.
- If for some tracks, there is no new point associated to it, mark the track as missing point, and use Kalman filter which is using 4 state (last point's x and y and x, y velocity) and history of this track to predict the next position. And also keep a missing record, if a track keep missing for over a threshold frames (at here using 10 as default), then judge it as missing. Don't predict it any more and push Point(-1, -1) to this track.
- If for some new coming points, for which its distances to all tracks are too large(larger than a threshold called missing_newcoming_threshold), then mark points as new coming points. Push new coming points to track history and use Point(-1, -1) to pad missing history.
Part 2:
In this part, there is no segmentaion or location given, so we need to get segmentation and labels from images with contours.
1. Notice that contour in this image is red, so use imple color filter to get contours' binary image. Then apply dilation to this image to make sure contours are connected. Then find out all inside points in all objects. Then we get a binary segmentation image. Then label it and find mass point, if the mass point is at the outside of the object, use the nearest point to this mass point on contour as mass point.
2. After finding of mass point, use these points to track fishes. Use the same algorithm as mentioned above.
Experiments
Part 1:
Use the algorithm implemented above to get track histories. When creating a new track, we will assign a rand color to it. Then we draw lines using track histories and given random color. And also draw raw segmentation as background. Then output it as video.
Part 2:
Read images and use algorithms given above to calculate objects locations. Then use these locations to build tracks. Then use raw image as background, draw history tracks on it, every track use certain given color. Then output it as video.
Results
Part 1: Output of this part are saved as videos:bat_tracking_trackonly.avi, bat_tracking_withbat.avi
At here provide some screen shots of bat_tracking_withbat.avi.
Screen Shots | ||
 |
 |
 |
Part 2: output are saved as videos.
fish_tracking_withfish.avi, fish_tracking_withfish_raw.avi
At here provide some screen shot of video "fish_tracking_withfish_raw.avi"
Screen Shots | ||
 |
 |
 |
Discussion
Discussion about the algorithms I used:
1. Easy tracking situations and challengin situations. If check the tracks in bat videos carely, we will find that if a bat is very close to another bat's last mass position point, it's very possible for the program to assign that bat wrongly to another track. This is caused by the use of greedy algorithm and the only 2 frames analysis. Because at here we only process images frame by frame, if we wrongly associate a bat to another track, it will not be fixed automatically. So we can see track exchange between bats. But for frames that the distance between bats are far enough, this algorithm will not make much mistake and can work well.2. As discussed above in algorithm part, when a new point coming and the distances between it and all exist tracks are larger than a threshold, then that is a new coming point. Push it to history record and pad it with Point(-1, -1). And we use a counter to calculate continuous missing in a track(when we find it again, counter will reset), if counter is larger than a certain point, then we judge this track as missing. All later points in this track are made to Point(-1, -1).
3. If a animals touch and occlude each other, then there will exist at least one missing track, then use kalman filter to predict the next position for one of them. After a while, we will find that we can see animals again.
4. If a spurious detections only occur for a shor time and don't connect with other measurements in subsequent frames, then we will find it will occur and missed quickly in tracking history. Then we can finally filter it out between its valid history is too short.
5. I use the velocity of the objects, and it actually works well. If we only consider the distances between the objects in subsequent frames, we will find that it easy for tracker to lost the object. And sometimes when a object suddenly speed up, velocity will still lost object. So if we introduce acceleration, the model will be more precise.
Conclusions
Algorithm implemented in part 1 and 2 both works well. We only use 2 frames at a time to associate points to history and use kalmen filter to predict missing points. We can get almost precise tracks, but still not perfect. If consider more frames together, we will get more precise history tracking.
Credits and Bibliography
Materials discussed in class
http://opencv.org/
http://www.tk4479.net/xiao__run/article/details/77478579
https://stackoverflow.com/questions/25494595/multiple-object-tracking-with-kalman-filter