Single-Object Tracking Evaluation

While the term “single-object tracking” can be sometimes ambiguous, in this task, we focus on generic single object tracking, also known as model-free tracking. In particular, for an input video sequence and the initial bounding box of the target object in the first frame, we requires a tracking algorithm to locate the target bounding boxes in the subsequent video frames. The tracking targets in these sequences include pedestrians, cars, buses, and animals. 

For this task, the performance is evaluated by the success and precision scores, same as in [1]. Notably, the success score is used as the primary metric for ranking methods. The metrics are described in the following table. 

Success Score100%The area under the curve (AUC) based on the percentage of successfully tracked frames vs. the bounding box overlap threshold
Precision Score100%The percentage of frames where the centers of the tracked object are within 20 pixels to the groundtruth

In addition, similar to [2], we have manually tagged the sequences with 12 attributes, which represents the challenging aspects in single-object tracking in drone view. We will report the performance in each attribute for comprehensive evaluation.

ARCAspect Ratio Change: the fraction of ground truth aspect ratio in the first frame and at least one subsequent frame is outside the range [0.5, 2].
BCBackground Clutter: the background near the target has similar appearance as the target.
CMCamera Motion: abrupt motion of the camera.
FMFast Motion: motion of the ground truth bounding box is larger than 20 pixels between two consecutive frames.
FOCFull Occlusion: the target is fully occluded.
IVIllumination Variation: the illumination of the target changes significantly.
LRLow Resolution: at least one ground truth bounding box has less than 400 pixels.
OVOut-of-View: some portion of the target leaves the view.
POCPartial Occlusion: the target is partially occluded.
SOBSimilar Object: there are objects of similar shape or same type near the target.
SVScale Variation: the ratio of initial and at least one subsequent bounding box is outside the range [0.5, 2].
VCViewpoint Change: viewpoint affects target appearance significantly.

The evaluation code for single-object tracking is available on the VisDrone github.


[1] Y. Wu, J. Lim, and M. Yang, “Object tracking benchmark,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 9, pp. 1834–1848, 2015.
[2] M. Mueller, N. Smith, B. Ghanem, “A Benchmark and Simulator for UAV Tracking,” European Conference on Computer Vision, vol. 1, pp. 445-461, 2016.