MVTec Software GmbH
 

Toolkit for Measuring the Accuracy of Object Trackers

Abstract

Toolkit for Measuring the Accuracy of Object Trackers

The accuracy of object detectors and trackers is most commonly evaluated by the Intersection over Union (IoU) of the tracker prediction and the ground truth. In all of the common tracking benchmarks, the ground truth is restricted to axis-aligned or oriented boxes. To help evaluate the accuracy of trackers more precisely, we present a toolkit which works with ground truth segmentations. To gain a perspective on how well all approaches restricted to boxes can perform, we present upper bounds for all box-based trackers of the Visual Object Tracking (VOT) and Densely Annotated Video Segmentation (DAVIS) challenges. The toolkit is easy-to-use, and arbitrary trackers from Python, Matlab, or HALCON can be added.

Publications

If you use the provided data, please cite the following work:

  • Tobias Böttger, Patrick Follmann, Michael Fauser: Measuring the Accuracy of Object Detectors and Trackers; in: Proceedings of the 39th German Conference on Pattern Recognition (GCPR), 415-426, September 2017.
  • Tobias Böttger, Patrick Follmann: The Benefits of Evaluating Tracker Performance using Pixel-wise Segmentations; in: IEEE International Conference on Computer Vision (ICCV), 1983-1991, October 2017.

Toolkit download

Download the code and upper bounds here:

Download data set (5 MB)

License

© Copyright 2017 MVTec Software GmbH

The code is released under the permissive modified BSD license. The data is released under the Creative Commons Attribution 4.0 International License (CC-BY-4.0).