##############################################################################
# This file provides installation instructions to reproduce the results      #
# of the GCPR paper ID 8:                                                    #
# MEASURING THE ACCURACY OF OBJECT DETECTORS AND TRACKERS                    #
#                                                                            #
# Additionally it contains a manual on how to use the App that was within    #
# the supplementary material.                                                #
#                                                                            #
# The code is licensed under the permissive modified BSD licence.            #
# All use of this software is subject to the terms of the revised BSD        #
# license.  You should have received a copy of this license along            #
# with this source code in a file named "license.txt."                       #
#                                                                            #
# The annotaion data is licensed under the Creative Commons Attribution 4.0  #
# International License (CC-BY-4.0).                                         #
# You should have received a copy of the license along with this             #
# work (please see data_license.txt). If not, see                            #
# <http://creativecommons.org/licenses/by/4.0>.                              #
#                                                                            #
# If you use the code or the data, please cite the corresponding paper:      #
# T. Böttger, P. Follmann, M. Fauser: Measuring the Accuracy of Object       #
# Detectors and Trackers, In: Proceedings of the 39th German Conference      #
# on Pattern Recognition (GCPR), 415-426, September 2017                     #
#                                                                            #
# Copyright (c) MVTec Software GmbH.  All rights reserved.                   #
##############################################################################

INSTALLATION INSTRUCTIONS AND MANUAL

We refer to the base-directory (where this file is contained) as $BASE_DIR.

$BASE_DIR/data:

  LOADING THE DATASETS

  First, if you don't have the DAVIS or VOTSEG datasets you can simply
  download and unzip them running the script

  load_data.sh
  
  The script also places the box_overlaps.csv and tracker_boxes.csv in the correct
  location. If you already have the datasets, only the last step in the script
  needs to be executed.

HALCON SCRIPTS - $BASE_DIR/code/hdev:

  This folder contains several HDevelop scripts that you can use if you have a
  valid HALCON license.
  We used HALCON to optimize the box parameters with respect to the IoU. The
  attached programs will reproduce the original values as they are within our
  paper.
  To switch between DAVIS and votseg2016, just manipulate the first code line
  in the scripts (Hence, change the variable ImageFolder)
  
  In order to use the HDevelop scripts you need to add the path
  $BASE_DIR/code/hdev/procedures
  to the list of procedure directories in HDevelop.

  There are four scripts which can be used to view the results or optimize the
  box parameters for new segmentations:
  
  theoretical_trackers_csv.hdev  -> This script allows to optimize the best box
  parameters for the DAVIS and votset dataset. The script allows to read and
  write the results to .csv file. The optmization of the complete datasets
  may take a while.
  
  theoretical_tracker_measures.hdev -> given a csv file containing the theoretical
  tracker ('tracker_boxes.csv' per default), this script computes the absolute
  and relative IoUs of the tracker within the file. Please note, your tracker
  will need to be added in TrackersNoScale, TrackersNoRotation or
  TrackersRect2 (lines 38-40).
  
  compare_tracker_results_csv.hdev -> this script plots the absolute IoU of all
  tracker and theoretical tracker given in the input file FileName
  (line 20, default 'tracker_boxes.csv')
  
  display_initial_rectangles.hdev  -> This script displays the 5 initial
  values which are used for the box parameter optimization. An arbitrary
  region can be drawn, and the best coordinates are computed. This script is
  merely there to give a better understanding of what is optimized.

PYTHON SCRIPTS AND APP - $BASE_DIR/code/python:

  In order to use our Python scripts, please add the following line to your
  .bashrc-file:
  
  export PYTHONPATH=$BASE_DIR/code/python/lib:$PYTHONPATH
  
  If you did not download the datasets using our script, please use a
  configuration file to correct the paths. An example is 
  $BASE_DIR/code/config_vot.yml
  to use the VOT dataset instead of the default DAVIS.
  Note that you have to call python scripts with optional argument --cfg
  <config_file> then.
  
  NECESSARY PYTHON MODULES

  We recommend using a python package manager such as Anaconda
  (https://conda.io/docs/). We are using Python 2.7. The following modules
  have to be installed in order to run the app:
    - easydict
    - matplotlib (v.1.5.1)
    - pandas
    - pillow
    - pyyaml

  EVALUATING ACCURACY MEASURES APP

  The "Evaluating Accuracy Measures" app is written in Python and Tkinter.
  
  To use on DAVIS, run the file 
  $BASE_DIR/code/python/bin/evaluate_tracker_accuracy.py

  In order to use it with the VOTseg dataset instead of DAVIS, please use:

  $BASE_DIR/code/python/bin/evaluate_tracker_accuracy.py \
    --cfg $BASE_DIR/code/python/config_vot.yml

  Note: The app was developed on a monitor with a rather high resolution. If
  the app does not fit your screen modify the configs STREAM_SIZE and
  PLOT_SIZE in the config-file.


  ADDING A NEW TRACKER TO THE APP

    WRITING THE TRACKING DATA TO THE CSV-FILES
  
  The app is reading box-coordinates and IoU-values from CSV-files that are
  located within the image-folders of each sequence (tracker_boxes.csv and
  box_overlaps.csv). If you want to add your own tracker you will first have
  to append the corresponding columns to these CSV-files.

  An example how to do so using Python is demonstrated in the script:

  bin/write_tracker_boxes_to_csv.py

  Note that per default this script generates new CSV-files
  tracker_boxes_new.csv and box_overlaps_new.csv. Hence you can first test
  your new tracker before you change the default CSV files. You will have to 
  change the output-csv-filenames inside this script if you want to append the
  values to tracker_boxes.csv and box_overlaps.csv.
  Also pay attention that you give a unique name to your tracker, if the name
  already exists the columns with the previous values will be overwritten!

  Please see the script for more detailed instructions!

  gen_box_overlaps_csv.py -> This script generates the IoU overlaps for all
  trackers within the tracker boxes csv file ('tracker_boxes.csv' per default).
  Only the IoU for tracker which are not already in the box_overlap file are
  added ('box_overlaps.csv' per default).

  We cannot guarantee that the IoU results are exactly the same as for HALCON. 

  In particular, also the runtime is a lot higher for Python than for HALCON
  (e.g. calculate absolute IoU of box with segmentation HALCON 0.025 ms vs
  Python 3.6 ms on average for DAVIS).

  If you have a tracker in MATLAB, please see the section below.

    ADDING THE TRACKER TO THE APP

  If you have already added tracker overlaps and boxes to the csv-file, you
  can simply add your new tracker by appending the list of REAL_TRACKERS
  inside your config-file. An example can be found in 

  config_davis_new_tracker.yml


MATLAB SCRIPTS - $BASE_DIR/code/matlab:
$BASE_DIR/code/matlab:
  
  The python app and the HALCON evaluation scripts are reading the
  box-coordinates and IoU-values from CSV-files that
  are located within the image-folders of each sequence (tracker_boxes.csv and
  box_overlaps.csv). If you want to add your own MATLAB tracker, all you have
  to do is to append the corresponding columns to these CSV-files. An example
  of how easy this can be done is provided in $BASE_DIR/code/matlab/example.m
  

  Essentially, the tracker results need to be converted to the 5 box
  parameters -> [row,col,width,height,phi]. For example, the results of all

  vot compliant trackers can be converted with convert_region2box_param.m.
  
  The function add_col_csv.m appends your tracker results to the csv file.
  
  To calculate the box overlaps, either call 
  $BASE_DIR/code/hdev/theoretical_tracker_measures or
  $BASE_DIR/code/python/gen_box_overlaps_csv.py

  The tracker can now be viewed with the python app. 
