ClassesClasses | | Operators

train_class_svmtrain_class_svmTrainClassSvmTrainClassSvm (Operator)

Name

train_class_svmtrain_class_svmTrainClassSvmTrainClassSvm — Train a support vector machine.

Signature

train_class_svm( : : SVMHandle, Epsilon, TrainMode : )

Herror train_class_svm(const Hlong SVMHandle, double Epsilon, const char* TrainMode)

Herror T_train_class_svm(const Htuple SVMHandle, const Htuple Epsilon, const Htuple TrainMode)

void TrainClassSvm(const HTuple& SVMHandle, const HTuple& Epsilon, const HTuple& TrainMode)

void HClassSvm::TrainClassSvm(double Epsilon, const HTuple& TrainMode) const

void HClassSvm::TrainClassSvm(double Epsilon, const HString& TrainMode) const

void HClassSvm::TrainClassSvm(double Epsilon, const char* TrainMode) const

static void HOperatorSet.TrainClassSvm(HTuple SVMHandle, HTuple epsilon, HTuple trainMode)

void HClassSvm.TrainClassSvm(double epsilon, HTuple trainMode)

void HClassSvm.TrainClassSvm(double epsilon, string trainMode)

Description

train_class_svmtrain_class_svmTrainClassSvmTrainClassSvmTrainClassSvm trains the support vector machine (SVM) given in SVMHandleSVMHandleSVMHandleSVMHandleSVMHandle. Before the SVM can be trained, the training samples to be used for the training must be added to the SVM using add_sample_class_svmadd_sample_class_svmAddSampleClassSvmAddSampleClassSvmAddSampleClassSvm or read_samples_class_svmread_samples_class_svmReadSamplesClassSvmReadSamplesClassSvmReadSamplesClassSvm.

Technically, training an SVM means solving a convex quadratic optimization problem. This implies that it can be assured that training terminates after finite steps at the global optimum. In order to recognize termination, the gradient of the function that is optimized internally must fall below a threshold, which is set in EpsilonEpsilonEpsilonEpsilonepsilon. By default, a value of 0.001 should be used for EpsilonEpsilonEpsilonEpsilonepsilon since this yields the best results in practice. A too big value leads to a too early termination and might result in suboptimal solutions. With a too small value the optimization requires a longer time, often without changing the recognition rate significantly. Nevertheless, if longer training times are possible, a smaller value than 0.001 might be chosen. There are two common reasons for changing EpsilonEpsilonEpsilonEpsilonepsilon: First, if you specified a very small value for Nu when calling (create_class_svmcreate_class_svmCreateClassSvmCreateClassSvmCreateClassSvm), e.g., Nu = 0.001, a smaller EpsilonEpsilonEpsilonEpsilonepsilon might significantly improve the recognition rate. A second case is the determination of the optimal kernel function and its parameterization (e.g., the KernelParam-Nu pair for the RBF kernel) with the computationally intensive n-fold cross validation. Here, choosing a bigger EpsilonEpsilonEpsilonEpsilonepsilon reduces the computational time without changing the parameters of the optimal kernel that would be obtained when using the default EpsilonEpsilonEpsilonEpsilonepsilon. After the optimal KernelParam-Nu pair is obtained, the final training is conducted with a small EpsilonEpsilonEpsilonEpsilonepsilon.

The duration of the training depends on the training data, in particular on the number of resulting support vectors (SVs), and EpsilonEpsilonEpsilonEpsilonepsilon. It can lie between seconds and several hours. It is therefore recommended to choose the SVM parameter Nu in create_class_svmcreate_class_svmCreateClassSvmCreateClassSvmCreateClassSvm so that as few SVs as possible are generated without decreasing the recognition rate. Special care must be taken with the parameter Nu in create_class_svmcreate_class_svmCreateClassSvmCreateClassSvmCreateClassSvm so that the optimization starts from a feasible region. If too many training errors are chosen with a too big Nu, an exception is raised. In this case, an SVM with the same training data, but with smaller Nu must be trained.

With the parameter TrainModeTrainModeTrainModeTrainModetrainMode you can choose between different training modes. Normally, you train an SVM without additional information and TrainModeTrainModeTrainModeTrainModetrainMode is set to 'default'"default""default""default""default". If multiple SVMs for the same data set but with different kernels are trained, subsequent training runs can reuse optimization results and thus speedup the overall training time of all runs. For this mode, in TrainModeTrainModeTrainModeTrainModetrainMode a SVM handle of a previously trained SVM is passed. Note that the SVM handle passed in SVMHandleSVMHandleSVMHandleSVMHandleSVMHandle and the SVMHandle passed in TrainModeTrainModeTrainModeTrainModetrainMode must have the same training data, the same mode and the same number of classes (see create_class_svmcreate_class_svmCreateClassSvmCreateClassSvmCreateClassSvm). The application for this training mode is the evaluation of different kernel functions given the same training set. In the literature this is referred to as alpha seeding.

With TrainModeTrainModeTrainModeTrainModetrainMode = 'add_sv_to_train_set'"add_sv_to_train_set""add_sv_to_train_set""add_sv_to_train_set""add_sv_to_train_set" it is possible to append the support vectors that were generated by a previous call of train_class_svmtrain_class_svmTrainClassSvmTrainClassSvmTrainClassSvm to the currently saved training set. This mode has two typical application areas: First, it is possible to gradually train a SVM. For this, the complete training set is divided into disjunctive chunks. The first chunk is trained normally using TrainModeTrainModeTrainModeTrainModetrainMode = 'default'"default""default""default""default". Afterwards, the previous training set is removed with clear_samples_class_svmclear_samples_class_svmClearSamplesClassSvmClearSamplesClassSvmClearSamplesClassSvm, the next chunk is added with add_sample_class_svmadd_sample_class_svmAddSampleClassSvmAddSampleClassSvmAddSampleClassSvm and trained with TrainModeTrainModeTrainModeTrainModetrainMode = 'add_sv_to_train_set'"add_sv_to_train_set""add_sv_to_train_set""add_sv_to_train_set""add_sv_to_train_set". This is repeated until all chunks are trained. This approach has the advantage that even huge training data sets can be trained efficiently with respect to memory consumption. A second application area for this mode is that a general purpose classifier can be specialized by adding characteristic training samples and then retraining it. Please note that the preprocessing (as described in create_class_svmcreate_class_svmCreateClassSvmCreateClassSvmCreateClassSvm) is not changed when training with TrainModeTrainModeTrainModeTrainModetrainMode = 'add_sv_to_train_set'"add_sv_to_train_set""add_sv_to_train_set""add_sv_to_train_set""add_sv_to_train_set".

Execution Information

This operator modifies the state of the following input parameter:

The value of this parameter may not be shared across multiple threads without external synchronization.

Parameters

SVMHandleSVMHandleSVMHandleSVMHandleSVMHandle (input_control, state is modified)  class_svm HClassSvm, HTupleHTupleHtuple (integer) (IntPtr) (Hlong) (Hlong)

SVM handle.

EpsilonEpsilonEpsilonEpsilonepsilon (input_control)  real HTupleHTupleHtuple (real) (double) (double) (double)

Stop parameter for training.

Default value: 0.001

Suggested values: 0.00001, 0.0001, 0.001, 0.01, 0.1

TrainModeTrainModeTrainModeTrainModetrainMode (input_control)  number HTupleHTupleHtuple (string / integer) (string / int / long) (HString / Hlong) (char* / Hlong)

Mode of training. For normal operation: 'default'. If SVs already included in the SVM should be used for training: 'add_sv_to_train_set'. For alpha seeding: the respective SVM handle.

Default value: 'default' "default" "default" "default" "default"

List of values: 'add_sv_to_train_set'"add_sv_to_train_set""add_sv_to_train_set""add_sv_to_train_set""add_sv_to_train_set", 'default'"default""default""default""default"

Example (HDevelop)

* Train an SVM
create_class_svm (NumFeatures, 'rbf', 0.01, 0.01, NumClasses,\
                  'one-versus-all', 'normalization', NumFeatures,\
                  SVMHandle)
read_samples_class_svm (SVMHandle, 'samples.mtf')
train_class_svm (SVMHandle, 0.001, 'default')
write_class_svm (SVMHandle, 'classifier.svm')
clear_class_svm (SVMHandle)

Result

If the parameters are valid the operator train_class_svmtrain_class_svmTrainClassSvmTrainClassSvmTrainClassSvm returns the value 2 (H_MSG_TRUE). If necessary, an exception is raised.

Possible Predecessors

add_sample_class_svmadd_sample_class_svmAddSampleClassSvmAddSampleClassSvmAddSampleClassSvm, read_samples_class_svmread_samples_class_svmReadSamplesClassSvmReadSamplesClassSvmReadSamplesClassSvm

Possible Successors

classify_class_svmclassify_class_svmClassifyClassSvmClassifyClassSvmClassifyClassSvm, write_class_svmwrite_class_svmWriteClassSvmWriteClassSvmWriteClassSvm, create_class_lut_svmcreate_class_lut_svmCreateClassLutSvmCreateClassLutSvmCreateClassLutSvm

Alternatives

train_dl_classifier_batchtrain_dl_classifier_batchTrainDlClassifierBatchTrainDlClassifierBatchTrainDlClassifierBatch, read_class_svmread_class_svmReadClassSvmReadClassSvmReadClassSvm

See also

create_class_svmcreate_class_svmCreateClassSvmCreateClassSvmCreateClassSvm

References

John Shawe-Taylor, Nello Cristianini: “Kernel Methods for Pattern Analysis”; Cambridge University Press, Cambridge; 2004.
Bernhard Schölkopf, Alexander J.Smola: “Learning with Kernels”; MIT Press, London; 1999.

Module

Foundation


ClassesClasses | | Operators