`create_class_svm`

— Create a support vector machine for pattern classification.

**create_class_svm**( : : *NumFeatures*, *KernelType*, *KernelParam*, *Nu*, *NumClasses*, *Mode*, *Preprocessing*, *NumComponents* : *SVMHandle*)

`create_class_svm`

creates a support vector machine that can
be used for pattern classification. The dimension of the patterns
to be classified is specified in * NumFeatures*, the number of
different classes in

`NumClasses`

For a binary classification problem in which the classes are
linearly separable the SVM algorithm selects data vectors from the
training set that are utilized to construct the optimal separating
hyperplane between different classes. This hyperplane is optimal in
the sense that the margin between the convex hulls of the different
classes is maximized. The training patterns that are located at the
margin define the hyperplane and are called *support vectors* (SV).

Classification of a feature vector z is performed with the
following formula:
Here, are the support vectors,
encodes their class membership () and
the weight coefficients. The distance of
the hyperplane to the origin is b. The and b
are determined during training with `train_class_svm`

. Note
that only a subset of the original training set (: number
of support vectors) is necessary for the definition of the decision boundary
and therefore data vectors that are not support vectors are discarded. The
classification speed depends on the evaluation of the dot product between
support vectors and the feature vector to be classified, and hence depends on
the length of the feature vector and the number of support
vectors.

For classification problems in which the classes are not linearly
separable the algorithm is extended in two ways. First, during
training a certain amount of errors (overlaps) is compensated with
the use of slack variables. This means that the
are upper bounded by a regularization constant. To enable an
intuitive control of the amount of training errors, the
*Nu*-SVM version of the training algorithm is used. Here, the
regularization parameter * Nu* is an asymptotic upper bound on
the number of training errors and an asymptotic lower bound on the
number of support vectors. As a rule of thumb, the parameter

`Nu`

`Nu`

`train_class_svm`

for more details). Since this can only be
determined during training, an exception can only be raised there.
In this case, a new SVM with `Nu`

Second, because the above SVM exclusively calculates dot products between the feature vectors, it is possible to incorporate a kernel function into the training and testing algorithm. This means that the dot products are substituted by a kernel function, which implicitly performs the dot product in a higher dimensional feature space. Given the appropriate kernel transformation, an originally not linearly separable classification task becomes linearly separable in the higher dimensional feature space.

Different kernel functions can be selected with the parameter
* KernelType*. For

`KernelType`

`KernelParam`

The radial basis function (RBF) * KernelType* =

`KernelParam`

`Nu`

With * KernelType* =

`KernelParam`

As a rule of thumb, the RBF kernel provides a good choice for most
of the classification problems and should therefore be used in
almost all cases. Nevertheless, the linear and polynomial kernels
might be better suited for certain applications and can be tested
for comparison. Please note that the novelty-detection * Mode*
and the operator

`reduce_class_svm`

are provided only for the RBF
kernel.
* Mode* specifies the general classification task, which is
either how to break down a multi-class decision problem to binary
sub-cases or whether to use a special classifier mode called

`Mode`

`Mode`

`Mode`

A special case of classification is * Mode* =

`NumClasses`

`Nu`

The parameters * Preprocessing* and

`NumComponents`

`Preprocessing`

`NumComponents`

For all other values of * Preprocessing*, the training data
set is used to compute a transformation of the feature vectors
during the training as well as later in the classification.

For * Preprocessing* =

`NumComponents`

`'circularity'`

(unit: scalar) and
`'area'`

(unit: pixel squared)). The normalization transformation
should be performed in general, because it increases the numerical
stability during training/testing.
For * Preprocessing* =

`NumComponents`

`NumFeatures`

`get_prep_info_class_svm`

can be used
to determine how much information each transformed component
contains. Hence, it aids the selection of `NumComponents`

The transformation specified by * Preprocessing* =

`Preprocessing`

`NumClasses`

`NumFeatures`

`get_prep_info_class_svm`

. Like principal
component analysis, canonical variates can be used to reduce the
amount of data without losing a large amount of information, while
additionally optimizing the separability of the classes after the
data reduction. The computation of the canonical variates is also
called linear discriminant analysis.
For the last two types of transformations
(*'principal_components'* and *'canonical_variates'*),
the length of input data of the SVM is determined by
* NumComponents*, whereas

`NumFeatures`

After the SVM has been created with `create_class_svm`

,
typically training samples are added to the SVM by repeatedly
calling `add_sample_class_svm`

or
`read_samples_class_svm`

. After this, the SVM is typically
trained using `train_class_svm`

. Hereafter, the SVM can be
saved using `write_class_svm`

. Alternatively, the SVM can be
used immediately after training to classify data using
`classify_class_svm`

.

A comparison of the SVM and the multi-layer perceptron (MLP) (see
`create_class_mlp`

) typically shows that SVMs are generally
faster at training, especially for huge training sets, and achieve
slightly better recognition rates than MLPs. The MLP is faster at
classification and should therefore be preferred in time critical
applications. Please note that this guideline assumes optimal
tuning of the parameters.

- Multithreading type: reentrant (runs in parallel with non-exclusive operators).
- Multithreading scope: global (may be called from any thread).
- Processed without parallelization.

This operator returns a handle. Note that the state of an instance of this handle type may be changed by specific operators even though the handle is used as an input parameter by those operators.

`NumFeatures`

`→`

(integer)
Number of input variables (features) of the SVM.

Default: 10

Suggested values: 1, 2, 3, 4, 5, 8, 10, 15, 20, 30, 40, 50, 60, 70, 80, 90, 100

Restriction:
`NumFeatures >= 1`

`KernelType`

`→`

(string)
The kernel type.

Default: 'rbf'

List of values: 'linear' , 'polynomial_homogeneous' , 'polynomial_inhomogeneous' , 'rbf'

`KernelParam`

`→`

(real)
Additional parameter for the kernel function. In case of RBF kernel the value for . For polynomial kernel the degree

Default: 0.02

Suggested values: 0.01, 0.02, 0.05, 0.1, 0.5

`Nu`

`→`

(real)
Regularization constant of the SVM.

Default: 0.05

Suggested values: 0.0001, 0.001, 0.01, 0.05, 0.1, 0.2, 0.3

Restriction:
`Nu > 0.0 && Nu < 1.0`

`NumClasses`

`→`

(integer)
Number of classes.

Default: 5

Suggested values: 2, 3, 4, 5, 6, 7, 8, 9, 10

Restriction:
`NumClasses >= 1`

`Mode`

`→`

(string)
The mode of the SVM.

Default: 'one-versus-one'

List of values: 'novelty-detection' , 'one-versus-all' , 'one-versus-one'

`Preprocessing`

`→`

(string)
Type of preprocessing used to transform the feature vectors.

Default: 'normalization'

List of values: 'canonical_variates' , 'none' , 'normalization' , 'principal_components'

`NumComponents`

`→`

(integer)
Preprocessing parameter: Number of transformed
features (ignored for * Preprocessing* =

`Preprocessing`

Default: 10

Suggested values: 1, 2, 3, 4, 5, 8, 10, 15, 20, 30, 40, 50, 60, 70, 80, 90, 100

Restriction:
`NumComponents >= 1`

`SVMHandle`

`→`

(handle)
SVM handle.

create_class_svm (NumFeatures, 'rbf', 0.01, 0.01, NumClasses,\ 'one-versus-all', 'normalization', NumFeatures,\ SVMHandle) * Generate and add the training data for J := 0 to NumData-1 by 1 * Generate training features and classes * Data = [...] * Class = ... add_sample_class_svm (SVMHandle, Data, Class) endfor * Train the SVM train_class_svm (SVMHandle, 0.001, 'default') * Use the SVM to classify unknown data for J := 0 to N-1 by 1 * Extract features * Features = [...] classify_class_svm (SVMHandle, Features, 1, Class) endfor

If the parameters are valid the operator `create_class_svm`

returns the value `2` (
`H_MSG_TRUE`)
. If necessary, an exception is
raised.

`read_dl_classifier`

,
`create_class_mlp`

,
`create_class_gmm`

`clear_class_svm`

,
`train_class_svm`

,
`classify_class_svm`

Bernhard Schölkopf, Alexander J.Smola: “Learning with Kernels”;
MIT Press, London; 1999.

John Shawe-Taylor, Nello Cristianini: “Kernel Methods for Pattern
Analysis”; Cambridge University Press, Cambridge; 2004.

Foundation