select_feature_set_knnT_select_feature_set_knnSelectFeatureSetKnnSelectFeatureSetKnnselect_feature_set_knn (Operator)

Name

select_feature_set_knnT_select_feature_set_knnSelectFeatureSetKnnSelectFeatureSetKnnselect_feature_set_knn — Selects an optimal subset from a set of features to solve a certain classification problem.

Signature

select_feature_set_knn( : : ClassTrainDataHandle, SelectionMethod, GenParamName, GenParamValue : KNNHandle, SelectedFeatureIndices, Score)

Herror T_select_feature_set_knn(const Htuple ClassTrainDataHandle, const Htuple SelectionMethod, const Htuple GenParamName, const Htuple GenParamValue, Htuple* KNNHandle, Htuple* SelectedFeatureIndices, Htuple* Score)

void SelectFeatureSetKnn(const HTuple& ClassTrainDataHandle, const HTuple& SelectionMethod, const HTuple& GenParamName, const HTuple& GenParamValue, HTuple* KNNHandle, HTuple* SelectedFeatureIndices, HTuple* Score)

HTuple HClassKnn::SelectFeatureSetKnn(const HClassTrainData& ClassTrainDataHandle, const HString& SelectionMethod, const HTuple& GenParamName, const HTuple& GenParamValue, HTuple* Score)

HTuple HClassKnn::SelectFeatureSetKnn(const HClassTrainData& ClassTrainDataHandle, const HString& SelectionMethod, const HString& GenParamName, double GenParamValue, HTuple* Score)

HTuple HClassKnn::SelectFeatureSetKnn(const HClassTrainData& ClassTrainDataHandle, const char* SelectionMethod, const char* GenParamName, double GenParamValue, HTuple* Score)

HTuple HClassKnn::SelectFeatureSetKnn(const HClassTrainData& ClassTrainDataHandle, const wchar_t* SelectionMethod, const wchar_t* GenParamName, double GenParamValue, HTuple* Score)   (Windows only)

HClassKnn HClassTrainData::SelectFeatureSetKnn(const HString& SelectionMethod, const HTuple& GenParamName, const HTuple& GenParamValue, HTuple* SelectedFeatureIndices, HTuple* Score) const

HClassKnn HClassTrainData::SelectFeatureSetKnn(const HString& SelectionMethod, const HString& GenParamName, double GenParamValue, HTuple* SelectedFeatureIndices, HTuple* Score) const

HClassKnn HClassTrainData::SelectFeatureSetKnn(const char* SelectionMethod, const char* GenParamName, double GenParamValue, HTuple* SelectedFeatureIndices, HTuple* Score) const

HClassKnn HClassTrainData::SelectFeatureSetKnn(const wchar_t* SelectionMethod, const wchar_t* GenParamName, double GenParamValue, HTuple* SelectedFeatureIndices, HTuple* Score) const   (Windows only)

static void HOperatorSet.SelectFeatureSetKnn(HTuple classTrainDataHandle, HTuple selectionMethod, HTuple genParamName, HTuple genParamValue, out HTuple KNNHandle, out HTuple selectedFeatureIndices, out HTuple score)

HTuple HClassKnn.SelectFeatureSetKnn(HClassTrainData classTrainDataHandle, string selectionMethod, HTuple genParamName, HTuple genParamValue, out HTuple score)

HTuple HClassKnn.SelectFeatureSetKnn(HClassTrainData classTrainDataHandle, string selectionMethod, string genParamName, double genParamValue, out HTuple score)

HClassKnn HClassTrainData.SelectFeatureSetKnn(string selectionMethod, HTuple genParamName, HTuple genParamValue, out HTuple selectedFeatureIndices, out HTuple score)

HClassKnn HClassTrainData.SelectFeatureSetKnn(string selectionMethod, string genParamName, double genParamValue, out HTuple selectedFeatureIndices, out HTuple score)

def select_feature_set_knn(class_train_data_handle: HHandle, selection_method: str, gen_param_name: MaybeSequence[str], gen_param_value: MaybeSequence[Union[int, str, float]]) -> Tuple[HHandle, Sequence[str], Sequence[float]]

Description

select_feature_set_knnselect_feature_set_knnSelectFeatureSetKnnSelectFeatureSetKnnSelectFeatureSetKnnselect_feature_set_knn selects an optimal subset from a set of features to solve a certain classification problem. The classification problem has to be specified with annotated training data in ClassTrainDataHandleClassTrainDataHandleClassTrainDataHandleClassTrainDataHandleclassTrainDataHandleclass_train_data_handle and will be classified by a a k-nearest neighbors classifier. Details of the properties of this classifier can be found in create_class_knncreate_class_knnCreateClassKnnCreateClassKnnCreateClassKnncreate_class_knn.

The result of the operator is a trained classifier that is returned in KNNHandleKNNHandleKNNHandleKNNHandleKNNHandleknnhandle. Additionally, the list of indices or names of the selected features is returned in SelectedFeatureIndicesSelectedFeatureIndicesSelectedFeatureIndicesSelectedFeatureIndicesselectedFeatureIndicesselected_feature_indices. To use this classifier, calculate for new input data all features mentioned in SelectedFeatureIndicesSelectedFeatureIndicesSelectedFeatureIndicesSelectedFeatureIndicesselectedFeatureIndicesselected_feature_indices and pass them to the classifier.

A possible application of this operator can be a comparison of different parameter sets for certain feature extraction techniques. Another application is to search for a property that is discriminating between different classes of parts or classes of errors.

To define the features that should be selected from ClassTrainDataHandleClassTrainDataHandleClassTrainDataHandleClassTrainDataHandleclassTrainDataHandleclass_train_data_handle, the dimensions of the feature vectors in ClassTrainDataHandleClassTrainDataHandleClassTrainDataHandleClassTrainDataHandleclassTrainDataHandleclass_train_data_handle can be grouped into subfeatures by calling set_feature_lengths_class_train_dataset_feature_lengths_class_train_dataSetFeatureLengthsClassTrainDataSetFeatureLengthsClassTrainDataSetFeatureLengthsClassTrainDataset_feature_lengths_class_train_data. A subfeature can contain several subsequent elements of a feature vector. The operator decides for each of these subfeatures, if it is better to use it for the classification or leave it out.

The indices of the selected subfeatures are returned in SelectedFeatureIndicesSelectedFeatureIndicesSelectedFeatureIndicesSelectedFeatureIndicesselectedFeatureIndicesselected_feature_indices. If names were set in set_feature_lengths_class_train_dataset_feature_lengths_class_train_dataSetFeatureLengthsClassTrainDataSetFeatureLengthsClassTrainDataSetFeatureLengthsClassTrainDataset_feature_lengths_class_train_data, these names are returned instead of the indices. If set_feature_lengths_class_train_dataset_feature_lengths_class_train_dataSetFeatureLengthsClassTrainDataSetFeatureLengthsClassTrainDataSetFeatureLengthsClassTrainDataset_feature_lengths_class_train_data was not called for ClassTrainDataHandleClassTrainDataHandleClassTrainDataHandleClassTrainDataHandleclassTrainDataHandleclass_train_data_handle before, each element of the feature vector is considered as a subfeature.

The selection method SelectionMethodSelectionMethodSelectionMethodSelectionMethodselectionMethodselection_method is either a greedy search 'greedy'"greedy""greedy""greedy""greedy""greedy" (iteratively add the feature with highest gain) or the dynamically oscillating search 'greedy_oscillating'"greedy_oscillating""greedy_oscillating""greedy_oscillating""greedy_oscillating""greedy_oscillating" (add the feature with highest gain and test then if any of the already added features can be left out without great loss). The method 'greedy'"greedy""greedy""greedy""greedy""greedy" is generally preferable, since it is faster. Only in cases when the subfeatures are low-dimensional or redundant, the method 'greedy_oscillating'"greedy_oscillating""greedy_oscillating""greedy_oscillating""greedy_oscillating""greedy_oscillating" should be chosen.

The optimization criterion is the classification rate of a two-fold cross-validation of the training data. The best achieved value is returned in ScoreScoreScoreScorescorescore.

The k-NN classifier can be parameterized using the following values in GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name and GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value:

'num_neighbors'"num_neighbors""num_neighbors""num_neighbors""num_neighbors""num_neighbors":

The number of minimally evaluated nodes, increase this value for high dimensional data.

Possible values: '1'"1""1""1""1""1", '2'"2""2""2""2""2", '5'"5""5""5""5""5", '10'"10""10""10""10""10"

Default value: '1'"1""1""1""1""1"

'num_trees'"num_trees""num_trees""num_trees""num_trees""num_trees":

Number of search trees in the k-NN classifier

Possible values: '1'"1""1""1""1""1", '4'"4""4""4""4""4", '10'"10""10""10""10""10"

Default value: '4'"4""4""4""4""4"

Attention

This operator may take considerable time, depending on the size of the data set in the training file, and the number of features.

Please note, that this operator should not be called, if only a small set of training data is available. Due to the risk of overfitting the operator select_feature_set_knnselect_feature_set_knnSelectFeatureSetKnnSelectFeatureSetKnnSelectFeatureSetKnnselect_feature_set_knn may deliver a classifier with a very high score. However, the classifier may perform poorly when tested.

Execution Information

This operator returns a handle. Note that the state of an instance of this handle type may be changed by specific operators even though the handle is used as an input parameter by those operators.

Parameters

ClassTrainDataHandleClassTrainDataHandleClassTrainDataHandleClassTrainDataHandleclassTrainDataHandleclass_train_data_handle (input_control)  class_train_data HClassTrainData, HTupleHHandleHTupleHtuple (handle) (IntPtr) (HHandle) (handle)

Handle of the training data.

SelectionMethodSelectionMethodSelectionMethodSelectionMethodselectionMethodselection_method (input_control)  string HTuplestrHTupleHtuple (string) (string) (HString) (char*)

Method to perform the selection.

Default value: 'greedy' "greedy" "greedy" "greedy" "greedy" "greedy"

List of values: 'greedy'"greedy""greedy""greedy""greedy""greedy", 'greedy_oscillating'"greedy_oscillating""greedy_oscillating""greedy_oscillating""greedy_oscillating""greedy_oscillating"

GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name (input_control)  string(-array) HTupleMaybeSequence[str]HTupleHtuple (string) (string) (HString) (char*)

Names of generic parameters to configure the selection process and the classifier.

Default value: []

List of values: 'num_neighbors'"num_neighbors""num_neighbors""num_neighbors""num_neighbors""num_neighbors", 'num_trees'"num_trees""num_trees""num_trees""num_trees""num_trees"

GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value (input_control)  number(-array) HTupleMaybeSequence[Union[int, str, float]]HTupleHtuple (real / integer / string) (double / int / long / string) (double / Hlong / HString) (double / Hlong / char*)

Values of generic parameters to configure the selection process and the classifier.

Default value: []

Suggested values: 1, 2, 3

KNNHandleKNNHandleKNNHandleKNNHandleKNNHandleknnhandle (output_control)  class_knn HClassKnn, HTupleHHandleHTupleHtuple (handle) (IntPtr) (HHandle) (handle)

A trained k-NN classifier using only the selected features.

SelectedFeatureIndicesSelectedFeatureIndicesSelectedFeatureIndicesSelectedFeatureIndicesselectedFeatureIndicesselected_feature_indices (output_control)  string-array HTupleSequence[str]HTupleHtuple (string) (string) (HString) (char*)

The selected feature set, contains indices or names.

ScoreScoreScoreScorescorescore (output_control)  real-array HTupleSequence[float]HTupleHtuple (real) (double) (double) (double)

The achieved score using two-fold cross-validation.

Example (HDevelop)

* Find out which of the two features distinguishes two Classes
NameFeature1 := 'Good Feature'
NameFeature2 := 'Bad Feature'
LengthFeature1 := 3
LengthFeature2 := 2
* Create training data
create_class_train_data (LengthFeature1+LengthFeature2,\
  ClassTrainDataHandle)
* Define the features which are in the training data
set_feature_lengths_class_train_data (ClassTrainDataHandle, [LengthFeature1,\
  LengthFeature2], [NameFeature1, NameFeature2])
* Add training data
*                                                         |Feat1| |Feat2|
add_sample_class_train_data (ClassTrainDataHandle, 'row', [1,1,1,  2,1  ], 0)
add_sample_class_train_data (ClassTrainDataHandle, 'row', [2,2,2,  2,1  ], 1)
add_sample_class_train_data (ClassTrainDataHandle, 'row', [1,1,1,  3,4  ], 0)
add_sample_class_train_data (ClassTrainDataHandle, 'row', [2,2,2,  3,4  ], 1)
add_sample_class_train_data (ClassTrainDataHandle, 'row', [0,0,1,  5,6  ], 0)
add_sample_class_train_data (ClassTrainDataHandle, 'row', [2,3,2,  5,6  ], 1)
* Add more data
* ...
* Select the better feature with the k-NN classifier
select_feature_set_knn (ClassTrainDataHandle, 'greedy', [], [], KNNHandle,\
  SelectedFeatureKNN, Score)
* Use the classifier
* ...

Result

If the parameters are valid, the operator select_feature_set_knnselect_feature_set_knnSelectFeatureSetKnnSelectFeatureSetKnnSelectFeatureSetKnnselect_feature_set_knn returns the value 2 (H_MSG_TRUE). If necessary, an exception is raised.

Possible Predecessors

create_class_train_datacreate_class_train_dataCreateClassTrainDataCreateClassTrainDataCreateClassTrainDatacreate_class_train_data, add_sample_class_train_dataadd_sample_class_train_dataAddSampleClassTrainDataAddSampleClassTrainDataAddSampleClassTrainDataadd_sample_class_train_data, set_feature_lengths_class_train_dataset_feature_lengths_class_train_dataSetFeatureLengthsClassTrainDataSetFeatureLengthsClassTrainDataSetFeatureLengthsClassTrainDataset_feature_lengths_class_train_data

Possible Successors

classify_class_knnclassify_class_knnClassifyClassKnnClassifyClassKnnClassifyClassKnnclassify_class_knn

Alternatives

select_feature_set_mlpselect_feature_set_mlpSelectFeatureSetMlpSelectFeatureSetMlpSelectFeatureSetMlpselect_feature_set_mlp, select_feature_set_svmselect_feature_set_svmSelectFeatureSetSvmSelectFeatureSetSvmSelectFeatureSetSvmselect_feature_set_svm, select_feature_set_gmmselect_feature_set_gmmSelectFeatureSetGmmSelectFeatureSetGmmSelectFeatureSetGmmselect_feature_set_gmm

See also

select_feature_set_trainf_knnselect_feature_set_trainf_knnSelectFeatureSetTrainfKnnSelectFeatureSetTrainfKnnSelectFeatureSetTrainfKnnselect_feature_set_trainf_knn, gray_featuresgray_featuresGrayFeaturesGrayFeaturesGrayFeaturesgray_features, region_featuresregion_featuresRegionFeaturesRegionFeaturesRegionFeaturesregion_features

Module

Foundation