SVM

Description

The Support Vector Machine (SVM) classifier is a powerful classifier that works well on a wide range of classification problems, particularly on more complex problems that other classifiers (such as the KNN, GMM or ANBC algorithms).

The SVM class provides a front end to the LIBSVM library.

The SVM algorithm is part of the GRT classification modules.

Advantages

The Support Vector Machine (SVM) classifier is a powerful classifier that works well on a wide range of classification problems, even problems in high dimensions and that are not linearly separable.

Disadvantages

The main disadvantage of the SVM algorithm is that it has several key parameters that need to be set correctly to achieve the best classification results for any given problem. Parameters that may result in an excellent classification accuracy for problem A, may result in a poor classification accuracy for problem B. The user may, therefore, have to experiment with a number of different parameter settings in order to achieve a satisfactory result. The main parameters that the user should experiment with are the SVM kernel type (which can be set by the setKernelType(UINT kernelType) method), the SVM type (which can be set by the setSVMType(UINT svmType) method), and the kernel-specific parameters (such as gamma, degree, nu, etc.).

Training Data Format

You should use the LabelledClassificationData data structure to train the SVM classifier.

Example Code

This examples demonstrates how to initialize, train, and use the SVM algorithm for classification. The example loads the data shown in the image below and uses this to train the SVM algorithm. The data is a recording of a Wii-mote being held in 5 different orientations, the top graph shows the raw accelerometer data from the recording (showing the x, y, and z accelerometer data), while the bottom graph shows the label recorded for each sample (you can see the 5 different classes in the label data). You can download the actual dataset in the Code & Resources section below.

SVM Training Data
The data is a recording of a Wii-mote being held in 5 different orientations, the top graph shows the raw accelerometer data from the recording (showing the x, y, and z accelerometer data), while the bottom graph shows the label recorded for each sample (you can see the 5 different classes in the label data). WiiAccelerometerData.jpg
/*
 GRT SVM Example
 This examples demonstrates how to initialize, train, and use the SVM algorithm for classification.

 The Support Vector Machine (SVM) classifier is a powerful classifier that works well on a wide range of classification problems, even problems in high dimensions and that are not linearly separable.

 In this example we create an instance of a SVM algorithm and then train the algorithm using some pre-recorded training data.
 The trained SVM algorithm is then used to predict the class label of some test data.

 This example shows you how to:
 - Create an initialize the SVM algorithm using a LINEAR kernel
 - Load some LabelledClassificationData from a file and partition the training data into a training dataset and a test dataset
 - Train the SVM algorithm using the training dataset
 - Test the SVM algorithm using the test dataset
 - Manually compute the accuracy of the classifier
*/


//You might need to set the specific path of the GRT header relative to your project
#include "GRT.h"
using namespace GRT;

int main (int argc, const char * argv[])
{
    //Create a new SVM classifier with a linear kernel
    //Other kernel options you could choose are: POLY_KERNEL, RBF_KERNEL, SIGMOID_KERNEL, PRECOMPUTED_KERNEL
    SVM svm(SVM::LINEAR_KERNEL);

    //The SVM will typically work much better if we scale the training and prediction data, so turn scaling on
    svm.enableScaling( true );

    //Train the classifier with some training data
    LabelledClassificationData trainingData;

    if( !trainingData.loadDatasetFromFile("SVMTrainingData.txt") ){
        cout << "Failed to load training data!\n";
        return EXIT_FAILURE;
    }

    //Use 20% of the training dataset to create a test dataset
    LabelledClassificationData testData = trainingData.partition( 80 );

    //Train the classifier
    if( !svm.train( trainingData ) ){
        cout << "Failed to train classifier!\n";
        return EXIT_FAILURE;
    }

    //Save the svm model to a file
    if( !svm.saveModelToFile("SVMModel.txt") ){
        cout << "Failed to save the classifier model!\n";
        return EXIT_FAILURE;
    }

    //Load the knn model from a file
    if( !svm.loadModelFromFile("SVMModel.txt") ){
        cout << "Failed to load the classifier model!\n";
        return EXIT_FAILURE;
    }

    //Use the test dataset to test the SVM model
    double accuracy = 0;
    for(UINT i=0; i<testData.getNumSamples(); i++){
        //Get the i'th test sample
        UINT classLabel = testData[i].getClassLabel();
        vector< double > inputVector = testData[i].getSample();

        //Perform a prediction using the classifier
        bool predictSuccess = svm.predict( inputVector );

        if( !predictSuccess ){
            cout << "Failed to perform prediction for test sampel: " << i <<"\n";
            return EXIT_FAILURE;
        }

        //Get the predicted class label
        UINT predictedClassLabel = svm.getPredictedClassLabel();
        vector< double > classLikelihoods = svm.getClassLikelihoods();
        vector< double > classDistances = svm.getClassDistances();

        //Update the accuracy
        if( classLabel == predictedClassLabel ) accuracy++;

        cout << "TestSample: " << i <<  " ClassLabel: " << classLabel << " PredictedClassLabel: " << predictedClassLabel << endl;
    }

    cout << "Test Accuracy: " << accuracy/double(testData.getNumSamples())*100.0 << "%" << endl;

    return EXIT_SUCCESS;
}

Code & Resources

SVMExample.cpp SVMTrainingData.txt

Documentation

You can find the documentation for this class at SVM documentation.