Class Label Timeout Filter

Description

The Class Label Timeout Filter is a useful post-processing module which debounces a gesture (i.e. it stops a single gesture from being recognized multiple times over a short time frame). For instance, it is normally the case that whenever a user performs a gesture, such as a swipe gesture for example, that the recognition system may recognize this single gesture several times because the user's movements are being sensed at a high sample rate (i.e. 100Hz). The Class Label Timeout Filter can be used to ensure that a gesture, such as the previous swipe gesture example, is only recognize once within any given timespan.

The Class Label Timeout Filter module is controlled through two parameters: the timeoutDuration parameter and filterMode parameter. The timeoutDuration sets how long (in milliseconds) the Class Label Timeout Filter will debounce valid predicted class labels for, after it has viewed the first valid predicted class label which triggers the debounce mode to be activated. A valid predicted class label is simply a predicted class label that is not the default null rejection class label (i.e. a label with the class value of 0). The filterMode parameter sets how the Class Label Timeout Filter reacts to different predicted class labels (different from the predicted class label that triggers the debounce mode to be activated). There are two options for the filterMode:

  • ALL_CLASS_LABELS
  • INDEPENDENT_CLASS_LABELS

In the ALL_CLASS_LABELS filterMode, after the debounce mode has been activated, all class labels will be ignored until the current timeoutDuration period has elapsed, regardless of which class actually triggered the timeout. Alternatively, in the INDEPENDENT_CLASS_LABELS mode, the debounce mode will be reset if a different predicted class label is detected that is different from the predicted class label that initially triggered the debounce mode to be activated. For instance, if the debounce mode was activated with the class label of 1, and then class 2 was input into the class label filter, then the debounce mode would be reset to class 2, even if the timeoutDuration for class 1 had not expired.

The timeoutDuration and filterMode parameters can be set using the setTimeoutDuration(double timeoutDuration) and setFilterMode(UINT filterMode) methods.

The Class Label Timeout Filter module is part of the GRT post-processing modules.

Advantages

The Class Label Timeout Filter is a useful post-processing module for debouncing gestures in a continuous stream of data.

Disadvantages

The main limitation of the Class Label Timeout Filter is that the user must choose the timeoutDuration empirically, if this timeoutDuration is set too low then multiple gesture events may be triggered from a single gesture, alternatively if this parameter is set too high then valid gestures may be missed because the system is ignoring any new gestures because of a previously triggered gesture.

Example Code

This example demonstrates how to create a new gesture recognition pipeline and add an ANBC module for classification along with a Class Label Timeout Filter as a post-processing module. The ANBC classifier is trained with some dummy data (which consists of 3 simple classes generated from 3 different Gaussian distributions). Some test data is then used to demonstrate the result of using the Class Label Timeout Filter to post process the predicted class label from the ANBC algorithm. The test data contains a continuous stream of data from class 1, followed by class 2 and then class 3. The time between each prediction is set to 100 milliseconds, with the timeout for the Class Label Timeout Filter set to 1000 milliseconds. The image below illustrates the results of the example. You can download the actual example code and datasets in the Code & Resources section below.

Class Label Timeout Filter Results
The results of running the test dataset through the Class Label Timeout Filter (with a timeoutDuration of 1000 and a filterMode of ALL_CLASS_LABELS). The processed class label (i.e. the output of the Class Label Timeout Filter) can be seen in red with the unprocessed predicted class label (i.e. the output of the ANBC algorithm) shown in green. You can see how the Class Label Timeout Filter only outputs the class label every second, outputting the default null gesture class label of 0 all other times. This is due to the timeoutDuration value, which was set to 1000 milliseconds (i.e. 1 second). ClassLabelTimeoutFilterResultsExampleImage1.jpg
/*
 GRT Class Label Filter Example
 This examples demonstrates how to use the Class Label Filter to filter the predicted class label of any classification module
 using the gesture recognition pipeline.

 The Class Label Filter is a useful post-processing module which can remove erroneous or sporadic prediction spikes that may be
 made by a classifier on a continuous input stream of data.  For instance, imagine a classifier that correctly outputs the predicted
 class label of 1 for a large majority of the time that a user is performing gesture 1, but every so often (perhaps due to sensor noise),
 the classifier outputs the class label of 2.  In this instance the class label filter can be used to remove these sporadic prediction
 values, with the output of the class label filter in this instance being 1.

 In this example we create a new gesture recognition pipeline and add an ANBC module for classification along with a Class Label Filter
 as a post-processing module.  The ANBC classifier is then trained with some dummy data (which consists of 3 simple classes generated
 from 3 different Gaussian distributions).  Some test data is then used to demonstrate the result of using the Class Label Filter to
 post process the predicted class label from the ANBC algorithm.  The test data contains a continuous stream of data from class 1, with
 a few sporadic class 2 and 3 labels added randomly to the stream.

 This example shows you how to:
 - Create an initialize a GestureRecognitionPipeline and add an ANBC module and a ClassLabelFilter module
 - Load some LabelledClassificationData from a file and train the classifier
 - Use some test data to demonstrate the effect of using a Class Label Filter
 - Print the Processed and Unprocessed Predicted Class Label
 - Write these results to a file
 */


//You might need to set the specific path of the GRT header relative to your project
#include "GRT.h"
using namespace GRT;

int main (int argc, const char * argv[])
{
    //Create a new gesture recognition pipeline
    GestureRecognitionPipeline pipeline;

    //Add an ANBC module
    pipeline.setClassifier( ANBC() );

    //Add a ClassLabelTimeoutFilter as a post processing module with a timeout value of 1000 milliseconds and all class labels
    pipeline.addPostProcessingModule( ClassLabelTimeoutFilter(1000,ClassLabelTimeoutFilter::ALL_CLASS_LABELS) );

    //Load some training data to train and test the classifier
    LabelledClassificationData trainingData;
    LabelledClassificationData testData;

    if( !trainingData.loadDatasetFromFile("ClassLabelTimeoutFilterTrainingData.txt") ){
        cout << "Failed to load training data!\n";
        return EXIT_FAILURE;
    }

    if( !testData.loadDatasetFromFile("ClassLabelTimeoutFilterTestData.txt") ){
        cout << "Failed to load training data!\n";
        return EXIT_FAILURE;
    }

    //Train the classifier
    if( !pipeline.train( trainingData ) ){
        cout << "Failed to train classifier!\n";
        return EXIT_FAILURE;
    }

    //Use the test dataset to demonstrate the output of the ClassLabelFilter
    fstream file;
    file.open("ClassLabelTimeoutFilterResults.txt",fstream::out);

    if( !file.is_open() ){
        cout << "Failed to open file to save results!\n";
        return EXIT_FAILURE;
    }

    for(UINT i=0; i<testData.getNumSamples(); i++){
        vector< double > inputVector = testData[i].getSample();

        if( !pipeline.predict( inputVector ) ){
            cout << "Failed to perform prediction for test sampel: " << i <<"\n";
            return EXIT_FAILURE;
        }

        //Get the predicted class label (this will be the processed class label)
        UINT predictedClassLabel = pipeline.getPredictedClassLabel();

        //Get the unprocessed class label (i.e. the direct output of the classifier)
        UINT unprocessedClassLabel = pipeline.getUnProcessedPredictedClassLabel();

        //Save the results to a file
        file << predictedClassLabel << "\t" << unprocessedClassLabel << endl;

        //Also print the results to the screen
        cout << "Processed Class Label: \t" << predictedClassLabel << "\tUnprocessed Class Label: \t" << unprocessedClassLabel << endl;

        //Sleep for 100 millisecond
        Util::sleep( 100 );
    }

    file.close();

    return EXIT_SUCCESS;
}

Code & Resources

ClassLabelTimeoutFilterExample.cpp ClassLabelTimeoutFilterTrainingData.txt ClassLabelTimeoutFilterTestData.txt

Documentation

You can find the documentation for this class at ClassLabelTimeoutFilter documentation.