Multidimensional Regression

Description

The Multidimensional Regression class acts as a meta-algorithm for regression that allows several one-dimensional regression algorithms (such as Linear Regression), to be combined together to allow an M-dimensional signal to be mapped to an N-dimensional signal. This works by training N separate regression algorithms (one for each dimension), each with an M-dimensional input.

Multidimensional Regression
Multidimensional Regression: Multidimensional Regression is a meta-algorithm that allows several one-dimensional regression algorithms (such as Linear Regression), to be combined together to allow an M-dimensional input signal to be mapped to an N-dimensional signal. This works by training N separate regression algorithms (one for each dimension), each with an M-dimensional input. In this example, a 3-dimensional signal is mapped to a 2-dimensional output signal using the Multidimensional Regression meta-algorithm. MultidimensionalRegression.png

In addition to enabling one-dimensional regression algorithms (such as Linear Regression) to be used for mapping N-dimensional output signals, Multidimensional Regression can also be useful for multi-dimensional regression algorithms (such as Multi Layer Perceptron), as it enables you to train N separate MLP algorithms (one for each output signal), which might provide better mapping results than trying to train one MLP algorithm that can successfully map all N-dimensions at once.

The Multidimensional Regression algorithm is a supervised learning algorithm that can be used for regression for any type of M-dimensional signal.

The Multidimensional Regression algorithm is part of the GRT regression modules.

Advantages

The Multidimensional Regression algorithm is a meta-algorithm for regression that can map an M-dimensional signal to a N-dimensional signal. You can use any of the GRT regression algorithms with Multidimensional Regression.

Disadvantages

The main limitation of the Multidimensional Regression algorithm is that, if you are mapping an M-dimensinal signal to an N-dimensional signal, then the Multidimensional Regression algorithm needs to train N independent regression algorithms.

Things To Know

You should always enable scaling with Multidimensional Regression, as this will give you much better results.

Training Data Format

You should use the LabelledRegressionData data structure to train the Multidimensional Regression algorithm.

Example Code

This examples demonstrates how to initialize, train, and use the Multidimensional Regression algorithm for regression.

In this example we create an instance of an Multidimensional Regression algorithm, using Linear Regression as the regression module and then train a regression model using some pre-recorded training data. The trained model is then used to perform regression on the test data.

Both the training and test datasets have 2 input dimensions and 3 target dimensions.

You can download the training and test datasets in the Code & Resources section below.

/*
 GRT Multidimensional Regression Example
 This examples demonstrates how to initialize, train, and use the Multidimensional Regression algorithm.

 This example shows you how to:
 - Create an initialize the Multidimensional Regression algorithm
 - Create a new instance of a GestureRecognitionPipeline and add the regression instance to the pipeline
 - Load some LabelledRegressionData from a file
 - Train the Multidimensional Regression algorithm using the training dataset
 - Test the Multidimensional Regression algorithm using the test dataset
*/


#include "GRT.h"
using namespace GRT;

int main (int argc, const char * argv[])
{
    //Turn on the training log so we can print the training status of the MultidimensionalRegression to the screen
    TrainingLog::enableLogging( true );

    //Load the training data
    LabelledRegressionData trainingData;
    LabelledRegressionData testData;

    if( !trainingData.loadDatasetFromFile("MultidimensionalRegressionTrainingData.txt") ){
        cout << "ERROR: Failed to load training data!\n";
        return EXIT_FAILURE;
    }

    if( !testData.loadDatasetFromFile("MultidimensionalRegressionTestData.txt") ){
        cout << "ERROR: Failed to load test data!\n";
        return EXIT_FAILURE;
    }

    //Make sure the dimensionality of the training and test data matches
    if( trainingData.getNumInputDimensions() != testData.getNumInputDimensions() ){
        cout << "ERROR: The number of input dimensions in the training data (" << trainingData.getNumInputDimensions() << ")";
        cout << " does not match the number of input dimensions in the test data (" << testData.getNumInputDimensions() << ")\n";
        return EXIT_FAILURE;
    }

    if( testData.getNumTargetDimensions() != testData.getNumTargetDimensions() ){
        cout << "ERROR: The number of target dimensions in the training data (" << testData.getNumTargetDimensions() << ")";
        cout << " does not match the number of target dimensions in the test data (" << testData.getNumTargetDimensions() << ")\n";
        return EXIT_FAILURE;
    }

    cout << "Training and Test datasets loaded\n";

    //Print the stats of the datasets
    cout << "Training data stats:\n";
    trainingData.printStats();

    cout << "Test data stats:\n";
    testData.printStats();

    //Create a new gesture recognition pipeline
    GestureRecognitionPipeline pipeline;

    //Add a MultidimensionalRegression instance to the pipeline and set the regression algorithm to LinearRegression
    pipeline.setRegressifier( MultidimensionalRegression( LinearRegression(), true ) );

    //Train the MultidimensionalRegression model, which will train 3 LinearRegression models
    cout << "Training MultidimensionalRegression model...\n";
    if( !pipeline.train( trainingData ) ){
        cout << "ERROR: Failed to train MultidimensionalRegression model!\n";
        return EXIT_FAILURE;
    }

    cout << "Model trained.\n";

    if( !pipeline.savePipelineToFile( "Pipeline" ) ){
        cout << "ERROR: Failed to save pipeline!\n";
        return EXIT_FAILURE;
    }

    if( !pipeline.loadPipelineFromFile( "Pipeline" ) ){
        cout << "ERROR: Failed to load pipeline!\n";
        return EXIT_FAILURE;
    }

    //Test the model
    cout << "Testing MultidimensionalRegression model...\n";
    if( !pipeline.test( testData ) ){
        cout << "ERROR: Failed to test MultidimensionalRegression model!\n";
        return EXIT_FAILURE;
    }

    cout << "Test complete. Test RMS error: " << pipeline.getTestRMSError() << endl;

    //Run back over the test data again and output the results to a file
    fstream file;
    file.open("MultidimensionalRegressionResultsData.txt", fstream::out);

    for(UINT i=0; i<testData.getNumSamples(); i++){
        vector< double > inputVector = testData[i].getInputVector();
        vector< double > targetVector = testData[i].getTargetVector();

        //Map the input vector using the trained regression model
        if( !pipeline.predict( inputVector ) ){
            cout << "ERROR: Failed to map test sample " << i << endl;
            return EXIT_FAILURE;
        }

        //Get the mapped regression data
        vector< double > outputVector = pipeline.getRegressionData();

        //Write the mapped value and also the target value to the file
        for(UINT j=0; j<outputVector.size(); j++){
            file << outputVector[j] << "\t";
        }
        for(UINT j=0; j<targetVector.size(); j++){
            file << targetVector[j] << "\t";
        }
        file << endl;
    }

    //Close the file
    file.close();

    return EXIT_SUCCESS;
}

Code & Resources

MultidimensionalRegressionExample.cpp MultidimensionalRegressionTrainingData.txt MultidimensionalRegressionTestData.txt

Documentation

You can find the documentation for this class at Multidimensional Regression documentation.