# Logistic Regression

- Description
- Advantages
- Disadvantages
- Things To Know
- Training Data Format
- Example Code
- Code & Resources
- Documentation

## Description

The Logistic Regression algorithm is a simple regression algorithm that can map an N-dimensional signal to a 1-dimensional signal.

The Logistic Regression algorithm is a supervised learning algorithm that can be used for regression for any type of *N*-dimensional signal. If you want to use Logistic Regression for classification, then you should use the specific classification version: Logistic Regression Classifier.

The Logistic Regression algorithm is part of the GRT regression modules.

## Advantages

The Logistic Regression algorithm is a simple regression algorithm that can map an N-dimensional signal to a 1-dimensional signal.

## Disadvantages

Logistic Regression can only map an N-dimensional signal to a 1-dimension signal. If you need a regression algorithm that can map an N-dimensional to an M-dimensional signal, then you should try the MLP regression algorithm instead.

## Things To Know

You should always enable scaling with Logistic Regression, as this will give you much better results.

## Training Data Format

You should use the RegressionData data structure to train the Logistic Regression algorithm.

## Example Code

This examples demonstrates how to initialize, train, and use the Logistic Regression algorithm for regression.

The example loads the data shown in the image below and uses this to train the Logistic Regression algorithm. The data consists of the 3-axis gyro data from a Wii-mote, which has been labelled with a 1-dimensional target value (i.e. the value the Logistic Regression model should output given the 3-dimensional gyro input). The purpose of this exercise is to see if the Logistic Regression algorithm can learn to map the values of the Pitch axis of the gyro between 0 and 1, without the Roll and Yaw values corrupting the mapped output value. The Wii-mote was rotated left (around the Pitch axis) and then several seconds of training data was recorded with the target label set to 0. During this time the Wii-mote was moved around the Roll and Yaw axes, but not around the Pitch axis (as much as possible). The label was then changed to 1 and the Wii-mote was rotated right (around the Pitch axis) and then several seconds of training data was record, again the Wii-mote was moved around the Roll and Yaw axes, but not around the Pitch axis. This process was then repeat again to record the test dataset.

The images below show the recorded gyro and target values for both the training and test datasets. In the training and test images you can see the raw gyro data in the top row (with red = Roll, green = Pitch, blue = Yaw) and the target values in the bottom row.

You can download the training and test datasets in the Code & Resources section below.

**Gyro Training Data:**The data consists of the 3-axis gyro data from a Wii-mote, which has been labelled with a 1-dimensional target value (i.e. the value the Logistic Regression model should output given the 3-dimensional gyro input). The raw gyro data is in the top row (with red = Roll, green = Pitch, blue = Yaw) and the target value is in the bottom row. LogisticRegressionTrainingDataImage1.jpg

**Gyro Test Data:**The data consists of the 3-axis gyro data from a Wii-mote, which has been labelled with a 1-dimensional target value (i.e. the value the Logistic Regression model should output given the 3-dimensional gyro input). The raw gyro data is in the top row (with red = Roll, green = Pitch, blue = Yaw) and the target value is in the bottom row. LogisticRegressionTestDataImage1.jpg

**Logistic Regression Results Data:**This image show the output of the trained Logistic Regression (in blue) along with the target value (in green) from the test data set. You can see that the Logistic Regression effectively learns to map the 3-axis gyro input to the 1 dimensional target output, although the 'noise' from the Roll and Yaw axes still have some influence on the mapping of the Pitch axis. The RMS error of this test data was 0.0638. LogisticRegressionOutputResultsImage1.jpg

Logistic Regression Example

This examples demonstrates how to initialize, train, and use the LogisticRegression class for regression.

Logistic Regression is a simple but powerful regression algorithm that can map an N-dimensional signal to a 1-dimensional signal.

In this example we create an instance of an LogisticRegression algorithm and then use the algorithm to train a model using some pre-recorded training data.

The trained model is then used to perform regression on the test data.

This example shows you how to:

- Create an initialize the LogisticRegression algorithm for regression

- Create a new instance of a GestureRecognitionPipeline and add the regression instance to the pipeline

- Load some RegressionData from a file

- Train a LogisticRegression model using the training dataset

- Test the LogisticRegression model using the test dataset

- Save the output of the LogisticRegression algorithm to a file

*/

#include "GRT.h"

using namespace GRT;

int main (int argc, const char * argv[])

{

//Turn on the training log so we can print the training status of the LogisticRegression to the screen

TrainingLog::enableLogging( true );

//Load the training data

RegressionData trainingData;

RegressionData testData;

if( !trainingData.loadDatasetFromFile("LogisticRegressionTrainingData.txt") ){

cout << "ERROR: Failed to load training data!\n";

return EXIT_FAILURE;

}

if( !testData.loadDatasetFromFile("LogisticRegressionTestData.txt") ){

cout << "ERROR: Failed to load test data!\n";

return EXIT_FAILURE;

}

//Make sure the dimensionality of the training and test data matches

if( trainingData.getNumInputDimensions() != testData.getNumInputDimensions() ){

cout << "ERROR: The number of input dimensions in the training data (" << trainingData.getNumInputDimensions() << ")";

cout << " does not match the number of input dimensions in the test data (" << testData.getNumInputDimensions() << ")\n";

return EXIT_FAILURE;

}

if( testData.getNumTargetDimensions() != testData.getNumTargetDimensions() ){

cout << "ERROR: The number of target dimensions in the training data (" << testData.getNumTargetDimensions() << ")";

cout << " does not match the number of target dimensions in the test data (" << testData.getNumTargetDimensions() << ")\n";

return EXIT_FAILURE;

}

cout << "Training and Test datasets loaded\n";

//Print the stats of the datasets

cout << "Training data stats:\n";

trainingData.printStats();

cout << "Test data stats:\n";

testData.printStats();

//Create a new gesture recognition pipeline

GestureRecognitionPipeline pipeline;

//Add a LogisticRegression instance to the pipeline

pipeline.setRegressifier( LogisticRegression() );

//Train the LogisticRegression model

cout << "Training LogisticRegression model...\n";

if( !pipeline.train( trainingData ) ){

cout << "ERROR: Failed to train LogisticRegression model!\n";

return EXIT_FAILURE;

}

cout << "Model trained.\n";

//Test the model

cout << "Testing LogisticRegression model...\n";

if( !pipeline.test( testData ) ){

cout << "ERROR: Failed to test LogisticRegression model!\n";

return EXIT_FAILURE;

}

cout << "Test complete. Test RMS error: " << pipeline.getTestRMSError() << endl;

//Run back over the test data again and output the results to a file

fstream file;

file.open("LogisticRegressionResultsData.txt", fstream::out);

for(UINT i=0; i<testData.getNumSamples(); i++){

vector< double > inputVector = testData[i].getInputVector();

vector< double > targetVector = testData[i].getTargetVector();

//Map the input vector using the trained regression model

if( !pipeline.predict( inputVector ) ){

cout << "ERROR: Failed to map test sample " << i << endl;

return EXIT_FAILURE;

}

//Get the mapped regression data

vector< double > outputVector = pipeline.getRegressionData();

//Write the mapped value and also the target value to the file

for(UINT j=0; j<outputVector.size(); j++){

file << outputVector[j] << "\t";

}

for(UINT j=0; j<targetVector.size(); j++){

file << targetVector[j] << "\t";

}

file << endl;

}

//Close the file

file.close();

return EXIT_SUCCESS;

}

## Code & Resources

LogisticRegressionExample.cpp LogisticRegressionTrainingData.txt LogisticRegressionTestData.txt

## Documentation

You can find the documentation for this class at Logistic Regression documentation.