org.openimaj.ml.regression

Class LinearRegression

java.lang.Object

org.openimaj.ml.regression.LinearRegression

All Implemented Interfaces:

Cloneable, org.openimaj.math.model.EstimatableModel<double[],double[]>, org.openimaj.math.model.Model<double[],double[]>

public class LinearRegression
extends Object
implements org.openimaj.math.model.EstimatableModel<double[],double[]>

Given a set of independant variables a linear regressions finds the optimal vector B such that: (Y - Xb)^2 = 0 (Y - Xb)^{T}(Y-Xb) = 0 calculated by assuming a convex shape of (Y - Xb) with varying values of b (reasonable as the function is linear) and then calculating the point at which the first derivative of this function is 0. i.e.: d/db (y - Xb)^{T} (y - Xb) = -X^{T}(y - Xb) - X^{T}(y - Xb) = - 2 * X^{T}(y - Xb) which at the 0 is: - 2 * X^{T}(y - Xb) = 0 X^{T}(y - Xb) X^{T}y - X^{T}Xb = 0 X^{T}y = X^{T}Xb b = (X^{T} X)^{-1} X^{T} y Calculating this function directly behaves numerically badly when X is extremely skinny and tall (i.e. lots of data, fewer dimentions) so we calculate this using the SVD, using the SVD we can decompose X as: X = UDV^{T} s.t. U and V are orthonormal from this we can calculate: b = V D^{-1} U^{T} y which is equivilant but more numerically stable. Note that upon input any vector of independent variables x_n are automatically to turned into an n + 1 vector {1,x0,x1,...,xn} which handles the constant values added to y

Author:

Sina Samangooei (ss@ecs.soton.ac.uk)

Constructor Summary

Constructors

Constructor and Description
LinearRegression() linear regression model

Method Summary

Modifier and Type	Method and Description
LinearRegression	clone()
boolean	equals(Object obj)
void	estimate(double[][] yd, double[][] xd) As in estimate(List) but using double arrays for efficiency.
boolean	estimate(List<? extends org.openimaj.util.pair.IndependentPair<double[],double[]>> data)
void	estimate(Jama.Matrix y, Jama.Matrix x) As in estimate(List) but using double arrays for efficiency.
int	numItemsToEstimate()
double[]	predict(double[] data)
Jama.Matrix	predict(Jama.Matrix x) Helper function which adds the constant component to x and returns predicted values for y, one per row
String	toString()

Methods inherited from class java.lang.Object

finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

LinearRegression

public LinearRegression()

linear regression model

Method Detail

estimate

public boolean estimate(List<? extends org.openimaj.util.pair.IndependentPair<double[],double[]>> data)

Specified by:

estimate in interface org.openimaj.math.model.EstimatableModel<double[],double[]>

estimate

public void estimate(double[][] yd,
                     double[][] xd)

As in estimate(List) but using double arrays for efficiency.

Parameters:

yd -

xd -

estimate

public void estimate(Jama.Matrix y,
                     Jama.Matrix x)

As in estimate(List) but using double arrays for efficiency. Estimates: b = V D^{-1} U^{T} y s.t. X = UDV^{T}

Parameters:

y -

x -

predict

public double[] predict(double[] data)

Specified by:

predict in interface org.openimaj.math.model.Model<double[],double[]>

predict

public Jama.Matrix predict(Jama.Matrix x)

Helper function which adds the constant component to x and returns predicted values for y, one per row

Parameters:

x -

Returns:

predicted y

numItemsToEstimate

public int numItemsToEstimate()

Specified by:

numItemsToEstimate in interface org.openimaj.math.model.EstimatableModel<double[],double[]>

clone

public LinearRegression clone()

Specified by:

clone in interface org.openimaj.math.model.EstimatableModel<double[],double[]>

Overrides:

clone in class Object

equals

public boolean equals(Object obj)

Overrides:

equals in class Object

toString

public String toString()

Overrides:

toString in class Object