org.openimaj.ml.linear.learner.matlib

Class MatlibBilinearSparseOnlineLearner

java.lang.Object

org.openimaj.ml.linear.learner.matlib.MatlibBilinearSparseOnlineLearner

All Implemented Interfaces:

org.openimaj.io.ReadableBinary, org.openimaj.io.ReadWriteableBinary, org.openimaj.io.WriteableBinary, OnlineLearner<ch.akuhn.matrix.Matrix,ch.akuhn.matrix.Matrix>

public class MatlibBilinearSparseOnlineLearner
extends Object
implements OnlineLearner<ch.akuhn.matrix.Matrix,ch.akuhn.matrix.Matrix>, org.openimaj.io.ReadWriteableBinary

An implementation of a stochastic gradient decent with proximal perameter adjustment (for regularised parameters). Data is dealt with sequentially using a one pass implementation of the online proximal algorithm described in chapter 9 and 10 of: The Geometry of Constrained Structured Prediction: Applications to Inference and Learning of Natural Language Syntax, PhD, Andre T. Martins The implementation does the following: - When an X,Y is recieved: - Update currently held batch - If the batch is full: - While There is a great deal of change in U and W: - Calculate the gradient of W holding U fixed - Proximal update of W - Calculate the gradient of U holding W fixed - Proximal update of U - Calculate the gradient of Bias holding U and W fixed - flush the batch - return current U and W (same as last time is batch isn't filled yet)

Author:

Sina Samangooei (ss@ecs.soton.ac.uk)

Field Summary

Fields

Modifier and Type	Field and Description
protected ch.akuhn.matrix.Matrix	bias
protected Boolean	biasMode
protected ch.akuhn.matrix.Matrix	diagX
protected Double	eta0_u
protected Double	eta0_w
protected Double	lambda_u
protected Double	lambda_w
protected LossFunction	loss
protected BilinearLearnerParameters	params
protected Regulariser	regul
protected ch.akuhn.matrix.Matrix	u
protected ch.akuhn.matrix.Matrix	w

Constructor Summary

Constructors

Constructor and Description
MatlibBilinearSparseOnlineLearner() The default parameters.
MatlibBilinearSparseOnlineLearner(BilinearLearnerParameters params)

Method Summary

Modifier and Type	Method and Description
void	addU(int newUsers) Expand the U parameters matrix by added a set of rows.
void	addW(int newWords) Expand the W parameters matrix by added a set of rows.
byte[]	binaryHeader()
MatlibBilinearSparseOnlineLearner	clone()
static ch.akuhn.matrix.SparseMatrix	expandY(ch.akuhn.matrix.Matrix Y) Given a flat value matrix, makes a diagonal sparse matrix containing the values as the diagonal
ch.akuhn.matrix.Matrix	getBias()
BilinearLearnerParameters	getParams()
ch.akuhn.matrix.Matrix	getU()
ch.akuhn.matrix.Matrix	getW()
ch.akuhn.matrix.Matrix	predict(ch.akuhn.matrix.Matrix x)
void	process(ch.akuhn.matrix.Matrix X, ch.akuhn.matrix.Matrix Y)
void	readBinary(DataInput in)
void	reinitParams() must be called if any parameters are changed
void	setU(ch.akuhn.matrix.Matrix newu)
void	setW(ch.akuhn.matrix.Matrix neww)
protected ch.akuhn.matrix.Matrix	updateBias(ch.akuhn.matrix.Matrix biasGrad, double biasLossWeight)
protected ch.akuhn.matrix.Matrix	updateU(ch.akuhn.matrix.Matrix currentU, double uLossWeight, double uWeightedLambda)
protected ch.akuhn.matrix.Matrix	updateW(ch.akuhn.matrix.Matrix currentW, double wLossWeighted, double weightedLambda)
void	writeBinary(DataOutput out)

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

params

protected BilinearLearnerParameters params

w

protected ch.akuhn.matrix.Matrix w

u

protected ch.akuhn.matrix.Matrix u

loss

protected LossFunction loss

regul

protected Regulariser regul

lambda_w

protected Double lambda_w

lambda_u

protected Double lambda_u

biasMode

protected Boolean biasMode

bias

protected ch.akuhn.matrix.Matrix bias

diagX

protected ch.akuhn.matrix.Matrix diagX

eta0_u

protected Double eta0_u

eta0_w

protected Double eta0_w

Constructor Detail

MatlibBilinearSparseOnlineLearner

public MatlibBilinearSparseOnlineLearner()

The default parameters. These won't work with your dataset, i promise.

MatlibBilinearSparseOnlineLearner

public MatlibBilinearSparseOnlineLearner(BilinearLearnerParameters params)

Parameters:

params - the parameters used by this learner

Method Detail

reinitParams

public void reinitParams()

must be called if any parameters are changed

process

public void process(ch.akuhn.matrix.Matrix X,
                    ch.akuhn.matrix.Matrix Y)

Specified by:

process in interface OnlineLearner<ch.akuhn.matrix.Matrix,ch.akuhn.matrix.Matrix>

updateBias

protected ch.akuhn.matrix.Matrix updateBias(ch.akuhn.matrix.Matrix biasGrad,
                                            double biasLossWeight)

updateW

protected ch.akuhn.matrix.Matrix updateW(ch.akuhn.matrix.Matrix currentW,
                                         double wLossWeighted,
                                         double weightedLambda)

updateU

protected ch.akuhn.matrix.Matrix updateU(ch.akuhn.matrix.Matrix currentU,
                                         double uLossWeight,
                                         double uWeightedLambda)

expandY

public static ch.akuhn.matrix.SparseMatrix expandY(ch.akuhn.matrix.Matrix Y)

Given a flat value matrix, makes a diagonal sparse matrix containing the values as the diagonal

Parameters:

Y -

Returns:

the diagonalised Y

getParams

public BilinearLearnerParameters getParams()

Returns:

the current apramters

getU

public ch.akuhn.matrix.Matrix getU()

Returns:

the current user matrix

getW

public ch.akuhn.matrix.Matrix getW()

Returns:

the current word matrix

getBias

public ch.akuhn.matrix.Matrix getBias()

Returns:

the current bias (null if BilinearLearnerParameters.BIAS is false

addU

public void addU(int newUsers)

Expand the U parameters matrix by added a set of rows. If currently unset, this function does nothing (assuming U will be initialised in the first round) The new U parameters are initialised used BilinearLearnerParameters.EXPANDEDUINITSTRAT

Parameters:

newUsers - the number of new users to add

addW

public void addW(int newWords)

Expand the W parameters matrix by added a set of rows. If currently unset, this function does nothing (assuming W will be initialised in the first round) The new W parameters are initialised used BilinearLearnerParameters.EXPANDEDWINITSTRAT

Parameters:

newWords - the number of new words to add

clone

public MatlibBilinearSparseOnlineLearner clone()

Overrides:

clone in class Object

setU

public void setU(ch.akuhn.matrix.Matrix newu)

Parameters:

newu - set the model's U

setW

public void setW(ch.akuhn.matrix.Matrix neww)

Parameters:

neww - set the model's W

readBinary

public void readBinary(DataInput in)
                throws IOException

Specified by:

readBinary in interface org.openimaj.io.ReadableBinary

Throws:

IOException

binaryHeader

public byte[] binaryHeader()

Specified by:

binaryHeader in interface org.openimaj.io.ReadableBinary

Specified by:

binaryHeader in interface org.openimaj.io.WriteableBinary

writeBinary

public void writeBinary(DataOutput out)
                 throws IOException

Specified by:

writeBinary in interface org.openimaj.io.WriteableBinary

Throws:

IOException

predict

public ch.akuhn.matrix.Matrix predict(ch.akuhn.matrix.Matrix x)

Specified by:

predict in interface OnlineLearner<ch.akuhn.matrix.Matrix,ch.akuhn.matrix.Matrix>