org.deeplearning4j.optimize.solvers.accumulation.encoding.threshold

Class AdaptiveThresholdAlgorithm

java.lang.Object

org.deeplearning4j.optimize.solvers.accumulation.encoding.threshold.AdaptiveThresholdAlgorithm

All Implemented Interfaces:

Serializable, ThresholdAlgorithm

public class AdaptiveThresholdAlgorithm
extends Object
implements ThresholdAlgorithm

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static double	DEFAULT_DECAY_RATE
static double	DEFAULT_INITIAL_THRESHOLD
static double	DEFAULT_MAX_SPARSITY_TARGET
static double	DEFAULT_MIN_SPARSITY_TARGET

Constructor Summary

Constructors

Constructor and Description
AdaptiveThresholdAlgorithm() Create the adaptive threshold algorithm with the default initial threshold DEFAULT_INITIAL_THRESHOLD, default minimum sparsity target DEFAULT_MIN_SPARSITY_TARGET, default maximum sparsity target DEFAULT_MAX_SPARSITY_TARGET, and default decay rate DEFAULT_DECAY_RATE
AdaptiveThresholdAlgorithm(double initialThreshold) Create the adaptive threshold algorithm with the specified initial threshold, but defaults for the other values: default minimum sparsity target DEFAULT_MIN_SPARSITY_TARGET, default maximum sparsity target DEFAULT_MAX_SPARSITY_TARGET, and default decay rate DEFAULT_DECAY_RATE
AdaptiveThresholdAlgorithm(double initialThreshold, double minTargetSparsity, double maxTargetSparsity, double decayRate)

Method Summary

Modifier and Type	Method and Description
double	calculateThreshold(int iteration, int epoch, Double lastThreshold, Boolean lastWasDense, Double lastSparsityRatio, INDArray updatesPlusResidual)
AdaptiveThresholdAlgorithm	clone()
ThresholdAlgorithmReducer	newReducer() Create a new ThresholdAlgorithmReducer.
String	toString()

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_INITIAL_THRESHOLD

public static final double DEFAULT_INITIAL_THRESHOLD

See Also:

Constant Field Values

DEFAULT_MIN_SPARSITY_TARGET

public static final double DEFAULT_MIN_SPARSITY_TARGET

See Also:

Constant Field Values

DEFAULT_MAX_SPARSITY_TARGET

public static final double DEFAULT_MAX_SPARSITY_TARGET

See Also:

Constant Field Values

DEFAULT_DECAY_RATE

public static final double DEFAULT_DECAY_RATE

Constructor Detail

AdaptiveThresholdAlgorithm

public AdaptiveThresholdAlgorithm()

Create the adaptive threshold algorithm with the default initial threshold DEFAULT_INITIAL_THRESHOLD, default minimum sparsity target DEFAULT_MIN_SPARSITY_TARGET, default maximum sparsity target DEFAULT_MAX_SPARSITY_TARGET, and default decay rate DEFAULT_DECAY_RATE

AdaptiveThresholdAlgorithm

public AdaptiveThresholdAlgorithm(double initialThreshold)

Create the adaptive threshold algorithm with the specified initial threshold, but defaults for the other values: default minimum sparsity target DEFAULT_MIN_SPARSITY_TARGET, default maximum sparsity target DEFAULT_MAX_SPARSITY_TARGET, and default decay rate DEFAULT_DECAY_RATE

AdaptiveThresholdAlgorithm

public AdaptiveThresholdAlgorithm(double initialThreshold,
                                  double minTargetSparsity,
                                  double maxTargetSparsity,
                                  double decayRate)

Parameters:

initialThreshold - The initial threshold to use

minTargetSparsity - The minimum target sparsity ratio - for example 1e-4

maxTargetSparsity - The maximum target sparsity ratio - for example 1e-2

decayRate - The decay rate. For example 0.95

Method Detail

calculateThreshold

public double calculateThreshold(int iteration,
                                 int epoch,
                                 Double lastThreshold,
                                 Boolean lastWasDense,
                                 Double lastSparsityRatio,
                                 INDArray updatesPlusResidual)

Specified by:

calculateThreshold in interface ThresholdAlgorithm

Parameters:

iteration - Current neural network training iteration

epoch - Current neural network training epoch

lastThreshold - The encoding threshold used in the last iteration - if available. May be null for first iteration in an epoch (where no 'last iteration' value is available)

lastWasDense - Whether the last encoding was dense (true) or sparse (false). May be null for the first iteration in an epoch (where no 'last iteration' value is available)

lastSparsityRatio - The sparsity ratio of the last iteration. Sparsity ratio is defined as numElements(encoded)/length(updates). A sparsity ratio of 1.0 would mean all entries present in encoded representation; a sparsity ratio of 0.0 would mean the encoded vector did not contain any values. Note: when the last encoding was dense, lastSparsityRatio is always null - this means that the sparsity ratio is larger than 1/16 = 0.0625

updatesPlusResidual - The actual array (updates plus residual) that will be encoded using the threshold calculated/returned by this method

Returns:

newReducer

public ThresholdAlgorithmReducer newReducer()

Description copied from interface: ThresholdAlgorithm

Create a new ThresholdAlgorithmReducer. Note that implementations should NOT add the curret ThresholdAlgorithm to it.

Specified by:

newReducer in interface ThresholdAlgorithm

Returns:

ThresholdAlgorithmReducer

clone

public AdaptiveThresholdAlgorithm clone()

Specified by:

clone in interface ThresholdAlgorithm

Overrides:

clone in class Object

Returns:

A clone of the current threshold algorithm

toString

public String toString()

Overrides:

toString in class Object