FirstOrderMinimizer

Companion:
class
class Object
trait Matchable
class Any

Type members

Classlikes

Companion:
class
Companion:
object
case class FunctionValuesConverged[T](tolerance: Double, relative: Boolean, historyLength: Int) extends ConvergenceCheck[T]
Companion:
object
Companion:
class
case object MaxIterations extends ConvergenceReason
case class MonitorFunctionValuesCheck[T](f: T => Double, numFailures: Int, improvementRequirement: Double, evalFrequency: Int) extends ConvergenceCheck[T] with SerializableLogging
case class OptParams(batchSize: Int, regularization: Double, alpha: Double, maxIterations: Int, useL1: Boolean, tolerance: Double, useStochastic: Boolean, randomSeed: Int)

OptParams is a Configuration-compatible case class that can be used to select optimization routines at runtime.

OptParams is a Configuration-compatible case class that can be used to select optimization routines at runtime.

Configurations:

  1. useStochastic=false,useL1=false: LBFGS with L2 regularization
  2. useStochastic=false,useL1=true: OWLQN with L1 regularization
  3. useStochastic=true,useL1=false: AdaptiveGradientDescent with L2 regularization
  4. useStochastic=true,useL1=true: AdaptiveGradientDescent with L1 regularization
Value parameters:
alpha

rate of change to use, only applies to SGD.

batchSize

size of batches to use if useStochastic and you give a BatchDiffFunction

maxIterations,

how many iterations to do.

regularization

regularization constant to use.

tolerance

convergence tolerance, looking at both average improvement and the norm of the gradient.

useL1

if true, use L1 regularization. Otherwise, use L2.

useStochastic

if false, use LBFGS or OWLQN. If true, use some variant of Stochastic Gradient Descent.

case object SearchFailed extends ConvergenceReason
case class State[+T, +ConvergenceInfo, +History](x: T, value: Double, grad: T, adjustedValue: Double, adjustedGradient: T, iter: Int, initialAdjVal: Double, history: History, convergenceInfo: ConvergenceInfo, searchFailed: Boolean, var convergenceReason: Option[ConvergenceReason])

Tracks the information about the optimizer, including the current point, its value, gradient, and then any history. Also includes information for checking convergence.

Tracks the information about the optimizer, including the current point, its value, gradient, and then any history. Also includes information for checking convergence.

Value parameters:
adjustedGradient

f'(x) + r'(x), where r is any regularization added to the objective. For LBFGS, this is f'(x).

adjustedValue

f(x) + r(x), where r is any regularization added to the objective. For LBFGS, this is f(x).

convergenceReason

the convergence reason

grad

f.gradientAt(x)

history

any information needed by the optimizer to do updates.

initialAdjVal

f(x_0) + r(x_0), used for checking convergence

iter

what iteration number we are on.

searchFailed

did the line search fail?

value

f(x)

x

the current point being considered

Value members

Concrete methods

def defaultConvergenceCheck[T](maxIter: Int, tolerance: Double, relative: Boolean, fvalMemory: Int)(implicit space: NormedModule[T, Double]): ConvergenceCheck[T]
def functionValuesConverged[T](tolerance: Double, relative: Boolean, historyLength: Int): ConvergenceCheck[T]
def gradientConverged[T](tolerance: Double, relative: Boolean)(implicit space: NormedModule[T, Double]): ConvergenceCheck[T]
def monitorFunctionValues[T](f: T => Double, numFailures: Int, improvementRequirement: Double, evalFrequency: Int): ConvergenceCheck[T]

Runs the function, and if it fails to decreased by at least improvementRequirement numFailures times in a row, then we abort

Runs the function, and if it fails to decreased by at least improvementRequirement numFailures times in a row, then we abort

Value parameters:
evalFrequency

how often we run the evaluation