NonlinearMinimizer

class NonlinearMinimizer(proximal: Proximal, maxIters: Int, innerIters: Int, bfgsMemory: Int, rho: Double, alpha: Double, abstol: Double, reltol: Double) extends SerializableLogging

NonlinearMinimizer solves the problem that has the following structure minimize f(x) + g(x)

g(x) represents the following constraints

  1. x >= 0
  2. lb <= x <= ub
  3. L1(x)
  4. Aeq*x = beq
  5. aeq'x = beq
  6. 1'x = s, x >= 0 ProbabilitySimplex from the reference Proximal Algorithms by Boyd et al, Duchi et al

f(x) can be a smooth convex function defined by DiffFunction or a proximal operator. For now the exposed API takes DiffFunction

g(x) is defined by a proximal operator

For proximal algorithms like L1 through soft-thresholding and Huber Loss (look into library of proximal algorithms for further details) we provide ADMM based Proximal algorithm based on the following reference: https://web.stanford.edu/~boyd/papers/admm/logreg-l1/distr_l1_logreg.html

A subset of proximal operators are projection operators. For projection, NonlinearMinimizer companion object provides project API which generates a Spectral Projected Gradient (SPG) or Projected Quasi Newton (PQN) solver. For projection operators like positivity, bounds, probability simplex etc, these algorithms converges faster as compared to ADMM based proximal algorithm.

TO DO

  1. Implement FISTA / Nesterov's accelerated method and compare with ADMM
  2. For non-convex function experiment with TRON-like Primal solver
Companion:
object
trait Serializable
class Object
trait Matchable
class Any

Type members

Classlikes

case class State

Value members

Concrete methods

def iterations(primal: DiffFunction[BDV], init: BDV): Iterator[State]
def minimize(primal: DiffFunction[BDV], init: BDV): BDV

Inherited methods

protected def logger: LazyLogger
Inherited from:
SerializableLogging

Concrete fields