breeze.optimize.proximal
Type members
Classlikes
Supported constraints by QuadraticMinimizer object
Supported constraints by QuadraticMinimizer object
NonlinearMinimizer solves the problem that has the following structure minimize f(x) + g(x)
NonlinearMinimizer solves the problem that has the following structure minimize f(x) + g(x)
g(x) represents the following constraints
- x >= 0
- lb <= x <= ub
- L1(x)
- Aeq*x = beq
- aeq'x = beq
- 1'x = s, x >= 0 ProbabilitySimplex from the reference Proximal Algorithms by Boyd et al, Duchi et al
f(x) can be a smooth convex function defined by DiffFunction or a proximal operator. For now the exposed API takes DiffFunction
g(x) is defined by a proximal operator
For proximal algorithms like L1 through soft-thresholding and Huber Loss (look into library of proximal algorithms for further details) we provide ADMM based Proximal algorithm based on the following reference: https://web.stanford.edu/~boyd/papers/admm/logreg-l1/distr_l1_logreg.html
A subset of proximal operators are projection operators. For projection, NonlinearMinimizer companion object provides project API which generates a Spectral Projected Gradient (SPG) or Projected Quasi Newton (PQN) solver. For projection operators like positivity, bounds, probability simplex etc, these algorithms converges faster as compared to ADMM based proximal algorithm.
TO DO
- Implement FISTA / Nesterov's accelerated method and compare with ADMM
- For non-convex function experiment with TRON-like Primal solver
- Companion:
- object
Projection formula from Duchi et al's paper Efficient Projections onto the l1-Ball for Learning in High Dimensions
Projection formula from Duchi et al's paper Efficient Projections onto the l1-Ball for Learning in High Dimensions
PDCO dense quadratic program generator
PDCO dense quadratic program generator
Reference
Generates random instances of Quadratic Programming Problems 0.5x'Px + q'x s.t Ax = b lb <= x <= ub
nGram rank of quadratic problems to be generated
- Returns:
A is the equality constraint
Proximal operators and ADMM based Primal-Dual QP Solver
Proximal operators and ADMM based Primal-Dual QP Solver
Reference: http://www.stanford.edu/~boyd/papers/admm/quadprog/quadprog.html
It solves problem that has the following structure
1/2 x'Hx + f'x + g(x) s.t Aeqx = b
g(x) represents the following constraints which covers ALS based matrix factorization use-cases
- x >= 0
- lb <= x <= ub
- L1(x)
- L2(x)
- Generic regularization on x
- Value parameters:
- Aeq
rhs matrix for equality constraints
- abstol
ADMM absolute tolerance
- alpha
over-relaxation parameter default 1.0 1.5 - 1.8 can improve convergence
- beq
lhs constants for equality constraints
- nGram
rank of dense gram matrix
- proximal
proximal operator to be used
- reltol
ADMM relative tolerance
- Companion:
- object