apple.metalperformanceshaders

Class MPSNNOptimizerAdam

java.lang.Object

org.moe.natj.general.NativeObject

org.moe.natj.objc.ObjCObject

apple.NSObject

apple.metalperformanceshaders.MPSKernel

apple.metalperformanceshaders.MPSNNOptimizer

apple.metalperformanceshaders.MPSNNOptimizerAdam

All Implemented Interfaces:

NSCoding, NSCopying, NSSecureCoding, NSObject

public class MPSNNOptimizerAdam
extends MPSNNOptimizer

MPSNNOptimizerAdam

The MPSNNOptimizerAdam performs an Adam Update

Initialization time m[0] = 0 (Initialize initial 1st moment vector aka momentum, user is responsible for this) v[0] = 0 (Initialize initial 2nd moment vector aka velocity, user is responsible for this) t = 0 (Initialize timestep)

https://arxiv.org/abs/1412.6980

At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t)

m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)

where, g is gradient of error wrt variable v[t] is velocity m[t] is momentum

Nested Class Summary

Nested classes/interfaces inherited from class apple.NSObject

NSObject.Function_instanceMethodForSelector_ret, NSObject.Function_methodForSelector_ret

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	MPSNNOptimizerAdam(org.moe.natj.general.Pointer peer)

Method Summary

Modifier and Type	Method and Description
boolean	_supportsSecureCoding() This property must return YES on all classes that allow secure coding.
static boolean	accessInstanceVariablesDirectly()
static MPSNNOptimizerAdam	alloc()
static MPSNNOptimizerAdam	allocWithZone(org.moe.natj.general.ptr.VoidPtr zone)
static boolean	automaticallyNotifiesObserversForKey(java.lang.String key)
double	beta1() [@property] beta1
double	beta2() [@property] beta2
static void	cancelPreviousPerformRequestsWithTarget(java.lang.Object aTarget)
static void	cancelPreviousPerformRequestsWithTargetSelectorObject(java.lang.Object aTarget, org.moe.natj.objc.SEL aSelector, java.lang.Object anArgument)
static NSArray<java.lang.String>	classFallbacksForKeyedArchiver()
static org.moe.natj.objc.Class	classForKeyedUnarchiver()
static java.lang.String	debugDescription_static()
static java.lang.String	description_static()
void	encodeToCommandBufferBatchNormalizationGradientStateBatchNormalizationSourceStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNBatchNormalizationState batchNormalizationGradientState, MPSCNNBatchNormalizationState batchNormalizationSourceState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, NSArray<? extends MPSVector> maximumVelocityVectors, MPSCNNNormalizationGammaAndBetaState resultState) Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update
void	encodeToCommandBufferBatchNormalizationGradientStateBatchNormalizationSourceStateInputMomentumVectorsInputVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNBatchNormalizationState batchNormalizationGradientState, MPSCNNBatchNormalizationState batchNormalizationSourceState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, MPSCNNNormalizationGammaAndBetaState resultState) Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update
void	encodeToCommandBufferBatchNormalizationStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNBatchNormalizationState batchNormalizationState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, NSArray<? extends MPSVector> maximumVelocityVectors, MPSCNNNormalizationGammaAndBetaState resultState) Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update
void	encodeToCommandBufferBatchNormalizationStateInputMomentumVectorsInputVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNBatchNormalizationState batchNormalizationState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, MPSCNNNormalizationGammaAndBetaState resultState) Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update
void	encodeToCommandBufferConvolutionGradientStateConvolutionSourceStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNConvolutionGradientState convolutionGradientState, MPSCNNConvolutionWeightsAndBiasesState convolutionSourceState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, NSArray<? extends MPSVector> maximumVelocityVectors, MPSCNNConvolutionWeightsAndBiasesState resultState) Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update
void	encodeToCommandBufferConvolutionGradientStateConvolutionSourceStateInputMomentumVectorsInputVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNConvolutionGradientState convolutionGradientState, MPSCNNConvolutionWeightsAndBiasesState convolutionSourceState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, MPSCNNConvolutionWeightsAndBiasesState resultState) Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update
void	encodeToCommandBufferInputGradientMatrixInputValuesMatrixInputMomentumMatrixInputVelocityMatrixMaximumVelocityMatrixResultValuesMatrix(MTLCommandBuffer commandBuffer, MPSMatrix inputGradientMatrix, MPSMatrix inputValuesMatrix, MPSMatrix inputMomentumMatrix, MPSMatrix inputVelocityMatrix, MPSMatrix maximumVelocityMatrix, MPSMatrix resultValuesMatrix)
void	encodeToCommandBufferInputGradientMatrixInputValuesMatrixInputMomentumMatrixInputVelocityMatrixResultValuesMatrix(MTLCommandBuffer commandBuffer, MPSMatrix inputGradientMatrix, MPSMatrix inputValuesMatrix, MPSMatrix inputMomentumMatrix, MPSMatrix inputVelocityMatrix, MPSMatrix resultValuesMatrix)
void	encodeToCommandBufferInputGradientVectorInputValuesVectorInputMomentumVectorInputVelocityVectorMaximumVelocityVectorResultValuesVector(MTLCommandBuffer commandBuffer, MPSVector inputGradientVector, MPSVector inputValuesVector, MPSVector inputMomentumVector, MPSVector inputVelocityVector, MPSVector maximumVelocityVector, MPSVector resultValuesVector) Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update
void	encodeToCommandBufferInputGradientVectorInputValuesVectorInputMomentumVectorInputVelocityVectorResultValuesVector(MTLCommandBuffer commandBuffer, MPSVector inputGradientVector, MPSVector inputValuesVector, MPSVector inputMomentumVector, MPSVector inputVelocityVector, MPSVector resultValuesVector) Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update
float	epsilon() [@property] epsilon
static long	hash_static()
MPSNNOptimizerAdam	init()
MPSNNOptimizerAdam	initWithCoder(NSCoder aDecoder) NS_DESIGNATED_INITIALIZER
MPSNNOptimizerAdam	initWithCoderDevice(NSCoder aDecoder, java.lang.Object device) NSSecureCoding compatability
MPSNNOptimizerAdam	initWithDevice(java.lang.Object device) Standard init with default properties per filter type
MPSNNOptimizerAdam	initWithDeviceBeta1Beta2EpsilonTimeStepOptimizerDescriptor(MTLDevice device, double beta1, double beta2, float epsilon, long timeStep, MPSNNOptimizerDescriptor optimizerDescriptor) Full initialization for the adam update
MPSNNOptimizerAdam	initWithDeviceLearningRate(MTLDevice device, float learningRate) Convenience initialization for the adam update
static NSObject.Function_instanceMethodForSelector_ret	instanceMethodForSelector(org.moe.natj.objc.SEL aSelector)
static NSMethodSignature	instanceMethodSignatureForSelector(org.moe.natj.objc.SEL aSelector)
static boolean	instancesRespondToSelector(org.moe.natj.objc.SEL aSelector)
static boolean	isSubclassOfClass(org.moe.natj.objc.Class aClass)
static NSSet<java.lang.String>	keyPathsForValuesAffectingValueForKey(java.lang.String key)
static MPSNNOptimizerAdam	new_objc()
static boolean	resolveClassMethod(org.moe.natj.objc.SEL sel)
static boolean	resolveInstanceMethod(org.moe.natj.objc.SEL sel)
void	setTimeStep(long value) [@property] timeStep
static void	setVersion_static(long aVersion)
static org.moe.natj.objc.Class	superclass_static()
static boolean	supportsSecureCoding()
long	timeStep() [@property] timeStep
static long	version_static()

Methods inherited from class apple.metalperformanceshaders.MPSNNOptimizer

applyGradientClipping, gradientClipMax, gradientClipMin, gradientRescale, learningRate, regularizationScale, regularizationType, setApplyGradientClipping, setLearningRate

Methods inherited from class apple.metalperformanceshaders.MPSKernel

copyWithZone, copyWithZoneDevice, device, encodeWithCoder, label, options, setLabel, setOptions

Methods inherited from class apple.NSObject

accessibilityActivate, accessibilityActivationPoint, accessibilityAssistiveTechnologyFocusedIdentifiers, accessibilityAttributedHint, accessibilityAttributedLabel, accessibilityAttributedUserInputLabels, accessibilityAttributedValue, accessibilityContainerType, accessibilityCustomActions, accessibilityCustomRotors, accessibilityDecrement, accessibilityDragSourceDescriptors, accessibilityDropPointDescriptors, accessibilityElementAtIndex, accessibilityElementCount, accessibilityElementDidBecomeFocused, accessibilityElementDidLoseFocus, accessibilityElementIsFocused, accessibilityElements, accessibilityElementsHidden, accessibilityFrame, accessibilityHint, accessibilityIncrement, accessibilityLabel, accessibilityLanguage, accessibilityNavigationStyle, accessibilityPath, accessibilityPerformEscape, accessibilityPerformMagicTap, accessibilityRespondsToUserInteraction, accessibilityScroll, accessibilityTextualContext, accessibilityTraits, accessibilityUserInputLabels, accessibilityValue, accessibilityViewIsModal, addObserverForKeyPathOptionsContext, attemptRecoveryFromErrorOptionIndex, attemptRecoveryFromErrorOptionIndexDelegateDidRecoverSelectorContextInfo, autoContentAccessingProxy, awakeAfterUsingCoder, awakeFromNib, class_objc, classForCoder, classForKeyedArchiver, copy, dealloc, debugDescription, description, dictionaryWithValuesForKeys, didChangeValueForKey, didChangeValueForKeyWithSetMutationUsingObjects, didChangeValuesAtIndexesForKey, doesNotRecognizeSelector, fileManagerShouldProceedAfterError, fileManagerWillProcessPath, finalize_objc, forwardingTargetForSelector, forwardInvocation, hash, indexOfAccessibilityElement, isAccessibilityElement, isEqual, isKindOfClass, isMemberOfClass, isProxy, methodForSelector, methodSignatureForSelector, mutableArrayValueForKey, mutableArrayValueForKeyPath, mutableCopy, mutableOrderedSetValueForKey, mutableOrderedSetValueForKeyPath, mutableSetValueForKey, mutableSetValueForKeyPath, observationInfo, observeValueForKeyPathOfObjectChangeContext, performSelector, performSelectorInBackgroundWithObject, performSelectorOnMainThreadWithObjectWaitUntilDone, performSelectorOnMainThreadWithObjectWaitUntilDoneModes, performSelectorOnThreadWithObjectWaitUntilDone, performSelectorOnThreadWithObjectWaitUntilDoneModes, performSelectorWithObject, performSelectorWithObjectAfterDelay, performSelectorWithObjectAfterDelayInModes, performSelectorWithObjectWithObject, prepareForInterfaceBuilder, provideImageDataBytesPerRowOrigin_Size_UserInfo, removeObserverForKeyPath, removeObserverForKeyPathContext, replacementObjectForCoder, replacementObjectForKeyedArchiver, respondsToSelector, self, setAccessibilityActivationPoint, setAccessibilityAttributedHint, setAccessibilityAttributedLabel, setAccessibilityAttributedUserInputLabels, setAccessibilityAttributedValue, setAccessibilityContainerType, setAccessibilityCustomActions, setAccessibilityCustomRotors, setAccessibilityDragSourceDescriptors, setAccessibilityDropPointDescriptors, setAccessibilityElements, setAccessibilityElementsHidden, setAccessibilityFrame, setAccessibilityHint, setAccessibilityLabel, setAccessibilityLanguage, setAccessibilityNavigationStyle, setAccessibilityPath, setAccessibilityRespondsToUserInteraction, setAccessibilityTextualContext, setAccessibilityTraits, setAccessibilityUserInputLabels, setAccessibilityValue, setAccessibilityViewIsModal, setIsAccessibilityElement, setNilValueForKey, setObservationInfo, setShouldGroupAccessibilityChildren, setValueForKey, setValueForKeyPath, setValueForUndefinedKey, setValuesForKeysWithDictionary, shouldGroupAccessibilityChildren, superclass, validateValueForKeyError, validateValueForKeyPathError, valueForKey, valueForKeyPath, valueForUndefinedKey, willChangeValueForKey, willChangeValueForKeyWithSetMutationUsingObjects, willChangeValuesAtIndexesForKey

Methods inherited from class org.moe.natj.objc.ObjCObject

equals, toString

Methods inherited from class org.moe.natj.general.NativeObject

finalize, getPeer, getPeerPointer

Methods inherited from class java.lang.Object

clone, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

MPSNNOptimizerAdam

protected MPSNNOptimizerAdam(org.moe.natj.general.Pointer peer)

Method Detail

accessInstanceVariablesDirectly

public static boolean accessInstanceVariablesDirectly()

alloc

public static MPSNNOptimizerAdam alloc()

allocWithZone

public static MPSNNOptimizerAdam allocWithZone(org.moe.natj.general.ptr.VoidPtr zone)

automaticallyNotifiesObserversForKey

public static boolean automaticallyNotifiesObserversForKey(java.lang.String key)

beta1

public double beta1()

[@property] beta1

The beta1 at which we update values

Default value is 0.9

beta2

public double beta2()

[@property] beta2

The beta2 at which we update values

Default value is 0.999

cancelPreviousPerformRequestsWithTarget

public static void cancelPreviousPerformRequestsWithTarget(java.lang.Object aTarget)

cancelPreviousPerformRequestsWithTargetSelectorObject

public static void cancelPreviousPerformRequestsWithTargetSelectorObject(java.lang.Object aTarget,
                                                                         org.moe.natj.objc.SEL aSelector,
                                                                         java.lang.Object anArgument)

classFallbacksForKeyedArchiver

public static NSArray<java.lang.String> classFallbacksForKeyedArchiver()

classForKeyedUnarchiver

public static org.moe.natj.objc.Class classForKeyedUnarchiver()

debugDescription_static

public static java.lang.String debugDescription_static()

description_static

public static java.lang.String description_static()

encodeToCommandBufferBatchNormalizationGradientStateBatchNormalizationSourceStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState

public void encodeToCommandBufferBatchNormalizationGradientStateBatchNormalizationSourceStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState(MTLCommandBuffer commandBuffer,
                                                                                                                                                                       MPSCNNBatchNormalizationState batchNormalizationGradientState,
                                                                                                                                                                       MPSCNNBatchNormalizationState batchNormalizationSourceState,
                                                                                                                                                                       NSArray<? extends MPSVector> inputMomentumVectors,
                                                                                                                                                                       NSArray<? extends MPSVector> inputVelocityVectors,
                                                                                                                                                                       NSArray<? extends MPSVector> maximumVelocityVectors,
                                                                                                                                                                       MPSCNNNormalizationGammaAndBetaState resultState)

Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update

The following operations would be applied At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t)

m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) maxVel[t] = max(maxVel[t-1],v[t]) variable = variable - lr[t] * m[t] / (sqrt(maxVel[t]) + epsilon)

Parameters:

commandBuffer - A valid MTLCommandBuffer to receive the encoded kernel.

batchNormalizationGradientState - A valid MPSCNNBatchNormalizationState object which specifies the input state with gradients for this update.

batchNormalizationSourceState - A valid MPSCNNBatchNormalizationState object which specifies the input state with original gamma/beta for this update.

inputMomentumVectors - An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updated

inputVelocityVectors - An array MPSVector object which specifies the gradient velocity vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updated

maximumVelocityVectors - An array MPSVector object which specifies the maximum velocity vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updated. May be nil, if nil then normal Adam optimizer behaviour is followed.

resultState - A valid MPSCNNNormalizationGammaAndBetaState object which specifies the resultValues state which will be updated and overwritten.

encodeToCommandBufferBatchNormalizationGradientStateBatchNormalizationSourceStateInputMomentumVectorsInputVelocityVectorsResultState

public void encodeToCommandBufferBatchNormalizationGradientStateBatchNormalizationSourceStateInputMomentumVectorsInputVelocityVectorsResultState(MTLCommandBuffer commandBuffer,
                                                                                                                                                 MPSCNNBatchNormalizationState batchNormalizationGradientState,
                                                                                                                                                 MPSCNNBatchNormalizationState batchNormalizationSourceState,
                                                                                                                                                 NSArray<? extends MPSVector> inputMomentumVectors,
                                                                                                                                                 NSArray<? extends MPSVector> inputVelocityVectors,
                                                                                                                                                 MPSCNNNormalizationGammaAndBetaState resultState)

Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update

The following operations would be applied

t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t)

m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)

Parameters:

commandBuffer - A valid MTLCommandBuffer to receive the encoded kernel.

batchNormalizationGradientState - A valid MPSCNNBatchNormalizationState object which specifies the input state with gradients for this update.

batchNormalizationSourceState - A valid MPSCNNBatchNormalizationState object which specifies the input state with original gamma/beta for this update.

inputMomentumVectors - An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updated

inputVelocityVectors - An array MPSVector object which specifies the gradient velocity vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updated

resultState - A valid MPSCNNNormalizationGammaAndBetaState object which specifies the resultValues state which will be updated and overwritten.

encodeToCommandBufferBatchNormalizationStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState

public void encodeToCommandBufferBatchNormalizationStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState(MTLCommandBuffer commandBuffer,
                                                                                                                                  MPSCNNBatchNormalizationState batchNormalizationState,
                                                                                                                                  NSArray<? extends MPSVector> inputMomentumVectors,
                                                                                                                                  NSArray<? extends MPSVector> inputVelocityVectors,
                                                                                                                                  NSArray<? extends MPSVector> maximumVelocityVectors,
                                                                                                                                  MPSCNNNormalizationGammaAndBetaState resultState)

Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update

The following operations would be applied At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t)

m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) maxVel[t] = max(maxVel[t-1],v[t]) variable = variable - lr[t] * m[t] / (sqrt(maxVel[t]) + epsilon)

Parameters:

commandBuffer - A valid MTLCommandBuffer to receive the encoded kernel.

batchNormalizationState - A valid MPSCNNBatchNormalizationState object which specifies the input state with gradients and original gamma/beta for this update.

inputMomentumVectors - An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updated

inputVelocityVectors - An array MPSVector object which specifies the gradient velocity vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updated

maximumVelocityVectors - An array MPSVector object which specifies the maximum velocity vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updated. May be nil, if nil then normal Adam optimizer behaviour is followed.

resultState - A valid MPSCNNNormalizationGammaAndBetaState object which specifies the resultValues state which will be updated and overwritten.

encodeToCommandBufferBatchNormalizationStateInputMomentumVectorsInputVelocityVectorsResultState

public void encodeToCommandBufferBatchNormalizationStateInputMomentumVectorsInputVelocityVectorsResultState(MTLCommandBuffer commandBuffer,
                                                                                                            MPSCNNBatchNormalizationState batchNormalizationState,
                                                                                                            NSArray<? extends MPSVector> inputMomentumVectors,
                                                                                                            NSArray<? extends MPSVector> inputVelocityVectors,
                                                                                                            MPSCNNNormalizationGammaAndBetaState resultState)

Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update

The following operations would be applied

t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t)

m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)

Parameters:

commandBuffer - A valid MTLCommandBuffer to receive the encoded kernel.

batchNormalizationState - A valid MPSCNNBatchNormalizationState object which specifies the input state with gradients and original gamma/beta for this update.

inputMomentumVectors - An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updated

inputVelocityVectors - An array MPSVector object which specifies the gradient velocity vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updated

resultState - A valid MPSCNNNormalizationGammaAndBetaState object which specifies the resultValues state which will be updated and overwritten.

encodeToCommandBufferConvolutionGradientStateConvolutionSourceStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState

public void encodeToCommandBufferConvolutionGradientStateConvolutionSourceStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState(MTLCommandBuffer commandBuffer,
                                                                                                                                                         MPSCNNConvolutionGradientState convolutionGradientState,
                                                                                                                                                         MPSCNNConvolutionWeightsAndBiasesState convolutionSourceState,
                                                                                                                                                         NSArray<? extends MPSVector> inputMomentumVectors,
                                                                                                                                                         NSArray<? extends MPSVector> inputVelocityVectors,
                                                                                                                                                         NSArray<? extends MPSVector> maximumVelocityVectors,
                                                                                                                                                         MPSCNNConvolutionWeightsAndBiasesState resultState)

Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update

The following operations would be applied At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t)

m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) maxVel[t] = max(maxVel[t-1],v[t]) variable = variable - lr[t] * m[t] / (sqrt(maxVel[t]) + epsilon)

Parameters:

commandBuffer - A valid MTLCommandBuffer to receive the encoded kernel.

convolutionGradientState - A valid MPSCNNConvolutionGradientState object which specifies the input state with gradients for this update.

convolutionSourceState - A valid MPSCNNConvolutionWeightsAndBiasesState object which specifies the input state with values to be updated.

inputMomentumVectors - An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updated

inputVelocityVectors - An array MPSVector object which specifies the gradient velocity vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updated

maximumVelocityVectors - An array MPSVector object which specifies the maximum velocity vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updated. May be nil, if nil then normal Adam optimizer behaviour is followed.

resultState - A valid MPSCNNConvolutionWeightsAndBiasesState object which specifies the resultValues state which will be updated and overwritten.

encodeToCommandBufferConvolutionGradientStateConvolutionSourceStateInputMomentumVectorsInputVelocityVectorsResultState

public void encodeToCommandBufferConvolutionGradientStateConvolutionSourceStateInputMomentumVectorsInputVelocityVectorsResultState(MTLCommandBuffer commandBuffer,
                                                                                                                                   MPSCNNConvolutionGradientState convolutionGradientState,
                                                                                                                                   MPSCNNConvolutionWeightsAndBiasesState convolutionSourceState,
                                                                                                                                   NSArray<? extends MPSVector> inputMomentumVectors,
                                                                                                                                   NSArray<? extends MPSVector> inputVelocityVectors,
                                                                                                                                   MPSCNNConvolutionWeightsAndBiasesState resultState)

Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update

The following operations would be applied

t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t)

m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)

Parameters:

commandBuffer - A valid MTLCommandBuffer to receive the encoded kernel.

convolutionGradientState - A valid MPSCNNConvolutionGradientState object which specifies the input state with gradients for this update.

convolutionSourceState - A valid MPSCNNConvolutionWeightsAndBiasesState object which specifies the input state with values to be updated.

inputMomentumVectors - An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updated

inputVelocityVectors - An array MPSVector object which specifies the gradient velocity vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updated

resultState - A valid MPSCNNConvolutionWeightsAndBiasesState object which specifies the resultValues state which will be updated and overwritten.

encodeToCommandBufferInputGradientMatrixInputValuesMatrixInputMomentumMatrixInputVelocityMatrixMaximumVelocityMatrixResultValuesMatrix

public void encodeToCommandBufferInputGradientMatrixInputValuesMatrixInputMomentumMatrixInputVelocityMatrixMaximumVelocityMatrixResultValuesMatrix(MTLCommandBuffer commandBuffer,
                                                                                                                                                   MPSMatrix inputGradientMatrix,
                                                                                                                                                   MPSMatrix inputValuesMatrix,
                                                                                                                                                   MPSMatrix inputMomentumMatrix,
                                                                                                                                                   MPSMatrix inputVelocityMatrix,
                                                                                                                                                   MPSMatrix maximumVelocityMatrix,
                                                                                                                                                   MPSMatrix resultValuesMatrix)

encodeToCommandBufferInputGradientMatrixInputValuesMatrixInputMomentumMatrixInputVelocityMatrixResultValuesMatrix

public void encodeToCommandBufferInputGradientMatrixInputValuesMatrixInputMomentumMatrixInputVelocityMatrixResultValuesMatrix(MTLCommandBuffer commandBuffer,
                                                                                                                              MPSMatrix inputGradientMatrix,
                                                                                                                              MPSMatrix inputValuesMatrix,
                                                                                                                              MPSMatrix inputMomentumMatrix,
                                                                                                                              MPSMatrix inputVelocityMatrix,
                                                                                                                              MPSMatrix resultValuesMatrix)

encodeToCommandBufferInputGradientVectorInputValuesVectorInputMomentumVectorInputVelocityVectorMaximumVelocityVectorResultValuesVector

public void encodeToCommandBufferInputGradientVectorInputValuesVectorInputMomentumVectorInputVelocityVectorMaximumVelocityVectorResultValuesVector(MTLCommandBuffer commandBuffer,
                                                                                                                                                   MPSVector inputGradientVector,
                                                                                                                                                   MPSVector inputValuesVector,
                                                                                                                                                   MPSVector inputMomentumVector,
                                                                                                                                                   MPSVector inputVelocityVector,
                                                                                                                                                   MPSVector maximumVelocityVector,
                                                                                                                                                   MPSVector resultValuesVector)

Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update

The following operations would be applied At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t)

m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) maxVel[t] = max(maxVel[t-1],v[t]) variable = variable - lr[t] * m[t] / (sqrt(maxVel[t]) + epsilon)

Parameters:

commandBuffer - A valid MTLCommandBuffer to receive the encoded kernel.

inputGradientVector - A valid MPSVector object which specifies the input vector of gradients for this update.

inputValuesVector - A valid MPSVector object which specifies the input vector of values to be updated.

inputMomentumVector - A valid MPSVector object which specifies the gradient momentum vector which will be updated and overwritten.

inputVelocityVector - A valid MPSVector object which specifies the gradient velocity vector which will be updated and overwritten.

maximumVelocityVector - A valid MPSVector object which specifies the maximum velocity vector which will be updated and overwritten. May be nil, if nil then normal Adam optimizer behaviour is followed.

resultValuesVector - A valid MPSCNNConvolutionWeightsAndBiasesState object which specifies the resultValues state which will be updated and overwritten.

encodeToCommandBufferInputGradientVectorInputValuesVectorInputMomentumVectorInputVelocityVectorResultValuesVector

public void encodeToCommandBufferInputGradientVectorInputValuesVectorInputMomentumVectorInputVelocityVectorResultValuesVector(MTLCommandBuffer commandBuffer,
                                                                                                                              MPSVector inputGradientVector,
                                                                                                                              MPSVector inputValuesVector,
                                                                                                                              MPSVector inputMomentumVector,
                                                                                                                              MPSVector inputVelocityVector,
                                                                                                                              MPSVector resultValuesVector)

Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update

The following operations would be applied

t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t)

m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)

Parameters:

commandBuffer - A valid MTLCommandBuffer to receive the encoded kernel.

inputGradientVector - A valid MPSVector object which specifies the input vector of gradients for this update.

inputValuesVector - A valid MPSVector object which specifies the input vector of values to be updated.

inputMomentumVector - A valid MPSVector object which specifies the gradient momentum vector which will be updated and overwritten.

inputVelocityVector - A valid MPSVector object which specifies the gradient velocity vector which will be updated and overwritten.

resultValuesVector - A valid MPSVector object which specifies the resultValues vector which will be updated and overwritten.

epsilon

public float epsilon()

[@property] epsilon

The epsilon at which we update values

This value is usually used to ensure to avoid divide by 0, default value is 1e-8

hash_static

public static long hash_static()

init

public MPSNNOptimizerAdam init()

Overrides:

init in class MPSNNOptimizer

initWithCoder

public MPSNNOptimizerAdam initWithCoder(NSCoder aDecoder)

Description copied from interface: NSCoding

NS_DESIGNATED_INITIALIZER

Specified by:

initWithCoder in interface NSCoding

Overrides:

initWithCoder in class MPSNNOptimizer

initWithCoderDevice

public MPSNNOptimizerAdam initWithCoderDevice(NSCoder aDecoder,
                                              java.lang.Object device)

Description copied from class: MPSKernel

NSSecureCoding compatability

While the standard NSSecureCoding/NSCoding method -initWithCoder: should work, since the file can't know which device your data is allocated on, we have to guess and may guess incorrectly. To avoid that problem, use initWithCoder:device instead.

Overrides:

initWithCoderDevice in class MPSNNOptimizer

Parameters:

aDecoder - The NSCoder subclass with your serialized MPSKernel

device - The MTLDevice on which to make the MPSKernel

Returns:

A new MPSKernel object, or nil if failure.

initWithDevice

public MPSNNOptimizerAdam initWithDevice(java.lang.Object device)

Description copied from class: MPSKernel

Standard init with default properties per filter type

Overrides:

initWithDevice in class MPSNNOptimizer

Parameters:

device - The device that the filter will be used on. May not be NULL.

Returns:

a pointer to the newly initialized object. This will fail, returning nil if the device is not supported. Devices must be MTLFeatureSet_iOS_GPUFamily2_v1 or later.

initWithDeviceBeta1Beta2EpsilonTimeStepOptimizerDescriptor

public MPSNNOptimizerAdam initWithDeviceBeta1Beta2EpsilonTimeStepOptimizerDescriptor(MTLDevice device,
                                                                                     double beta1,
                                                                                     double beta2,
                                                                                     float epsilon,
                                                                                     long timeStep,
                                                                                     MPSNNOptimizerDescriptor optimizerDescriptor)

Full initialization for the adam update

Parameters:

device - The device on which the kernel will execute.

beta1 - The beta1 to update values

beta2 - The beta2 to update values

epsilon - The epsilon at which we update values

timeStep - The timeStep at which values will start updating

optimizerDescriptor - The optimizerDescriptor which will have a bunch of properties to be applied

Returns:

A valid MPSNNOptimizerAdam object or nil, if failure.

initWithDeviceLearningRate

public MPSNNOptimizerAdam initWithDeviceLearningRate(MTLDevice device,
                                                     float learningRate)

Convenience initialization for the adam update

Parameters:

device - The device on which the kernel will execute.

learningRate - The learningRate at which we will update values

Returns:

A valid MPSNNOptimizerAdam object or nil, if failure.

instanceMethodForSelector

public static NSObject.Function_instanceMethodForSelector_ret instanceMethodForSelector(org.moe.natj.objc.SEL aSelector)

instanceMethodSignatureForSelector

public static NSMethodSignature instanceMethodSignatureForSelector(org.moe.natj.objc.SEL aSelector)

instancesRespondToSelector

public static boolean instancesRespondToSelector(org.moe.natj.objc.SEL aSelector)

isSubclassOfClass

public static boolean isSubclassOfClass(org.moe.natj.objc.Class aClass)

keyPathsForValuesAffectingValueForKey

public static NSSet<java.lang.String> keyPathsForValuesAffectingValueForKey(java.lang.String key)

new_objc

public static MPSNNOptimizerAdam new_objc()

resolveClassMethod

public static boolean resolveClassMethod(org.moe.natj.objc.SEL sel)

resolveInstanceMethod

public static boolean resolveInstanceMethod(org.moe.natj.objc.SEL sel)

setTimeStep

public void setTimeStep(long value)

[@property] timeStep

Current timeStep for the update, number of times update has occurred

setVersion_static

public static void setVersion_static(long aVersion)

superclass_static

public static org.moe.natj.objc.Class superclass_static()

supportsSecureCoding

public static boolean supportsSecureCoding()

_supportsSecureCoding

public boolean _supportsSecureCoding()

Description copied from interface: NSSecureCoding

This property must return YES on all classes that allow secure coding. Subclasses of classes that adopt NSSecureCoding and override initWithCoder: must also override this method and return YES. The Secure Coding Guide should be consulted when writing methods that decode data.

Specified by:

_supportsSecureCoding in interface NSSecureCoding

Overrides:

_supportsSecureCoding in class MPSNNOptimizer

timeStep

public long timeStep()

[@property] timeStep

Current timeStep for the update, number of times update has occurred

version_static

public static long version_static()