apple.metalperformanceshaders

Class MPSCNNGradientKernel

java.lang.Object

org.moe.natj.general.NativeObject

org.moe.natj.objc.ObjCObject

apple.NSObject

apple.metalperformanceshaders.MPSKernel

apple.metalperformanceshaders.MPSCNNBinaryKernel

apple.metalperformanceshaders.MPSCNNGradientKernel

All Implemented Interfaces:

NSCoding, NSCopying, NSSecureCoding, NSObject

Direct Known Subclasses:

MPSCNNArithmeticGradient, MPSCNNBatchNormalizationGradient, MPSCNNBatchNormalizationStatisticsGradient, MPSCNNConvolutionGradient, MPSCNNConvolutionTransposeGradient, MPSCNNCrossChannelNormalizationGradient, MPSCNNDropoutGradient, MPSCNNGroupNormalizationGradient, MPSCNNInstanceNormalizationGradient, MPSCNNLocalContrastNormalizationGradient, MPSCNNLogSoftMaxGradient, MPSCNNNeuronGradient, MPSCNNPoolingGradient, MPSCNNSoftMaxGradient, MPSCNNSpatialNormalizationGradient, MPSCNNUpsamplingGradient, MPSNNGramMatrixCalculationGradient, MPSNNPadGradient, MPSNNReshapeGradient

public class MPSCNNGradientKernel
extends MPSCNNBinaryKernel

MPSCNNGradientKernel

Gradient kernels are the backwards pass of a MPSCNNKernel used during training to calculate gradient back propagation. These take as arguments the gradient result from the next filter and the source image for the forward version of the filter. There is also a MPSNNGradientState passed from MPSCNNKernel to MPSCNNGradientKernel that contains information about the MPSCNNKernel parameters at the time it encoded and possibly also additional MTLResources to enable it to do its job.

[@code] Training graph (partial):

---> input image ---------> MPSCNNKernel ------> resultImage ------>-->-->-->. \ | | '------. MPSNNGradientState loss estimation \ | | V V V <--- result gradient <- MPSCNNGradientKernel <--- input gradient <--<--<--<---'

In general operation, starting with the input image, the sequence of events is: 1a) Invoke padding policy to find result size for MPSCNNKernel. This also configures some MPSCNNKernel parameters such as offset. 1b) Use the MPSImageDescriptor from 1a to make resultImage. 1c) Call MPSCNNKernel -encode... 2) stages 1a-c are repeated for other forward passes in the inference portion of the graph 3) We estimate the loss resulting from the whole inference computation so far (see MPSCNNLoss.h> 4) stages 5a-c are repeated for corresponding backward gradient passes in the graph 5a) Invoke padding policy on the MPSCNNGradientKernel shown above. This sets the MPSCNNGradientKernel parameters to correspond with those in the forward pass 5b) The result gradient for the MPSCNNGradientKernel is created from the MPSImageDescriptor from 5a 5c) Call MPSCNNGradientKernel -encode with the input image, input gradient, result gradient and MPSNNGradientState 6) pass the result gradient on to leftward gradient passes. [@endcode]

For MPSCNNKernels that are trained, there may be other accompanying training kernels that need to be called in addition to the gradient kernel to update convolution weights or batch normalization parameters, for example. Steps 1a-c and 5a-c can be combined in a single -encode call. These return the result image or gradient out the left hand side.

For purposes of inheritance the gradient image is the MPSCNNBinaryKernel primary image and the source image is the MPSCNNBinaryKernel secondary image. Various secondary properties such as kernel size are copies of the forward inference pass parameters of similar name are set automatically when -[MPSCNNGradientKernel destinationImageDescriptorForSourceImages:sourceStates:] is called.

Nested Class Summary

Nested classes/interfaces inherited from class apple.NSObject

NSObject.Function_instanceMethodForSelector_ret, NSObject.Function_methodForSelector_ret

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	MPSCNNGradientKernel(org.moe.natj.general.Pointer peer)

Method Summary

Modifier and Type	Method and Description
boolean	_supportsSecureCoding() This property must return YES on all classes that allow secure coding.
static boolean	accessInstanceVariablesDirectly()
static MPSCNNGradientKernel	alloc()
static MPSCNNGradientKernel	allocWithZone(org.moe.natj.general.ptr.VoidPtr zone)
static boolean	automaticallyNotifiesObserversForKey(java.lang.String key)
static void	cancelPreviousPerformRequestsWithTarget(java.lang.Object aTarget)
static void	cancelPreviousPerformRequestsWithTargetSelectorObject(java.lang.Object aTarget, org.moe.natj.objc.SEL aSelector, java.lang.Object anArgument)
static NSArray<java.lang.String>	classFallbacksForKeyedArchiver()
static org.moe.natj.objc.Class	classForKeyedUnarchiver()
static java.lang.String	debugDescription_static()
static java.lang.String	description_static()
MPSImage	encodeToCommandBufferSourceGradientSourceImageGradientState(MTLCommandBuffer commandBuffer, MPSImage sourceGradient, MPSImage sourceImage, MPSState gradientState) Encode a gradient filter and return a gradient
void	encodeToCommandBufferSourceGradientSourceImageGradientStateDestinationGradient(MTLCommandBuffer commandBuffer, MPSImage sourceGradient, MPSImage sourceImage, MPSState gradientState, MPSImage destinationGradient) Encode a gradient filter and return a gradient
static long	hash_static()
MPSCNNGradientKernel	init()
MPSCNNGradientKernel	initWithCoder(NSCoder aDecoder) NS_DESIGNATED_INITIALIZER
MPSCNNGradientKernel	initWithCoderDevice(NSCoder aDecoder, java.lang.Object device) NSSecureCoding compatability
MPSCNNGradientKernel	initWithDevice(java.lang.Object device) Standard init with default properties per filter type
static NSObject.Function_instanceMethodForSelector_ret	instanceMethodForSelector(org.moe.natj.objc.SEL aSelector)
static NSMethodSignature	instanceMethodSignatureForSelector(org.moe.natj.objc.SEL aSelector)
static boolean	instancesRespondToSelector(org.moe.natj.objc.SEL aSelector)
static boolean	isSubclassOfClass(org.moe.natj.objc.Class aClass)
long	kernelOffsetX() [@property] kernelOffsetX
long	kernelOffsetY() [@property] kernelOffsetY
static NSSet<java.lang.String>	keyPathsForValuesAffectingValueForKey(java.lang.String key)
static MPSCNNGradientKernel	new_objc()
static boolean	resolveClassMethod(org.moe.natj.objc.SEL sel)
static boolean	resolveInstanceMethod(org.moe.natj.objc.SEL sel)
void	setKernelOffsetX(long value) [@property] kernelOffsetX
void	setKernelOffsetY(long value) [@property] kernelOffsetY
static void	setVersion_static(long aVersion)
static org.moe.natj.objc.Class	superclass_static()
static boolean	supportsSecureCoding()
static long	version_static()

Methods inherited from class apple.metalperformanceshaders.MPSCNNBinaryKernel

appendBatchBarrier, clipRect, destinationFeatureChannelOffset, destinationImageAllocator, destinationImageDescriptorForSourceImagesSourceStates, encodeToCommandBufferPrimaryImageSecondaryImage, encodeToCommandBufferPrimaryImageSecondaryImageDestinationImage, encodeToCommandBufferPrimaryImageSecondaryImageDestinationStateDestinationStateIsTemporary, encodingStorageSizeForPrimaryImageSecondaryImageSourceStatesDestinationImage, isBackwards, isResultStateReusedAcrossBatch, isStateModified, padding, primaryDilationRateX, primaryDilationRateY, primaryEdgeMode, primaryKernelHeight, primaryKernelWidth, primaryOffset, primarySourceFeatureChannelMaxCount, primarySourceFeatureChannelOffset, primaryStrideInPixelsX, primaryStrideInPixelsY, resultStateForPrimaryImageSecondaryImageSourceStatesDestinationImage, secondaryDilationRateX, secondaryDilationRateY, secondaryEdgeMode, secondaryKernelHeight, secondaryKernelWidth, secondaryOffset, secondarySourceFeatureChannelMaxCount, secondarySourceFeatureChannelOffset, secondaryStrideInPixelsX, secondaryStrideInPixelsY, setClipRect, setDestinationFeatureChannelOffset, setDestinationImageAllocator, setPadding, setPrimaryEdgeMode, setPrimaryOffset, setPrimarySourceFeatureChannelMaxCount, setPrimarySourceFeatureChannelOffset, setPrimaryStrideInPixelsX, setPrimaryStrideInPixelsY, setSecondaryEdgeMode, setSecondaryOffset, setSecondarySourceFeatureChannelMaxCount, setSecondarySourceFeatureChannelOffset, setSecondaryStrideInPixelsX, setSecondaryStrideInPixelsY, temporaryResultStateForCommandBufferPrimaryImageSecondaryImageSourceStatesDestinationImage

Methods inherited from class apple.metalperformanceshaders.MPSKernel

copyWithZone, copyWithZoneDevice, device, encodeWithCoder, label, options, setLabel, setOptions

Methods inherited from class apple.NSObject

accessibilityActivate, accessibilityActivationPoint, accessibilityAssistiveTechnologyFocusedIdentifiers, accessibilityAttributedHint, accessibilityAttributedLabel, accessibilityAttributedUserInputLabels, accessibilityAttributedValue, accessibilityContainerType, accessibilityCustomActions, accessibilityCustomRotors, accessibilityDecrement, accessibilityDragSourceDescriptors, accessibilityDropPointDescriptors, accessibilityElementAtIndex, accessibilityElementCount, accessibilityElementDidBecomeFocused, accessibilityElementDidLoseFocus, accessibilityElementIsFocused, accessibilityElements, accessibilityElementsHidden, accessibilityFrame, accessibilityHint, accessibilityIncrement, accessibilityLabel, accessibilityLanguage, accessibilityNavigationStyle, accessibilityPath, accessibilityPerformEscape, accessibilityPerformMagicTap, accessibilityRespondsToUserInteraction, accessibilityScroll, accessibilityTextualContext, accessibilityTraits, accessibilityUserInputLabels, accessibilityValue, accessibilityViewIsModal, addObserverForKeyPathOptionsContext, attemptRecoveryFromErrorOptionIndex, attemptRecoveryFromErrorOptionIndexDelegateDidRecoverSelectorContextInfo, autoContentAccessingProxy, awakeAfterUsingCoder, awakeFromNib, class_objc, classForCoder, classForKeyedArchiver, copy, dealloc, debugDescription, description, dictionaryWithValuesForKeys, didChangeValueForKey, didChangeValueForKeyWithSetMutationUsingObjects, didChangeValuesAtIndexesForKey, doesNotRecognizeSelector, fileManagerShouldProceedAfterError, fileManagerWillProcessPath, finalize_objc, forwardingTargetForSelector, forwardInvocation, hash, indexOfAccessibilityElement, isAccessibilityElement, isEqual, isKindOfClass, isMemberOfClass, isProxy, methodForSelector, methodSignatureForSelector, mutableArrayValueForKey, mutableArrayValueForKeyPath, mutableCopy, mutableOrderedSetValueForKey, mutableOrderedSetValueForKeyPath, mutableSetValueForKey, mutableSetValueForKeyPath, observationInfo, observeValueForKeyPathOfObjectChangeContext, performSelector, performSelectorInBackgroundWithObject, performSelectorOnMainThreadWithObjectWaitUntilDone, performSelectorOnMainThreadWithObjectWaitUntilDoneModes, performSelectorOnThreadWithObjectWaitUntilDone, performSelectorOnThreadWithObjectWaitUntilDoneModes, performSelectorWithObject, performSelectorWithObjectAfterDelay, performSelectorWithObjectAfterDelayInModes, performSelectorWithObjectWithObject, prepareForInterfaceBuilder, provideImageDataBytesPerRowOrigin_Size_UserInfo, removeObserverForKeyPath, removeObserverForKeyPathContext, replacementObjectForCoder, replacementObjectForKeyedArchiver, respondsToSelector, self, setAccessibilityActivationPoint, setAccessibilityAttributedHint, setAccessibilityAttributedLabel, setAccessibilityAttributedUserInputLabels, setAccessibilityAttributedValue, setAccessibilityContainerType, setAccessibilityCustomActions, setAccessibilityCustomRotors, setAccessibilityDragSourceDescriptors, setAccessibilityDropPointDescriptors, setAccessibilityElements, setAccessibilityElementsHidden, setAccessibilityFrame, setAccessibilityHint, setAccessibilityLabel, setAccessibilityLanguage, setAccessibilityNavigationStyle, setAccessibilityPath, setAccessibilityRespondsToUserInteraction, setAccessibilityTextualContext, setAccessibilityTraits, setAccessibilityUserInputLabels, setAccessibilityValue, setAccessibilityViewIsModal, setIsAccessibilityElement, setNilValueForKey, setObservationInfo, setShouldGroupAccessibilityChildren, setValueForKey, setValueForKeyPath, setValueForUndefinedKey, setValuesForKeysWithDictionary, shouldGroupAccessibilityChildren, superclass, validateValueForKeyError, validateValueForKeyPathError, valueForKey, valueForKeyPath, valueForUndefinedKey, willChangeValueForKey, willChangeValueForKeyWithSetMutationUsingObjects, willChangeValuesAtIndexesForKey

Methods inherited from class org.moe.natj.objc.ObjCObject

equals, toString

Methods inherited from class org.moe.natj.general.NativeObject

finalize, getPeer, getPeerPointer

Methods inherited from class java.lang.Object

clone, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

MPSCNNGradientKernel

protected MPSCNNGradientKernel(org.moe.natj.general.Pointer peer)

Method Detail

accessInstanceVariablesDirectly

public static boolean accessInstanceVariablesDirectly()

alloc

public static MPSCNNGradientKernel alloc()

allocWithZone

public static MPSCNNGradientKernel allocWithZone(org.moe.natj.general.ptr.VoidPtr zone)

automaticallyNotifiesObserversForKey

public static boolean automaticallyNotifiesObserversForKey(java.lang.String key)

cancelPreviousPerformRequestsWithTarget

public static void cancelPreviousPerformRequestsWithTarget(java.lang.Object aTarget)

cancelPreviousPerformRequestsWithTargetSelectorObject

public static void cancelPreviousPerformRequestsWithTargetSelectorObject(java.lang.Object aTarget,
                                                                         org.moe.natj.objc.SEL aSelector,
                                                                         java.lang.Object anArgument)

classFallbacksForKeyedArchiver

public static NSArray<java.lang.String> classFallbacksForKeyedArchiver()

classForKeyedUnarchiver

public static org.moe.natj.objc.Class classForKeyedUnarchiver()

debugDescription_static

public static java.lang.String debugDescription_static()

description_static

public static java.lang.String description_static()

encodeToCommandBufferSourceGradientSourceImageGradientState

public MPSImage encodeToCommandBufferSourceGradientSourceImageGradientState(MTLCommandBuffer commandBuffer,
                                                                            MPSImage sourceGradient,
                                                                            MPSImage sourceImage,
                                                                            MPSState gradientState)

Encode a gradient filter and return a gradient

During training, gradient filters are used to calculate the gradient associated with the loss for each feature channel in the forward pass source image. For those nodes that are trainable, these are then used to refine the value used in the trainable parameter. They consume a source gradient image which contains the gradients corresponding with the forward pass destination image, and calculate the gradients corresponding to the forward pass source image.

A gradient filter consumes a MPSNNGradientState object which captured various forward pass properties such as offset and edgeMode at the time the forward pass was encoded. These are transferred to the MPSCNNBinaryKernel secondary image properties automatically when this method creates its destination image.

Parameters:

commandBuffer - The MTLCommandBuffer on which to encode

sourceGradient - The gradient image from the "next" filter in the graph (in the inference direction)

sourceImage - The image used as source image by the forward inference pass

gradientState - The MPSNNGradientState or MPSNNBinaryGradientState subclass produced by the forward inference pass

Returns:

The result gradient from the gradient filter

encodeToCommandBufferSourceGradientSourceImageGradientStateDestinationGradient

public void encodeToCommandBufferSourceGradientSourceImageGradientStateDestinationGradient(MTLCommandBuffer commandBuffer,
                                                                                           MPSImage sourceGradient,
                                                                                           MPSImage sourceImage,
                                                                                           MPSState gradientState,
                                                                                           MPSImage destinationGradient)

Encode a gradient filter and return a gradient

During training, gradient filters are used to calculate the gradient associated with the loss for each feature channel in the forward pass source image. For those nodes that are trainable, these are then used to refine the value used in the trainable parameter. They consume a source gradient image which contains the gradients corresponding with the forward pass destination image, and calculate the gradients corresponding to the forward pass source image.

A gradient filter consumes a MPSNNGradientState object which captured various forward pass properties such as offset and edgeMode at the time the forward pass was encoded. These are transferred to the MPSCNNBinaryKernel secondary image properties automatically when you use -[MPSCNNGradientKernel destinationImageDescriptorForSourceImages:sourceStates:]. If you do not call this method, then you are responsible for configuring all of the primary and secondary image properties in MPSCNNBinaryKernel. Please see class description for expected ordering of operations.

Parameters:

commandBuffer - The MTLCommandBuffer on which to encode

sourceGradient - The gradient image from the "next" filter in the graph

sourceImage - The image used as source image from the forward pass

gradientState - The MPSNNGradientState and MPSNNBinaryGradientState subclass produced by the forward pass

destinationGradient - The MPSImage into which to write the filter result

hash_static

public static long hash_static()

init

public MPSCNNGradientKernel init()

Overrides:

init in class MPSCNNBinaryKernel

initWithCoder

public MPSCNNGradientKernel initWithCoder(NSCoder aDecoder)

Description copied from interface: NSCoding

NS_DESIGNATED_INITIALIZER

Specified by:

initWithCoder in interface NSCoding

Overrides:

initWithCoder in class MPSCNNBinaryKernel

initWithCoderDevice

public MPSCNNGradientKernel initWithCoderDevice(NSCoder aDecoder,
                                                java.lang.Object device)

NSSecureCoding compatability

While the standard NSSecureCoding/NSCoding method -initWithCoder: should work, since the file can't know which device your data is allocated on, we have to guess and may guess incorrectly. To avoid that problem, use initWithCoder:device instead.

Overrides:

initWithCoderDevice in class MPSCNNBinaryKernel

Parameters:

aDecoder - The NSCoder subclass with your serialized MPSKernel

device - The MTLDevice on which to make the MPSKernel

Returns:

A new MPSKernel object, or nil if failure.

initWithDevice

public MPSCNNGradientKernel initWithDevice(java.lang.Object device)

Standard init with default properties per filter type

Overrides:

initWithDevice in class MPSCNNBinaryKernel

Parameters:

device - The device that the filter will be used on. May not be NULL.

Returns:

A pointer to the newly initialized object. This will fail, returning nil if the device is not supported. Devices must be MTLFeatureSet_iOS_GPUFamily2_v1 or later.

instanceMethodForSelector

public static NSObject.Function_instanceMethodForSelector_ret instanceMethodForSelector(org.moe.natj.objc.SEL aSelector)

instanceMethodSignatureForSelector

public static NSMethodSignature instanceMethodSignatureForSelector(org.moe.natj.objc.SEL aSelector)

instancesRespondToSelector

public static boolean instancesRespondToSelector(org.moe.natj.objc.SEL aSelector)

isSubclassOfClass

public static boolean isSubclassOfClass(org.moe.natj.objc.Class aClass)

kernelOffsetX

public long kernelOffsetX()

[@property] kernelOffsetX

Offset in the kernel reference frame to position the kernel in the X dimension

In some cases, the input gradient must be upsampled with zero insertion to account for things like strides in the forward MPSCNNKernel pass. As such, the offset, which describes a X,Y offset in the source coordinate space is insufficient to fully describe the offset applied to a kernel. The kernel offset is the offset after upsampling. Both the source offset and kernel offset are additive: effective offset = source offset * stride + kernel offset. The offset is applied to the (upsampled) source gradient

kernelOffsetY

public long kernelOffsetY()

[@property] kernelOffsetY

Offset in the kernel reference frame to position the kernel in the Y dimension

In some cases, the input gradient must be upsampled with zero insertion to account for things like strides in the forward MPSCNNKernel pass. As such, the offset, which describes a X,Y offset in the source coordinate space is insufficient to fully describe the offset applied to a kernel. The kernel offset is the offset after upsampling. Both the source offset and kernel offset are additive: effective offset = source offset * stride + kernel offset. The offset is applied to the (upsampled) source gradient

keyPathsForValuesAffectingValueForKey

public static NSSet<java.lang.String> keyPathsForValuesAffectingValueForKey(java.lang.String key)

new_objc

public static MPSCNNGradientKernel new_objc()

resolveClassMethod

public static boolean resolveClassMethod(org.moe.natj.objc.SEL sel)

resolveInstanceMethod

public static boolean resolveInstanceMethod(org.moe.natj.objc.SEL sel)

setKernelOffsetX

public void setKernelOffsetX(long value)

[@property] kernelOffsetX

Offset in the kernel reference frame to position the kernel in the X dimension

In some cases, the input gradient must be upsampled with zero insertion to account for things like strides in the forward MPSCNNKernel pass. As such, the offset, which describes a X,Y offset in the source coordinate space is insufficient to fully describe the offset applied to a kernel. The kernel offset is the offset after upsampling. Both the source offset and kernel offset are additive: effective offset = source offset * stride + kernel offset. The offset is applied to the (upsampled) source gradient

setKernelOffsetY

public void setKernelOffsetY(long value)

[@property] kernelOffsetY

Offset in the kernel reference frame to position the kernel in the Y dimension

In some cases, the input gradient must be upsampled with zero insertion to account for things like strides in the forward MPSCNNKernel pass. As such, the offset, which describes a X,Y offset in the source coordinate space is insufficient to fully describe the offset applied to a kernel. The kernel offset is the offset after upsampling. Both the source offset and kernel offset are additive: effective offset = source offset * stride + kernel offset. The offset is applied to the (upsampled) source gradient

setVersion_static

public static void setVersion_static(long aVersion)

superclass_static

public static org.moe.natj.objc.Class superclass_static()

supportsSecureCoding

public static boolean supportsSecureCoding()

_supportsSecureCoding

public boolean _supportsSecureCoding()

Description copied from interface: NSSecureCoding

This property must return YES on all classes that allow secure coding. Subclasses of classes that adopt NSSecureCoding and override initWithCoder: must also override this method and return YES. The Secure Coding Guide should be consulted when writing methods that decode data.

Specified by:

_supportsSecureCoding in interface NSSecureCoding

Overrides:

_supportsSecureCoding in class MPSCNNBinaryKernel

version_static

public static long version_static()