public class MPSCNNConvolutionTranspose extends MPSCNNKernel
The MPSCNNConvolutionTranspose specifies a transposed convolution. The MPSCNNConvolutionTranspose convolves the input image with a set of filters, each producing one feature map in the output image.
Some third-party frameworks may rotate the weights spatially by 180 degrees for Convolution Transpose. MPS uses the weights specified by the developer as-is and does not perform any rotation. The developer may need to rotate the weights appropriately in case this rotation is needed before the convolution transpose is applied.
When the stride in any dimension is greater than 1, the convolution transpose puts (stride - 1) zeroes in-between the source image pixels to create an expanded image. Then a convolution is done over the expanded image to generate the output of the convolution transpose.
Intermediate image size = (srcSize - 1) * Stride + 1
Examples:
[@code] So in case of sride == 2 (this behaves same in both dimensions)
Source image: _______________ | | | | | | 1 | 2 | 3 | 4 | | | | | | ---------------
Intermediate Image: ___________________________ | | | | | | | | | 1 | 0 | 2 | 0 | 3 | 0 | 4 | | | | | | | | | ---------------------------
NOTE on Offset: There are 2 types of offsets defined: 1) The Offset defined in MPSCNNKernel from which MPSCNNConvolutionTranspose inherits. This offset is applied to from where the kernel will be applied on the source. 2) The kernelOffsetX and kernelOffsetY which is the offset applied to the kernel when it is finally applied on the intermediate image.
So totalOffset = Offset * stride + kernelOffset
The offset defined by user refers to the coordinate frame of the expanded image (we are showing only 1 dimension X it can be extended to Y dimension as well) :
X indicates where the convolution transpose begins:
Intermediate Image: Offset = 0, kernelOffset = 0 ___________________________ | | | | | | | | | 1 | 0 | 2 | 0 | 3 | 0 | 4 | | X | | | | | | | ---------------------------
X indicates where the convolution transpose begins:
Intermediate Image: Offset = 0, kernelOffset = 1 ___________________________ | | | | | | | | | 1 | 0 | 2 | 0 | 3 | 0 | 4 | | | X | | | | | | ---------------------------
X indicates where the convolution transpose begins:
Intermediate Image: Offset = 0, kernelOffset = -1 ___________________________ | | | | | | | | X | 1 | 0 | 2 | 0 | 3 | 0 | 4 | | | | | | | | | ---------------------------
So if the user wanted to apply an offset of 2 on the source image of convolution transpose:
Source image: _______________ | | | | | | 1 | 2 | 3 | 4 | | | | X | | ---------------
offset = 2, kernelOffset = 0
Intermediate Image: ___________________________ | | | | | | | | | 1 | 0 | 2 | 0 | 3 | 0 | 4 | | | | | | X | | | ---------------------------
[@endcode]
Note that if your application is not using MPSCNNConvolutionGradientState to configure the convolution transpose with respect to convolution, your application may do this using padding policy. In such case if convolution uses valid padding policy, than convolution transpose should use full padding policy and vice vera. Full padding remains full.
NSObject.Function_instanceMethodForSelector_ret, NSObject.Function_methodForSelector_ret| Modifier | Constructor and Description |
|---|---|
protected |
MPSCNNConvolutionTranspose(org.moe.natj.general.Pointer peer) |
| Modifier and Type | Method and Description |
|---|---|
boolean |
_supportsSecureCoding()
This property must return YES on all classes that allow secure coding.
|
static boolean |
accessInstanceVariablesDirectly() |
long |
accumulatorPrecisionOption()
Precision of accumulator used in convolution.
|
static MPSCNNConvolutionTranspose |
alloc() |
static MPSCNNConvolutionTranspose |
allocWithZone(org.moe.natj.general.ptr.VoidPtr zone) |
static boolean |
automaticallyNotifiesObserversForKey(java.lang.String key) |
static void |
cancelPreviousPerformRequestsWithTarget(java.lang.Object aTarget) |
static void |
cancelPreviousPerformRequestsWithTargetSelectorObject(java.lang.Object aTarget,
org.moe.natj.objc.SEL aSelector,
java.lang.Object anArgument) |
static NSArray<java.lang.String> |
classFallbacksForKeyedArchiver() |
static org.moe.natj.objc.Class |
classForKeyedUnarchiver() |
MPSCNNConvolutionDataSource |
dataSource()
[@property] dataSource
|
static java.lang.String |
debugDescription_static() |
static java.lang.String |
description_static() |
MPSImage |
encodeToCommandBufferSourceImageConvolutionGradientState(MTLCommandBuffer commandBuffer,
MPSImage sourceImage,
MPSCNNConvolutionGradientState convolutionGradientState)
Encode a MPSCNNKernel into a command Buffer.
|
void |
encodeToCommandBufferSourceImageConvolutionGradientStateDestinationImage(MTLCommandBuffer commandBuffer,
MPSImage sourceImage,
MPSCNNConvolutionGradientState convolutionGradientState,
MPSImage destinationImage) |
MPSImage |
encodeToCommandBufferSourceImageConvolutionGradientStateDestinationStateDestinationStateIsTemporary(MTLCommandBuffer commandBuffer,
MPSImage sourceImage,
MPSCNNConvolutionGradientState convolutionGradientState,
org.moe.natj.general.ptr.Ptr<MPSCNNConvolutionTransposeGradientState> outState,
boolean isTemporary)
These low level encode functions should be used during training.
|
MPSCNNConvolutionWeightsAndBiasesState |
exportWeightsAndBiasesWithCommandBufferResultStateCanBeTemporary(MTLCommandBuffer commandBuffer,
boolean resultStateCanBeTemporary)
GPU side export.
|
long |
groups()
[@property] groups
|
static long |
hash_static() |
MPSCNNConvolutionTranspose |
init() |
MPSCNNConvolutionTranspose |
initWithCoder(NSCoder aDecoder)
NS_DESIGNATED_INITIALIZER
|
MPSCNNConvolutionTranspose |
initWithCoderDevice(NSCoder aDecoder,
java.lang.Object device)
|
MPSCNNConvolutionTranspose |
initWithDevice(java.lang.Object device)
Standard init with default properties per filter type
|
MPSCNNConvolutionTranspose |
initWithDeviceWeights(MTLDevice device,
MPSCNNConvolutionDataSource weights)
Initializes a convolution transpose kernel
|
long |
inputFeatureChannels()
[@property] inputFeatureChannels
|
static NSObject.Function_instanceMethodForSelector_ret |
instanceMethodForSelector(org.moe.natj.objc.SEL aSelector) |
static NSMethodSignature |
instanceMethodSignatureForSelector(org.moe.natj.objc.SEL aSelector) |
static boolean |
instancesRespondToSelector(org.moe.natj.objc.SEL aSelector) |
static boolean |
isSubclassOfClass(org.moe.natj.objc.Class aClass) |
long |
kernelOffsetX()
[@property] kernelOffsetX
|
long |
kernelOffsetY()
[@property] kernelOffsetY
|
static NSSet<java.lang.String> |
keyPathsForValuesAffectingValueForKey(java.lang.String key) |
static MPSCNNConvolutionTranspose |
new_objc() |
long |
outputFeatureChannels()
[@property] outputFeatureChannels
|
void |
reloadWeightsAndBiasesFromDataSource()
CPU side reload.
|
void |
reloadWeightsAndBiasesWithCommandBufferState(MTLCommandBuffer commandBuffer,
MPSCNNConvolutionWeightsAndBiasesState state)
GPU side reload.
|
static boolean |
resolveClassMethod(org.moe.natj.objc.SEL sel) |
static boolean |
resolveInstanceMethod(org.moe.natj.objc.SEL sel) |
MPSCNNConvolutionTransposeGradientState |
resultStateForSourceImageSourceStatesDestinationImage(MPSImage sourceImage,
NSArray<? extends MPSState> sourceStates,
MPSImage destinationImage)
Allocate a MPCNNConvolutionTransposeGradientState to hold the results from a -encodeBatchToCommandBuffer...
|
void |
setAccumulatorPrecisionOption(long value)
Precision of accumulator used in convolution.
|
void |
setKernelOffsetX(long value)
[@property] kernelOffsetX
|
void |
setKernelOffsetY(long value)
[@property] kernelOffsetY
|
static void |
setVersion_static(long aVersion) |
static org.moe.natj.objc.Class |
superclass_static() |
static boolean |
supportsSecureCoding() |
MPSCNNConvolutionTransposeGradientState |
temporaryResultStateForCommandBufferSourceImageSourceStatesDestinationImage(MTLCommandBuffer commandBuffer,
MPSImage sourceImage,
NSArray<? extends MPSState> sourceStates,
MPSImage destinationImage)
Allocate a temporary MPSState (subclass) to hold the results from a -encodeBatchToCommandBuffer...
|
static long |
version_static() |
appendBatchBarrier, clipRect, destinationFeatureChannelOffset, destinationImageAllocator, destinationImageDescriptorForSourceImagesSourceStates, dilationRateX, dilationRateY, edgeMode, encodeToCommandBufferSourceImage, encodeToCommandBufferSourceImageDestinationImage, encodeToCommandBufferSourceImageDestinationStateDestinationImage, encodeToCommandBufferSourceImageDestinationStateDestinationStateIsTemporary, encodingStorageSizeForSourceImageSourceStatesDestinationImage, isBackwards, isResultStateReusedAcrossBatch, isStateModified, kernelHeight, kernelWidth, offset, padding, setClipRect, setDestinationFeatureChannelOffset, setDestinationImageAllocator, setEdgeMode, setOffset, setPadding, setSourceFeatureChannelMaxCount, setSourceFeatureChannelOffset, sourceFeatureChannelMaxCount, sourceFeatureChannelOffset, strideInPixelsX, strideInPixelsYcopyWithZone, copyWithZoneDevice, device, encodeWithCoder, label, options, setLabel, setOptionsaccessibilityActivate, accessibilityActivationPoint, accessibilityAssistiveTechnologyFocusedIdentifiers, accessibilityAttributedHint, accessibilityAttributedLabel, accessibilityAttributedUserInputLabels, accessibilityAttributedValue, accessibilityContainerType, accessibilityCustomActions, accessibilityCustomRotors, accessibilityDecrement, accessibilityDragSourceDescriptors, accessibilityDropPointDescriptors, accessibilityElementAtIndex, accessibilityElementCount, accessibilityElementDidBecomeFocused, accessibilityElementDidLoseFocus, accessibilityElementIsFocused, accessibilityElements, accessibilityElementsHidden, accessibilityFrame, accessibilityHint, accessibilityIncrement, accessibilityLabel, accessibilityLanguage, accessibilityNavigationStyle, accessibilityPath, accessibilityPerformEscape, accessibilityPerformMagicTap, accessibilityRespondsToUserInteraction, accessibilityScroll, accessibilityTextualContext, accessibilityTraits, accessibilityUserInputLabels, accessibilityValue, accessibilityViewIsModal, addObserverForKeyPathOptionsContext, attemptRecoveryFromErrorOptionIndex, attemptRecoveryFromErrorOptionIndexDelegateDidRecoverSelectorContextInfo, autoContentAccessingProxy, awakeAfterUsingCoder, awakeFromNib, class_objc, classForCoder, classForKeyedArchiver, copy, dealloc, debugDescription, description, dictionaryWithValuesForKeys, didChangeValueForKey, didChangeValueForKeyWithSetMutationUsingObjects, didChangeValuesAtIndexesForKey, doesNotRecognizeSelector, fileManagerShouldProceedAfterError, fileManagerWillProcessPath, finalize_objc, forwardingTargetForSelector, forwardInvocation, hash, indexOfAccessibilityElement, isAccessibilityElement, isEqual, isKindOfClass, isMemberOfClass, isProxy, methodForSelector, methodSignatureForSelector, mutableArrayValueForKey, mutableArrayValueForKeyPath, mutableCopy, mutableOrderedSetValueForKey, mutableOrderedSetValueForKeyPath, mutableSetValueForKey, mutableSetValueForKeyPath, observationInfo, observeValueForKeyPathOfObjectChangeContext, performSelector, performSelectorInBackgroundWithObject, performSelectorOnMainThreadWithObjectWaitUntilDone, performSelectorOnMainThreadWithObjectWaitUntilDoneModes, performSelectorOnThreadWithObjectWaitUntilDone, performSelectorOnThreadWithObjectWaitUntilDoneModes, performSelectorWithObject, performSelectorWithObjectAfterDelay, performSelectorWithObjectAfterDelayInModes, performSelectorWithObjectWithObject, prepareForInterfaceBuilder, provideImageDataBytesPerRowOrigin_Size_UserInfo, removeObserverForKeyPath, removeObserverForKeyPathContext, replacementObjectForCoder, replacementObjectForKeyedArchiver, respondsToSelector, self, setAccessibilityActivationPoint, setAccessibilityAttributedHint, setAccessibilityAttributedLabel, setAccessibilityAttributedUserInputLabels, setAccessibilityAttributedValue, setAccessibilityContainerType, setAccessibilityCustomActions, setAccessibilityCustomRotors, setAccessibilityDragSourceDescriptors, setAccessibilityDropPointDescriptors, setAccessibilityElements, setAccessibilityElementsHidden, setAccessibilityFrame, setAccessibilityHint, setAccessibilityLabel, setAccessibilityLanguage, setAccessibilityNavigationStyle, setAccessibilityPath, setAccessibilityRespondsToUserInteraction, setAccessibilityTextualContext, setAccessibilityTraits, setAccessibilityUserInputLabels, setAccessibilityValue, setAccessibilityViewIsModal, setIsAccessibilityElement, setNilValueForKey, setObservationInfo, setShouldGroupAccessibilityChildren, setValueForKey, setValueForKeyPath, setValueForUndefinedKey, setValuesForKeysWithDictionary, shouldGroupAccessibilityChildren, superclass, validateValueForKeyError, validateValueForKeyPathError, valueForKey, valueForKeyPath, valueForUndefinedKey, willChangeValueForKey, willChangeValueForKeyWithSetMutationUsingObjects, willChangeValuesAtIndexesForKeyprotected MPSCNNConvolutionTranspose(org.moe.natj.general.Pointer peer)
public static boolean accessInstanceVariablesDirectly()
public static MPSCNNConvolutionTranspose alloc()
public static MPSCNNConvolutionTranspose allocWithZone(org.moe.natj.general.ptr.VoidPtr zone)
public static boolean automaticallyNotifiesObserversForKey(java.lang.String key)
public static void cancelPreviousPerformRequestsWithTarget(java.lang.Object aTarget)
public static void cancelPreviousPerformRequestsWithTargetSelectorObject(java.lang.Object aTarget,
org.moe.natj.objc.SEL aSelector,
java.lang.Object anArgument)
public static NSArray<java.lang.String> classFallbacksForKeyedArchiver()
public static org.moe.natj.objc.Class classForKeyedUnarchiver()
public static java.lang.String debugDescription_static()
public static java.lang.String description_static()
public long groups()
Number of groups input and output channels are divided into.
public static long hash_static()
public MPSCNNConvolutionTranspose init()
init in class MPSCNNKernelpublic MPSCNNConvolutionTranspose initWithCoder(NSCoder aDecoder)
NSCodinginitWithCoder in interface NSCodinginitWithCoder in class MPSCNNKernelpublic MPSCNNConvolutionTranspose initWithCoderDevice(NSCoder aDecoder, java.lang.Object device)
initWithCoderDevice in class MPSCNNKernelaDecoder - The NSCoder subclass with your serialized MPSKerneldevice - The MTLDevice on which to make the MPSKernelpublic MPSCNNConvolutionTranspose initWithDevice(java.lang.Object device)
MPSCNNKernelinitWithDevice in class MPSCNNKerneldevice - The device that the filter will be used on. May not be NULL.public MPSCNNConvolutionTranspose initWithDeviceWeights(MTLDevice device, MPSCNNConvolutionDataSource weights)
device - The MTLDevice on which this MPSCNNConvolutionTranspose filter will be usedweights - A pointer to a object that conforms to the MPSCNNConvolutionDataSource
protocol. The MPSCNNConvolutionDataSource protocol declares the methods that an
instance of MPSCNNConvolutionTranspose uses to obtain the weights and bias terms
for the CNN convolutionTranspose filter. Currently we support only Float32 weights.public long inputFeatureChannels()
The number of feature channels per pixel in the input image.
public static NSObject.Function_instanceMethodForSelector_ret instanceMethodForSelector(org.moe.natj.objc.SEL aSelector)
public static NSMethodSignature instanceMethodSignatureForSelector(org.moe.natj.objc.SEL aSelector)
public static boolean instancesRespondToSelector(org.moe.natj.objc.SEL aSelector)
public static boolean isSubclassOfClass(org.moe.natj.objc.Class aClass)
public long kernelOffsetX()
Offset in X from which the kernel starts sliding
public long kernelOffsetY()
Offset in Y from which the kernel starts sliding
public static NSSet<java.lang.String> keyPathsForValuesAffectingValueForKey(java.lang.String key)
public static MPSCNNConvolutionTranspose new_objc()
public long outputFeatureChannels()
The number of feature channels per pixel in the output image.
public static boolean resolveClassMethod(org.moe.natj.objc.SEL sel)
public static boolean resolveInstanceMethod(org.moe.natj.objc.SEL sel)
public void setKernelOffsetX(long value)
Offset in X from which the kernel starts sliding
public void setKernelOffsetY(long value)
Offset in Y from which the kernel starts sliding
public static void setVersion_static(long aVersion)
public static org.moe.natj.objc.Class superclass_static()
public static boolean supportsSecureCoding()
public boolean _supportsSecureCoding()
NSSecureCoding_supportsSecureCoding in interface NSSecureCoding_supportsSecureCoding in class MPSCNNKernelpublic static long version_static()
public long accumulatorPrecisionOption()
See MPSNeuralNetworkTypes.h for discussion. Default is MPSNNConvolutionAccumulatorPrecisionOptionFloat.
public MPSCNNConvolutionDataSource dataSource()
dataSource with which convolution transpose object was created
public MPSImage encodeToCommandBufferSourceImageConvolutionGradientState(MTLCommandBuffer commandBuffer, MPSImage sourceImage, MPSCNNConvolutionGradientState convolutionGradientState)
In the first iteration on this method, encodeToCommandBuffer:sourceImage:destinationImage: some work was left for the developer to do in the form of correctly setting the offset property and sizing the result buffer. With the introduction of the padding policy (see padding property) the filter can do this work itself. If you would like to have some input into what sort of MPSImage (e.g. temporary vs. regular) or what size it is or where it is allocated, you may set the destinationImageAllocator to allocate the image yourself.
This method uses the MPSNNPadding padding property to figure out how to size the result image and to set the offset property. See discussion in MPSNeuralNetworkTypes.h.
Note: the regular encodeToCommandBuffer:sourceImage: method may be used when no state is needed, such as when the convolution transpose operation is not balanced by a matching convolution object upstream. These encode methods are for auto encoders where each convolution in inference pass is coupled with convolution transpose. In order for convolution transpose to correctly undo the convolution downsampling, MPSCNNConvolutionGradientState produced by convolution is needed by convolution transpose to correctly size destination image. These methods are only useful for inference only network. For training, use encode methods that take MPSCNNConvolutionTransposeGradientState below.
commandBuffer - The command buffersourceImage - A MPSImage to use as the source images for the filter.convolutionGradientState - A valid MPSCNNConvolutionGradientState from the MPSCNNConvoluton counterpart to this MPSCNNConvolutionTranspose.
If there is no forward convolution counterpart, pass NULL here. This state affects the sizing
the result.public void encodeToCommandBufferSourceImageConvolutionGradientStateDestinationImage(MTLCommandBuffer commandBuffer, MPSImage sourceImage, MPSCNNConvolutionGradientState convolutionGradientState, MPSImage destinationImage)
public MPSImage encodeToCommandBufferSourceImageConvolutionGradientStateDestinationStateDestinationStateIsTemporary(MTLCommandBuffer commandBuffer, MPSImage sourceImage, MPSCNNConvolutionGradientState convolutionGradientState, org.moe.natj.general.ptr.Ptr<MPSCNNConvolutionTransposeGradientState> outState, boolean isTemporary)
public MPSCNNConvolutionWeightsAndBiasesState exportWeightsAndBiasesWithCommandBufferResultStateCanBeTemporary(MTLCommandBuffer commandBuffer, boolean resultStateCanBeTemporary)
public void reloadWeightsAndBiasesFromDataSource()
public void reloadWeightsAndBiasesWithCommandBufferState(MTLCommandBuffer commandBuffer, MPSCNNConvolutionWeightsAndBiasesState state)
commandBuffer - Metal command buffer on which application update kernel was enqueued consuming MPSCNNConvolutionGradientState's gradientForWeights and gradientForBiases buffers
and producing updateBuffer metal buffer.state - MPSCNNConvolutionWeightsAndBiasesState containing weights and biases buffers which have updated weights produced by application's update kernel.
The state readcount will be decremented.public MPSCNNConvolutionTransposeGradientState resultStateForSourceImageSourceStatesDestinationImage(MPSImage sourceImage, NSArray<? extends MPSState> sourceStates, MPSImage destinationImage)
resultStateForSourceImageSourceStatesDestinationImage in class MPSCNNKernelsourceImage - The MPSImage consumed by the associated -encode call.sourceStates - The list of MPSCNNConvolutionGradientState consumed by the associated -encode call,
for a batch size of 1. In auto encoders, this state is produced by corresponding MPSCNNConvolution.destinationImage - The destination image for the encode callpublic void setAccumulatorPrecisionOption(long value)
See MPSNeuralNetworkTypes.h for discussion. Default is MPSNNConvolutionAccumulatorPrecisionOptionFloat.
public MPSCNNConvolutionTransposeGradientState temporaryResultStateForCommandBufferSourceImageSourceStatesDestinationImage(MTLCommandBuffer commandBuffer, MPSImage sourceImage, NSArray<? extends MPSState> sourceStates, MPSImage destinationImage)
MPSCNNKernelA graph may need to allocate storage up front before executing. This may be necessary to avoid using too much memory and to manage large batches. The function should allocate any MPSState objects that will be produced by an -encode call with the indicated sourceImages and sourceStates inputs. Though the states can be further adjusted in the ensuing -encode call, the states should be initialized with all important data and all MTLResource storage allocated. The data stored in the MTLResource need not be initialized, unless the ensuing -encode call expects it to be.
The MTLDevice used by the result is derived from the command buffer. The padding policy will be applied to the filter before this is called to give it the chance to configure any properties like MPSCNNKernel.offset.
CAUTION: The kernel must have all properties set to values that will ultimately be passed to the -encode call that writes to the state, before -resultStateForSourceImages:sourceStates:destinationImage: is called or behavior is undefined. Please note that -destinationImageDescriptorForSourceImages:sourceStates:destinationImage: will alter some of these properties automatically based on the padding policy. If you intend to call that to make the destination image, then you should call that before -resultStateForSourceImages:sourceStates:destinationImage:. This will ensure the properties used in the encode call and in the destination image creation match those used to configure the state.
The following order is recommended:
// Configure MPSCNNKernel properties first kernel.edgeMode = MPSImageEdgeModeZero; kernel.destinationFeatureChannelOffset = 128; // concatenation without the copy ...
// ALERT: will change MPSCNNKernel properties MPSImageDescriptor * d = [kernel destinationImageDescriptorForSourceImage: source sourceStates: states]; MPSTemporaryImage * dest = [MPSTemporaryImage temporaryImageWithCommandBuffer: cmdBuf imageDescriptor: d];
// Now that all properties are configured properly, we can make the result state // and call encode. MPSState * __nullable destState = [kernel temporaryResultStateForCommandBuffer: cmdBuf sourceImage: source sourceStates: states];
// This form of -encode will be declared by the MPSCNNKernel subclass [kernel encodeToCommandBuffer: cmdBuf sourceImage: source destinationState: destState destinationImage: dest ];
Default: returns nil
temporaryResultStateForCommandBufferSourceImageSourceStatesDestinationImage in class MPSCNNKernelcommandBuffer - The command buffer to allocate the temporary storage against
The state will only be valid on this command buffer.sourceImage - The MPSImage consumed by the associated -encode call.sourceStates - The list of MPSStates consumed by the associated -encode call,
for a batch size of 1.destinationImage - The destination image for the encode call