Package apple.metalperformanceshaders

Class MPSCNNConvolutionTranspose

  • All Implemented Interfaces:
    NSCoding, NSCopying, NSSecureCoding, NSObject
    public class MPSCNNConvolutionTranspose
extends MPSCNNKernel
    MPSCNNConvolutionTranspose [@dependency] This depends on Metal.framework The MPSCNNConvolutionTranspose specifies a transposed convolution. The MPSCNNConvolutionTranspose convolves the input image with a set of filters, each producing one feature map in the output image. Some third-party frameworks may rotate the weights spatially by 180 degrees for Convolution Transpose. MPS uses the weights specified by the developer as-is and does not perform any rotation. The developer may need to rotate the weights appropriately in case this rotation is needed before the convolution transpose is applied. When the stride in any dimension is greater than 1, the convolution transpose puts (stride - 1) zeroes in-between the source image pixels to create an expanded image. Then a convolution is done over the expanded image to generate the output of the convolution transpose. Intermediate image size = (srcSize - 1) * Stride + 1 Examples: [@code] So in case of sride == 2 (this behaves same in both dimensions) Source image: _______________ | | | | | | 1 | 2 | 3 | 4 | | | | | | --------------- Intermediate Image: ___________________________ | | | | | | | | | 1 | 0 | 2 | 0 | 3 | 0 | 4 | | | | | | | | | --------------------------- NOTE on Offset: There are 2 types of offsets defined: 1) The Offset defined in MPSCNNKernel from which MPSCNNConvolutionTranspose inherits. This offset is applied to from where the kernel will be applied on the source. 2) The kernelOffsetX and kernelOffsetY which is the offset applied to the kernel when it is finally applied on the intermediate image. So totalOffset = Offset * stride + kernelOffset The offset defined by user refers to the coordinate frame of the expanded image (we are showing only 1 dimension X it can be extended to Y dimension as well) : X indicates where the convolution transpose begins: Intermediate Image: Offset = 0, kernelOffset = 0 ___________________________ | | | | | | | | | 1 | 0 | 2 | 0 | 3 | 0 | 4 | | X | | | | | | | --------------------------- X indicates where the convolution transpose begins: Intermediate Image: Offset = 0, kernelOffset = 1 ___________________________ | | | | | | | | | 1 | 0 | 2 | 0 | 3 | 0 | 4 | | | X | | | | | | --------------------------- X indicates where the convolution transpose begins: Intermediate Image: Offset = 0, kernelOffset = -1 ___________________________ | | | | | | | | X | 1 | 0 | 2 | 0 | 3 | 0 | 4 | | | | | | | | | --------------------------- So if the user wanted to apply an offset of 2 on the source image of convolution transpose: Source image: _______________ | | | | | | 1 | 2 | 3 | 4 | | | | X | | --------------- offset = 2, kernelOffset = 0 Intermediate Image: ___________________________ | | | | | | | | | 1 | 0 | 2 | 0 | 3 | 0 | 4 | | | | | | X | | | --------------------------- [@endcode] Note that if your application is not using MPSCNNConvolutionGradientState to configure the convolution transpose with respect to convolution, your application may do this using padding policy. In such case if convolution uses valid padding policy, than convolution transpose should use full padding policy and vice vera. Full padding remains full. API-Since: 11.0

    • Constructor Detail

      • MPSCNNConvolutionTranspose

        protected MPSCNNConvolutionTranspose​(org.moe.natj.general.Pointer peer)

    • Method Detail

      • accessInstanceVariablesDirectly

        public static boolean accessInstanceVariablesDirectly()

      • automaticallyNotifiesObserversForKey

        public static boolean automaticallyNotifiesObserversForKey​(@NotNull
                                                           @NotNull java.lang.String key)

      • cancelPreviousPerformRequestsWithTarget

        public static void cancelPreviousPerformRequestsWithTarget​(@NotNull
                                                           @NotNull java.lang.Object aTarget)

      • cancelPreviousPerformRequestsWithTargetSelectorObject

        public static void cancelPreviousPerformRequestsWithTargetSelectorObject​(@NotNull
                                                                         @NotNull java.lang.Object aTarget,
                                                                         @NotNull
                                                                         @NotNull org.moe.natj.objc.SEL aSelector,
                                                                         @Nullable
                                                                         @Nullable java.lang.Object anArgument)

      • classFallbacksForKeyedArchiver

        @NotNull
public static @NotNull NSArray<java.lang.String> classFallbacksForKeyedArchiver()

      • classForKeyedUnarchiver

        @NotNull
public static @NotNull org.moe.natj.objc.Class classForKeyedUnarchiver()

      • debugDescription_static

        public static java.lang.String debugDescription_static()

      • description_static

        public static java.lang.String description_static()

      • groups

        public long groups()
        [@property] groups Number of groups input and output channels are divided into.

      • hash_static

        public static long hash_static()

      • initWithCoderDevice

        public MPSCNNConvolutionTranspose initWithCoderDevice​(@NotNull
                                                      @NotNull NSCoder aDecoder,
                                                      @NotNull
                                                      @NotNull java.lang.Object device)
        support
        Overrides:
        initWithCoderDevice in class MPSCNNKernel
        Parameters:
        aDecoder - The NSCoder subclass with your serialized MPSKernel
        device - The MTLDevice on which to make the MPSKernel
        Returns:
        A new MPSKernel object, or nil if failure. API-Since: 11.0

      • initWithDevice

        public MPSCNNConvolutionTranspose initWithDevice​(@NotNull
                                                 @NotNull java.lang.Object device)
        Description copied from class: MPSCNNKernel
        Standard init with default properties per filter type
        Overrides:
        initWithDevice in class MPSCNNKernel
        Parameters:
        device - The device that the filter will be used on. May not be NULL.
        Returns:
        A pointer to the newly initialized object. This will fail, returning nil if the device is not supported. Devices must be MTLFeatureSet_iOS_GPUFamily2_v1 or later.

      • initWithDeviceWeights

        public MPSCNNConvolutionTranspose initWithDeviceWeights​(@NotNull
                                                        @NotNull MTLDevice device,
                                                        @NotNull
                                                        @NotNull MPSCNNConvolutionDataSource weights)
        Initializes a convolution transpose kernel
        Parameters:
        device - The MTLDevice on which this MPSCNNConvolutionTranspose filter will be used
        weights - A pointer to a object that conforms to the MPSCNNConvolutionDataSource protocol. The MPSCNNConvolutionDataSource protocol declares the methods that an instance of MPSCNNConvolutionTranspose uses to obtain the weights and bias terms for the CNN convolutionTranspose filter. Currently we support only Float32 weights.
        Returns:
        A valid MPSCNNConvolutionTranspose object.

      • inputFeatureChannels

        public long inputFeatureChannels()
        [@property] inputFeatureChannels The number of feature channels per pixel in the input image.

      • instanceMethodSignatureForSelector

        public static NSMethodSignature instanceMethodSignatureForSelector​(org.moe.natj.objc.SEL aSelector)

      • instancesRespondToSelector

        public static boolean instancesRespondToSelector​(org.moe.natj.objc.SEL aSelector)

      • isSubclassOfClass

        public static boolean isSubclassOfClass​(org.moe.natj.objc.Class aClass)

      • kernelOffsetX

        public long kernelOffsetX()
        [@property] kernelOffsetX Offset in X from which the kernel starts sliding

      • kernelOffsetY

        public long kernelOffsetY()
        [@property] kernelOffsetY Offset in Y from which the kernel starts sliding

      • keyPathsForValuesAffectingValueForKey

        @NotNull
public static @NotNull NSSet<java.lang.String> keyPathsForValuesAffectingValueForKey​(@NotNull
                                                                                     @NotNull java.lang.String key)

      • outputFeatureChannels

        public long outputFeatureChannels()
        [@property] outputFeatureChannels The number of feature channels per pixel in the output image.

      • resolveClassMethod

        public static boolean resolveClassMethod​(org.moe.natj.objc.SEL sel)

      • resolveInstanceMethod

        public static boolean resolveInstanceMethod​(org.moe.natj.objc.SEL sel)

      • setKernelOffsetX

        public void setKernelOffsetX​(long value)
        [@property] kernelOffsetX Offset in X from which the kernel starts sliding

      • setKernelOffsetY

        public void setKernelOffsetY​(long value)
        [@property] kernelOffsetY Offset in Y from which the kernel starts sliding

      • setVersion_static

        public static void setVersion_static​(long aVersion)

      • superclass_static

        public static org.moe.natj.objc.Class superclass_static()

      • supportsSecureCoding

        public static boolean supportsSecureCoding()

      • _supportsSecureCoding

        public boolean _supportsSecureCoding()
        Description copied from interface: NSSecureCoding
        This property must return YES on all classes that allow secure coding. Subclasses of classes that adopt NSSecureCoding and override initWithCoder: must also override this method and return YES. The Secure Coding Guide should be consulted when writing methods that decode data.
        Specified by:
        _supportsSecureCoding in interface NSSecureCoding
        Overrides:
        _supportsSecureCoding in class MPSCNNKernel

      • version_static

        public static long version_static()

      • accumulatorPrecisionOption

        public long accumulatorPrecisionOption()
        Precision of accumulator used in convolution. See MPSNeuralNetworkTypes.h for discussion. Default is MPSNNConvolutionAccumulatorPrecisionOptionFloat. API-Since: 11.3

      • dataSource

        @NotNull
public @NotNull MPSCNNConvolutionDataSource dataSource()
        [@property] dataSource dataSource with which convolution transpose object was created API-Since: 13.0

      • encodeToCommandBufferSourceImageConvolutionGradientState

        @NotNull
public @NotNull MPSImage encodeToCommandBufferSourceImageConvolutionGradientState​(@NotNull
                                                                                  @NotNull MTLCommandBuffer commandBuffer,
                                                                                  @NotNull
                                                                                  @NotNull MPSImage sourceImage,
                                                                                  @Nullable
                                                                                  @Nullable MPSCNNConvolutionGradientState convolutionGradientState)
        Encode a MPSCNNKernel into a command Buffer. Create a texture to hold the result and return it. In the first iteration on this method, encodeToCommandBuffer:sourceImage:destinationImage: some work was left for the developer to do in the form of correctly setting the offset property and sizing the result buffer. With the introduction of the padding policy (see padding property) the filter can do this work itself. If you would like to have some input into what sort of MPSImage (e.g. temporary vs. regular) or what size it is or where it is allocated, you may set the destinationImageAllocator to allocate the image yourself. This method uses the MPSNNPadding padding property to figure out how to size the result image and to set the offset property. See discussion in MPSNeuralNetworkTypes.h. Note: the regular encodeToCommandBuffer:sourceImage: method may be used when no state is needed, such as when the convolution transpose operation is not balanced by a matching convolution object upstream. These encode methods are for auto encoders where each convolution in inference pass is coupled with convolution transpose. In order for convolution transpose to correctly undo the convolution downsampling, MPSCNNConvolutionGradientState produced by convolution is needed by convolution transpose to correctly size destination image. These methods are only useful for inference only network. For training, use encode methods that take MPSCNNConvolutionTransposeGradientState below.
        Parameters:
        commandBuffer - The command buffer
        sourceImage - A MPSImage to use as the source images for the filter.
        convolutionGradientState - A valid MPSCNNConvolutionGradientState from the MPSCNNConvoluton counterpart to this MPSCNNConvolutionTranspose. If there is no forward convolution counterpart, pass NULL here. This state affects the sizing the result.
        Returns:
        A MPSImage or MPSTemporaryImage allocated per the destinationImageAllocator containing the output of the graph. The offset property will be adjusted to reflect the offset used during the encode. The returned image will be automatically released when the command buffer completes. If you want to keep it around for longer, retain the image. (ARC will do this for you if you use it later.) API-Since: 11.3

      • encodeToCommandBufferSourceImageConvolutionGradientStateDestinationImage

        public void encodeToCommandBufferSourceImageConvolutionGradientStateDestinationImage​(@NotNull
                                                                                     @NotNull MTLCommandBuffer commandBuffer,
                                                                                     @NotNull
                                                                                     @NotNull MPSImage sourceImage,
                                                                                     @Nullable
                                                                                     @Nullable MPSCNNConvolutionGradientState convolutionGradientState,
                                                                                     @NotNull
                                                                                     @NotNull MPSImage destinationImage)
        API-Since: 11.3

      • encodeToCommandBufferSourceImageConvolutionGradientStateDestinationStateDestinationStateIsTemporary

        @NotNull
public @NotNull MPSImage encodeToCommandBufferSourceImageConvolutionGradientStateDestinationStateDestinationStateIsTemporary​(@NotNull
                                                                                                                             @NotNull MTLCommandBuffer commandBuffer,
                                                                                                                             @NotNull
                                                                                                                             @NotNull MPSImage sourceImage,
                                                                                                                             @Nullable
                                                                                                                             @Nullable MPSCNNConvolutionGradientState convolutionGradientState,
                                                                                                                             @NotNull
                                                                                                                             @NotNull org.moe.natj.general.ptr.Ptr<MPSCNNConvolutionTransposeGradientState> outState,
                                                                                                                             boolean isTemporary)
        These low level encode functions should be used during training. The first two encode functions, which return destination image on left hand side, takes in MPSCNNConvolutionGradientState that was produced by corresponding MPSCNNConvolution when there is one e.g. auto encoders. This state is used to correctly size destination being returned. These encode methods return MPSCNNConvoltionTransposeGradientState object on auto release pool to be consumed by MPSCNNConvolutionTransposeGradient. API-Since: 13.0

      • exportWeightsAndBiasesWithCommandBufferResultStateCanBeTemporary

        @NotNull
public @NotNull MPSCNNConvolutionWeightsAndBiasesState exportWeightsAndBiasesWithCommandBufferResultStateCanBeTemporary​(@NotNull
                                                                                                                        @NotNull MTLCommandBuffer commandBuffer,
                                                                                                                        boolean resultStateCanBeTemporary)
        GPU side export. Enqueue a kernel to export current weights and biases stored in MPSCNNConvoltionTranspose's internal buffers into weights and biases MTLBuffer * returned in MPSCNNConvolutionWeightsAndBiasesState. * * @param commandBuffer Metal command buffer on which export kernel is enqueued. * @param resultStateCanBeTemporary If FALSE, state returned will be non-temporary. If TRUE, returned state may or may not be temporary. * @return MPSCNNConvolutionWeightsAndBiasesState containing weights and biases buffer to which weights got exported. This state and be temporary or non-temporary depending on the flag resultStateCanBeTemporary API-Since: 13.0

      • reloadWeightsAndBiasesFromDataSource

        public void reloadWeightsAndBiasesFromDataSource()
        CPU side reload. Reload the updated weights and biases from data provider into internal weights and bias buffers. Weights and biases gradients needed for update are obtained from MPSCNNConvolutionTransposeGradientState object. Data provider passed in init call is used for this purpose. API-Since: 13.0

      • reloadWeightsAndBiasesWithCommandBufferState

        public void reloadWeightsAndBiasesWithCommandBufferState​(@NotNull
                                                         @NotNull MTLCommandBuffer commandBuffer,
                                                         @NotNull
                                                         @NotNull MPSCNNConvolutionWeightsAndBiasesState state)
        GPU side reload. Reload the updated weights and biases from update buffer produced by application enqueued metal kernel into internal weights and biases buffer. Weights and biases gradients needed for update are obtained from MPSCNNConvolutionTransposeGradientState object's gradientForWeights and gradientForBiases metal buffer.
        Parameters:
        commandBuffer - Metal command buffer on which application update kernel was enqueued consuming MPSCNNConvolutionGradientState's gradientForWeights and gradientForBiases buffers and producing updateBuffer metal buffer.
        state - MPSCNNConvolutionWeightsAndBiasesState containing weights and biases buffers which have updated weights produced by application's update kernel. The state readcount will be decremented. API-Since: 13.0

      • resultStateForSourceImageSourceStatesDestinationImage

        @Nullable
public @Nullable MPSCNNConvolutionTransposeGradientState resultStateForSourceImageSourceStatesDestinationImage​(@NotNull
                                                                                                               @NotNull MPSImage sourceImage,
                                                                                                               @Nullable
                                                                                                               @Nullable NSArray<? extends MPSState> sourceStates,
                                                                                                               @NotNull
                                                                                                               @NotNull MPSImage destinationImage)
        Allocate a MPCNNConvolutionTransposeGradientState to hold the results from a -encodeBatchToCommandBuffer... operation
        Overrides:
        resultStateForSourceImageSourceStatesDestinationImage in class MPSCNNKernel
        Parameters:
        sourceImage - The MPSImage consumed by the associated -encode call.
        sourceStates - The list of MPSCNNConvolutionGradientState consumed by the associated -encode call, for a batch size of 1. In auto encoders, this state is produced by corresponding MPSCNNConvolution.
        destinationImage - The destination image for the encode call
        Returns:
        The list of states produced by the -encode call for batch size of 1. -isResultStateReusedAcrossBatch returns YES for MPSCNNConvolutionTranspose so same state is used across entire batch. State object is not reusasable across batches. API-Since: 11.3

      • setAccumulatorPrecisionOption

        public void setAccumulatorPrecisionOption​(long value)
        Precision of accumulator used in convolution. See MPSNeuralNetworkTypes.h for discussion. Default is MPSNNConvolutionAccumulatorPrecisionOptionFloat. API-Since: 11.3

      • temporaryResultStateForCommandBufferSourceImageSourceStatesDestinationImage

        @Nullable
public @Nullable MPSCNNConvolutionTransposeGradientState temporaryResultStateForCommandBufferSourceImageSourceStatesDestinationImage​(@NotNull
                                                                                                                                     @NotNull MTLCommandBuffer commandBuffer,
                                                                                                                                     @NotNull
                                                                                                                                     @NotNull MPSImage sourceImage,
                                                                                                                                     @Nullable
                                                                                                                                     @Nullable NSArray<? extends MPSState> sourceStates,
                                                                                                                                     @NotNull
                                                                                                                                     @NotNull MPSImage destinationImage)
        API-Since: 11.3
        Overrides:
        temporaryResultStateForCommandBufferSourceImageSourceStatesDestinationImage in class MPSCNNKernel
        Parameters:
        commandBuffer - The command buffer to allocate the temporary storage against The state will only be valid on this command buffer.
        sourceImage - The MPSImage consumed by the associated -encode call.
        sourceStates - The list of MPSStates consumed by the associated -encode call, for a batch size of 1.
        destinationImage - The destination image for the encode call
        Returns:
        The list of states produced by the -encode call for batch size of 1. When the batch size is not 1, this function will be called repeatedly unless -isResultStateReusedAcrossBatch returns YES. If -isResultStateReusedAcrossBatch returns YES, then it will be called once per batch and the MPSStateBatch array will contain MPSStateBatch.length references to the same object. API-Since: 11.3

      • useStoredAccessor

        @Deprecated
public static boolean useStoredAccessor()
        Deprecated.