public class MPSTemporaryImage extends MPSImage
MPSTemporaryImages are for MPSImages with short lifetimes.
What is temporary memory? It is memory, plain and simple. Analogy: If we use an app as an analogy for a command buffer, then "Regular memory" (such as what backs a MPSImage or the typical MTLTexture) would be memory that you allocate at launch and never free. Temporary memory would be memory that you free when you are done with it so it can be used for something else as needed later in your app. You /could/ write your app to allocate everything you will ever need up front, but this is very inefficient and quite frankly a pain to plan out in advance. You don't do it for your app, so why would you do it for your command buffers?
Welcome to the 1970's! We have added a heap.
Unsurprisingly, MPSTemporaryImages can provide for profound reduction in the the amount of memory used by your application. Like malloc, MPS maintains a heap of memory usable in a command buffer. Over the lifetime of a command buffer, the same piece of memory may be reused many times. This means that each time the same memory is reused, it aliases with previous uses. If we aren't careful, we might find that needed data is overwritten by successive allocations. However, this is no different than accessing freed memory only to discover it doesn't contain what you thought it did anymore, so you should be able to keep out of trouble by following a few simple rules, like with malloc.
To this end, we added some restrictions to help you out and get a bit more performance. Some comments are appended in parentheses below to extend the analogy of command buffer = program:
- The textures are MTLStorageModePrivate. You can not, for example, use [MTLTexture getBytes...] or [MTLTexture replaceRegion...] with them. MPSTemporaryImages are strictly read and written by the GPU. (There is protected memory to prevent other processes from overwriting your heap.)
- The temporary image may be used only on a single MTLCommandBuffer. This limits the chronology to a single linear time stream. (The heap is specific to just one command buffer. There are no mutexes to coordinate timing of simultaneous access by multiple GPUs. Nor are we likely to like them if there were. So, we disallow it.)
- The readCount property must be managed correctly. Please see the description of the readCount property for full details. (The readCount is a reference count for the block of memory that holds your data. The usual undefined behaviors apply to reading data that has been released. We assert when we can to prevent that from happening accidentally, just as a program might segfault. The readCount counts procedural users of the object -- MPSKernel.encode... calls that read the MPSTemporaryImage. As each reads from it, the readCount is automatically decremented. The texture data will be freed in typical usage at the right time as the readCount reaches zero, typically with little user involvement other than to set the readCount up front. We did examine using the main MPSTemporaryImage reference count for this instead so that ARC would do work for you automatically. Alas, ARC destroys things at end of scope rather than promptly, sometimes resulting in greatly increased memory usage. These allocations are large! So, we use this method instead.)
Since MPSTemporaryImages can only be used with a single MTLCommandBuffer, and can not be used off the GPU, they generally should not be kept around past the completion of the MTLCommandBuffer. The lifetime of MPSTemporaryImages is expected to be typically extremely short, perhaps only a few lines of code. Like malloc, it is intended to be fairly cheap to make MPSTemporaryImages and throw them away. Please do so.
To keep the lifetime of the underlying texture allocation as short as possible, the underlying texture is not allocated until the first time the MPSTemporaryImage is used by a MPSCNNKernel or the .texture property is read. The readCount property serves to limit the lifetime on the other end.
You may use the MPSTemporaryImage.texture with MPSUnaryImageKernel -encode... methods, iff featureChannels <= 4 and the MTLTexture conforms to requirements of that MPSKernel. There is no locking mechanism provided to prevent a MTLTexture returned from the .texture property from becoming invalid when the readCount reaches 0.
Finally, MPSTemporaryImages are complicated to use with blit encoders. Your application should assume that the MPSTemporaryImage is backed by a MTLHeap, and consequently needs a MTLFence to ensure that compute command encoders and other encoders do not trip over one another with heap based memory. MPS will almost never use a blit encoder for this reason. If you do need one, then you will need to make a new compute encoder to block on whatever previous compute encoder last used the heap block. (MPS will not tell you who previously used the heap block. That encoder is almost certainly long dead anyway.) If concurrent encoders are involved, then a barrier might be needed. Within that compute encoder, you will call -updateFence. End the compute encoder, make a blit encoder wait for the fence, do the blit, update a new fence, then make a new compute encoder, wait for the second fence, then you can continue. Possibly the second do-nothing compute encoder needs to be ended so MPS can be called. Frankly, we don't bother with blit encoders and just write a compute operation for copy / clear as needed, or better yet find a way to eliminate the clear / copy pass so we don't have to pay for it. Note: the most common use of a blit encoder, -synchronizeResource: can not encounter this problem because MPSTemporaryImages live in GPU private memory and can not be read by the CPU.
MPSTemporaryImages can otherwise be used wherever MPSImages are used.
NSObject.Function_instanceMethodForSelector_ret, NSObject.Function_methodForSelector_ret| Modifier | Constructor and Description |
|---|---|
protected |
MPSTemporaryImage(org.moe.natj.general.Pointer peer) |
| Modifier and Type | Method and Description |
|---|---|
static boolean |
accessInstanceVariablesDirectly() |
static MPSTemporaryImage |
alloc() |
static MPSTemporaryImage |
allocWithZone(org.moe.natj.general.ptr.VoidPtr zone) |
static boolean |
automaticallyNotifiesObserversForKey(java.lang.String key) |
static void |
cancelPreviousPerformRequestsWithTarget(java.lang.Object aTarget) |
static void |
cancelPreviousPerformRequestsWithTargetSelectorObject(java.lang.Object aTarget,
org.moe.natj.objc.SEL aSelector,
java.lang.Object anArgument) |
static NSArray<java.lang.String> |
classFallbacksForKeyedArchiver() |
static org.moe.natj.objc.Class |
classForKeyedUnarchiver() |
static java.lang.String |
debugDescription_static() |
static MPSImageAllocator |
defaultAllocator()
Get a well known MPSImageAllocator that makes MPSTemporaryImages
|
static java.lang.String |
description_static() |
static long |
hash_static() |
MPSTemporaryImage |
init() |
MPSTemporaryImage |
initWithDeviceImageDescriptor(MTLDevice device,
MPSImageDescriptor imageDescriptor)
Initialize an empty image object
|
MPSTemporaryImage |
initWithParentImageSliceRangeFeatureChannels(MPSImage parent,
NSRange sliceRange,
long featureChannels)
Use -batchRepresentation or -subImageWithFeatureChannelRange instead
|
MPSTemporaryImage |
initWithTextureFeatureChannels(MTLTexture texture,
long featureChannels)
Initialize an MPSImage object using Metal texture.
|
static NSObject.Function_instanceMethodForSelector_ret |
instanceMethodForSelector(org.moe.natj.objc.SEL aSelector) |
static NSMethodSignature |
instanceMethodSignatureForSelector(org.moe.natj.objc.SEL aSelector) |
static boolean |
instancesRespondToSelector(org.moe.natj.objc.SEL aSelector) |
static boolean |
isSubclassOfClass(org.moe.natj.objc.Class aClass) |
static NSSet<java.lang.String> |
keyPathsForValuesAffectingValueForKey(java.lang.String key) |
static MPSTemporaryImage |
new_objc() |
static void |
prefetchStorageWithCommandBufferImageDescriptorList(MTLCommandBuffer commandBuffer,
NSArray<? extends MPSImageDescriptor> descriptorList)
Help MPS decide which allocations to make ahead of time
|
long |
readCount()
The number of times a temporary image may be read by a MPSCNNKernel
before its contents become undefined.
|
static boolean |
resolveClassMethod(org.moe.natj.objc.SEL sel) |
static boolean |
resolveInstanceMethod(org.moe.natj.objc.SEL sel) |
void |
setReadCount(long value)
The number of times a temporary image may be read by a MPSCNNKernel
before its contents become undefined.
|
static void |
setVersion_static(long aVersion) |
static org.moe.natj.objc.Class |
superclass_static() |
static MPSTemporaryImage |
temporaryImageWithCommandBufferImageDescriptor(MTLCommandBuffer commandBuffer,
MPSImageDescriptor imageDescriptor)
Initialize a MPSTemporaryImage for use on a MTLCommandBuffer
|
static MPSTemporaryImage |
temporaryImageWithCommandBufferTextureDescriptor(MTLCommandBuffer commandBuffer,
MTLTextureDescriptor textureDescriptor)
Low level interface for creating a MPSTemporaryImage using a MTLTextureDescriptor
|
static MPSTemporaryImage |
temporaryImageWithCommandBufferTextureDescriptorFeatureChannels(MTLCommandBuffer commandBuffer,
MTLTextureDescriptor textureDescriptor,
long featureChannels)
Low level interface for creating a MPSTemporaryImage using a MTLTextureDescriptor
|
static long |
version_static() |
device, featureChannelFormat, featureChannels, height, label, numberOfImages, parent, pixelFormat, pixelSize, precision, readBytesDataLayoutBytesPerRowBytesPerImageRegionFeatureChannelInfoImageIndex, readBytesDataLayoutBytesPerRowRegionFeatureChannelInfoImageIndex, readBytesDataLayoutImageIndex, resourceSize, setLabel, setPurgeableState, subImageWithFeatureChannelRange, synchronizeOnCommandBuffer, texture, textureType, usage, width, writeBytesDataLayoutBytesPerColumnBytesPerRowBytesPerImageRegionFeatureChannelInfoImageIndex, writeBytesDataLayoutBytesPerRowBytesPerImageRegionFeatureChannelInfoImageIndex, writeBytesDataLayoutBytesPerRowRegionFeatureChannelInfoImageIndex, writeBytesDataLayoutImageIndexaccessibilityActivate, accessibilityActivationPoint, accessibilityAssistiveTechnologyFocusedIdentifiers, accessibilityAttributedHint, accessibilityAttributedLabel, accessibilityAttributedUserInputLabels, accessibilityAttributedValue, accessibilityContainerType, accessibilityCustomActions, accessibilityCustomRotors, accessibilityDecrement, accessibilityDragSourceDescriptors, accessibilityDropPointDescriptors, accessibilityElementAtIndex, accessibilityElementCount, accessibilityElementDidBecomeFocused, accessibilityElementDidLoseFocus, accessibilityElementIsFocused, accessibilityElements, accessibilityElementsHidden, accessibilityFrame, accessibilityHint, accessibilityIncrement, accessibilityLabel, accessibilityLanguage, accessibilityNavigationStyle, accessibilityPath, accessibilityPerformEscape, accessibilityPerformMagicTap, accessibilityRespondsToUserInteraction, accessibilityScroll, accessibilityTextualContext, accessibilityTraits, accessibilityUserInputLabels, accessibilityValue, accessibilityViewIsModal, addObserverForKeyPathOptionsContext, attemptRecoveryFromErrorOptionIndex, attemptRecoveryFromErrorOptionIndexDelegateDidRecoverSelectorContextInfo, autoContentAccessingProxy, awakeAfterUsingCoder, awakeFromNib, class_objc, classForCoder, classForKeyedArchiver, copy, dealloc, debugDescription, description, dictionaryWithValuesForKeys, didChangeValueForKey, didChangeValueForKeyWithSetMutationUsingObjects, didChangeValuesAtIndexesForKey, doesNotRecognizeSelector, fileManagerShouldProceedAfterError, fileManagerWillProcessPath, finalize_objc, forwardingTargetForSelector, forwardInvocation, hash, indexOfAccessibilityElement, isAccessibilityElement, isEqual, isKindOfClass, isMemberOfClass, isProxy, methodForSelector, methodSignatureForSelector, mutableArrayValueForKey, mutableArrayValueForKeyPath, mutableCopy, mutableOrderedSetValueForKey, mutableOrderedSetValueForKeyPath, mutableSetValueForKey, mutableSetValueForKeyPath, observationInfo, observeValueForKeyPathOfObjectChangeContext, performSelector, performSelectorInBackgroundWithObject, performSelectorOnMainThreadWithObjectWaitUntilDone, performSelectorOnMainThreadWithObjectWaitUntilDoneModes, performSelectorOnThreadWithObjectWaitUntilDone, performSelectorOnThreadWithObjectWaitUntilDoneModes, performSelectorWithObject, performSelectorWithObjectAfterDelay, performSelectorWithObjectAfterDelayInModes, performSelectorWithObjectWithObject, prepareForInterfaceBuilder, provideImageDataBytesPerRowOrigin_Size_UserInfo, removeObserverForKeyPath, removeObserverForKeyPathContext, replacementObjectForCoder, replacementObjectForKeyedArchiver, respondsToSelector, self, setAccessibilityActivationPoint, setAccessibilityAttributedHint, setAccessibilityAttributedLabel, setAccessibilityAttributedUserInputLabels, setAccessibilityAttributedValue, setAccessibilityContainerType, setAccessibilityCustomActions, setAccessibilityCustomRotors, setAccessibilityDragSourceDescriptors, setAccessibilityDropPointDescriptors, setAccessibilityElements, setAccessibilityElementsHidden, setAccessibilityFrame, setAccessibilityHint, setAccessibilityLabel, setAccessibilityLanguage, setAccessibilityNavigationStyle, setAccessibilityPath, setAccessibilityRespondsToUserInteraction, setAccessibilityTextualContext, setAccessibilityTraits, setAccessibilityUserInputLabels, setAccessibilityValue, setAccessibilityViewIsModal, setIsAccessibilityElement, setNilValueForKey, setObservationInfo, setShouldGroupAccessibilityChildren, setValueForKey, setValueForKeyPath, setValueForUndefinedKey, setValuesForKeysWithDictionary, shouldGroupAccessibilityChildren, superclass, validateValueForKeyError, validateValueForKeyPathError, valueForKey, valueForKeyPath, valueForUndefinedKey, willChangeValueForKey, willChangeValueForKeyWithSetMutationUsingObjects, willChangeValuesAtIndexesForKeypublic static boolean accessInstanceVariablesDirectly()
public static MPSTemporaryImage alloc()
public static MPSTemporaryImage allocWithZone(org.moe.natj.general.ptr.VoidPtr zone)
public static boolean automaticallyNotifiesObserversForKey(java.lang.String key)
public static void cancelPreviousPerformRequestsWithTarget(java.lang.Object aTarget)
public static void cancelPreviousPerformRequestsWithTargetSelectorObject(java.lang.Object aTarget,
org.moe.natj.objc.SEL aSelector,
java.lang.Object anArgument)
public static NSArray<java.lang.String> classFallbacksForKeyedArchiver()
public static org.moe.natj.objc.Class classForKeyedUnarchiver()
public static java.lang.String debugDescription_static()
public static java.lang.String description_static()
public static long hash_static()
public static NSObject.Function_instanceMethodForSelector_ret instanceMethodForSelector(org.moe.natj.objc.SEL aSelector)
public static NSMethodSignature instanceMethodSignatureForSelector(org.moe.natj.objc.SEL aSelector)
public static boolean instancesRespondToSelector(org.moe.natj.objc.SEL aSelector)
public static boolean isSubclassOfClass(org.moe.natj.objc.Class aClass)
public static NSSet<java.lang.String> keyPathsForValuesAffectingValueForKey(java.lang.String key)
public static MPSTemporaryImage new_objc()
public static void prefetchStorageWithCommandBufferImageDescriptorList(MTLCommandBuffer commandBuffer, NSArray<? extends MPSImageDescriptor> descriptorList)
The texture cache that underlies the MPSTemporaryImage can automatically allocate new storage as needed as you create new temporary images. However, sometimes a more global view of what you plan to make is useful for maximizing memory reuse to get the most efficient operation. This class method hints to the cache what the list of images will be.
It is never necessary to call this method. It is purely a performance and memory optimization.
This method makes a conservative estimate of memory required and may not fully cover temporary memory needs, resulting in additional allocations later that could encounter pathological behavior, if they are too small. If the full extent and timing of the workload is known in advance, it is recommended that MPSHintTemporaryMemoryHighWaterMark() be used instead.
commandBuffer - The command buffer on which the MPSTemporaryImages will be useddescriptorList - A NSArray of MPSImageDescriptors, indicating images that will be createdpublic static boolean resolveClassMethod(org.moe.natj.objc.SEL sel)
public static boolean resolveInstanceMethod(org.moe.natj.objc.SEL sel)
public static void setVersion_static(long aVersion)
public static org.moe.natj.objc.Class superclass_static()
public static MPSTemporaryImage temporaryImageWithCommandBufferImageDescriptor(MTLCommandBuffer commandBuffer, MPSImageDescriptor imageDescriptor)
commandBuffer - The MTLCommandBuffer on which the MPSTemporaryImage will be exclusively usedimageDescriptor - A valid imageDescriptor describing the MPSImage format to create.public static MPSTemporaryImage temporaryImageWithCommandBufferTextureDescriptor(MTLCommandBuffer commandBuffer, MTLTextureDescriptor textureDescriptor)
This function provides access to MTLPixelFormats not typically covered by -initForCommandBuffer:imageDescriptor: The feature channels will be inferred from the MTLPixelFormat without changing the width. The following restrictions apply:
MTLTextureType must be MTLTextureType2D or MTLTextureType2DArray MTLTextureUsage must contain at least one of MTLTextureUsageShaderRead, MTLTextureUsageShaderWrite MTLStorageMode must be MTLStorageModePrivate depth must be 1
commandBuffer - The command buffer on which the MPSTemporaryImage may be usedtextureDescriptor - A texture descriptor describing the MPSTemporaryImage texturepublic static long version_static()
public MPSTemporaryImage init()
public MPSTemporaryImage initWithDeviceImageDescriptor(MTLDevice device, MPSImageDescriptor imageDescriptor)
MPSImageStorage to store data needed is allocated lazily on first use of MPSImage or when application calls MPSImage.texture
initWithDeviceImageDescriptor in class MPSImagedevice - The device that the image will be used. May not be NULL.imageDescriptor - The MPSImageDescriptor. May not be NULL.public MPSTemporaryImage initWithTextureFeatureChannels(MTLTexture texture, long featureChannels)
MPSImageApplication can let MPS framework allocate texture with properties specified in imageDescriptor using initWithDevice:MPSImageDescriptor API above. However in memory intensive application, you can save memory (and allocation/deallocation time) by using MPSTemporaryImage where MPS framework aggressively reuse memory underlying textures on same command buffer. See MPSTemporaryImage class for details below. However, in certain cases, application developer may want more control on allocation, placement, reusing/recycling of memory backing textures used in application using Metal Heaps API. In this case, application can create MPSImage from pre-allocated texture using initWithTexture:featureChannels.
MTLTextureType of texture can be MTLTextureType2D ONLY if featureChannels <= 4 in which case MPSImage.numberOfImages is set to 1. Else it should be MTLTextureType2DArray with arrayLength == numberOfImage * ((featureChannels + 3)/4). MPSImage.numberOfImages is set to texture.arrayLength / ((featureChannels + 3)/4).
For MTLTextures containing typical image data which application may obtain from MetalKit or other libraries such as that drawn from a JPEG or PNG, featureChannels should be set to number of valid color channel e.g. for RGB data, even thought MTLPixelFormat will be MTLPixelFormatRGBA, featureChannels should be set to 3.
initWithTextureFeatureChannels in class MPSImagetexture - The MTLTexture allocated by the user to be used as backing for MPSImage.featureChannels - Number of feature channels this texture contains.public long readCount()
MPSTemporaryImages must release their underlying textures for reuse immediately after last use. So as to facilitate *prompt* convenient memory recycling, each time a MPSTemporaryImage is read by a MPSCNNKernel -encode... method, its readCount is automatically decremented. When the readCount reaches 0, the underlying texture is automatically made available for reuse to MPS for its own needs and for other MPSTemporaryImages prior to return from the -encode.. function. The contents of the texture become undefined at this time.
By default, the readCount is initialized to 1, indicating a image that may be overwritten any number of times, but read only once.
You may change the readCount as desired to allow MPSCNNKernels to read the MPSTemporaryImage additional times. However, it is an error to change the readCount once it is zero. It is an error to read or write to a MPSTemporaryImage with a zero readCount. You may set the readCount to 0 yourself to cause the underlying texture to be returned to MPS. Writing to a MPSTemporaryImage does not adjust the readCount.
The Metal API Validation layer will assert if a MPSTemporaryImage is deallocated with non-zero readCount to help identify cases when resources are not returned promptly.
public void setReadCount(long value)
MPSTemporaryImages must release their underlying textures for reuse immediately after last use. So as to facilitate *prompt* convenient memory recycling, each time a MPSTemporaryImage is read by a MPSCNNKernel -encode... method, its readCount is automatically decremented. When the readCount reaches 0, the underlying texture is automatically made available for reuse to MPS for its own needs and for other MPSTemporaryImages prior to return from the -encode.. function. The contents of the texture become undefined at this time.
By default, the readCount is initialized to 1, indicating a image that may be overwritten any number of times, but read only once.
You may change the readCount as desired to allow MPSCNNKernels to read the MPSTemporaryImage additional times. However, it is an error to change the readCount once it is zero. It is an error to read or write to a MPSTemporaryImage with a zero readCount. You may set the readCount to 0 yourself to cause the underlying texture to be returned to MPS. Writing to a MPSTemporaryImage does not adjust the readCount.
The Metal API Validation layer will assert if a MPSTemporaryImage is deallocated with non-zero readCount to help identify cases when resources are not returned promptly.
public static MPSImageAllocator defaultAllocator()
public MPSTemporaryImage initWithParentImageSliceRangeFeatureChannels(MPSImage parent, NSRange sliceRange, long featureChannels)
MPSImageGenerally, you should call -batchRepresentation or -subImageWithFeatureChannelRange instead because they are safer. This is provided so that these interfaces will work with your MPSImage subclass.
initWithParentImageSliceRangeFeatureChannels in class MPSImageparent - The parent image that owns the texture. It may be a sub-image.sliceRange - The range of MTLTexture2dArray slices to be included in the sub-imagefeatureChannels - The number of feature channels in the new image. The number of images
is inferred.public static MPSTemporaryImage temporaryImageWithCommandBufferTextureDescriptorFeatureChannels(MTLCommandBuffer commandBuffer, MTLTextureDescriptor textureDescriptor, long featureChannels)
This function provides access to MTLPixelFormats not typically covered by -initForCommandBuffer:imageDescriptor: The number of images will be inferred from number of slices in the descriptor.arrayLength and the number of feature channels.
The following restrictions apply:
MTLTextureType must be MTLTextureType2D or MTLTextureType2DArray MTLTextureUsage must contain at least one of MTLTextureUsageShaderRead, MTLTextureUsageShaderWrite MTLStorageMode must be MTLStorageModePrivate
commandBuffer - The command buffer on which the MPSTemporaryImage may be usedtextureDescriptor - A texture descriptor describing the MPSTemporaryImage texture