public class MPSCNNLoss extends MPSCNNKernel
The MPSCNNLoss filter is only used for training. This filter performs both the forward and backward pass computations. Specifically, it computes the loss between the input (predictions) and target data (labels) and the loss gradient. The loss value can be a 1 x 1 x 1 image containing a scalar loss value or an image (of the same size as the input source image) with per feature channel losses. The loss value is used to determine whether to continue the training operation or to terminate it, once satisfactory results are achieved. The loss gradient is the first gradient computed for the backward pass and serves as input to the next gradient filter (in the backward direction).
The MPSCNNLoss filter is created with a MPSCNNLossDescriptor describing the type of a loss filter and the type of a reduction to use for computing the overall loss.
The MPSCNNLoss filter takes the output of the inference pass (predictions) as input. It also requires the target data (labels) and optionally, weights for the labels. If per-label weights are not supplied, there is an option to use a single weight value by setting the 'weight' properly on the MPSCNNLossDescriptor object. The labels and optional weights need to be supplied by the user using the MPSCNNLossLabels object. The labels and weights are described via the MPSCNNLossDataDescriptor objects, which are in turn used to initialize the MPSCNNLossLabels object.
If the specified reduction operation is MPSCNNReductionTypeNone, the destinationImage should be at least as large as the specified clipRect. The destinationImage will then contain per-element losses. Otherse, a reduction operation will be performed, according to the specified reduction type, and the filter will return a scalar value containing the overall loss. For more information on the available reduction types, see MPSCNNTypes.h. Also see MPSCNNLossDescriptor for the description of optional parameters.
Here is a code example:
// Setup MPSCNNLossDataDescriptor* labelsDescriptor = [MPSCNNLossDataDescriptor cnnLossDataDescriptorWithData: labelsData layout: MPSDataLayoutHeightxWidthxFeatureChannels size: labelsDataSize]; MPSCNNLossLabels* labels = [[MPSCNNLossLabels alloc] initWithDevice: device labelsDescriptor: labelsDescriptor]; MPSCNNLossDescriptor *lossDescriptor = [MPSCNNLossDescriptor cnnLossDescriptorWithType: (MPSCNNLossType)MPSCNNLossTypeMeanAbsoluteError reductionType: (MPSCNNReductionType)MPSCNNReductionTypeSum]; MPSCNNLoss* lossFilter = [[MPSCNNLoss alloc] initWithDevice: device lossDescriptor: lossDescriptor];
// Encode loss filter. // The sourceImage is the output of a previous layer, for example, the SoftMax layer. The lossGradientsImage // is the sourceGradient input image to the first gradient layer (in the backward direction), for example, // the SoftMax gradient filter. [lossFilter encodeToCommandBuffer: commandBuffer sourceImage: sourceImage labels: labels destinationImage: lossGradientsImage];
// In order to guarantee that the loss image data is correctly synchronized for CPU side access, // it is the application's responsibility to call the [labels synchronizeOnCommandBuffer:] // method before accessing the loss image data. [labels synchronizeOnCommandBuffer:commandBuffer]; MPSImage* lossImage = [labels lossImage];
For predictions (y) and labels (t), the available loss filter types are the following:
Mean Absolute Error loss filter. This filter measures the absolute error of the element-wise difference between the predictions and labels. This loss function is computed according to the following formulas: Compute losses: losses = |y - t| Compute weighted losses: weighted_losses = weight(s) * losses Compute overall loss: loss = reduce(weighted_losses, reductionType)
Mean Squared Error loss filter. This filter measures the squared error of the element-wise difference between the predictions and labels. This loss function is computed according to the following formulas: Compute losses: losses = (y - t)^2 Compute weighted losses: weighted_losses = weight(s) * losses Compute overall loss: loss = reduce(weighted_losses, reductionType)
SoftMax Cross Entropy loss filter. This loss filter is applied element-wise. This loss filter combines the LogSoftMax and Negative Log Likelihood operations in a single filter. It is useful for training a classification problem with multiple classes. This loss function is computed according to the following formulas: Compute losses: losses = -t * LogSoftMax(y) Compute weighted losses: weighted_losses = weight(s) * losses Compute overall loss: loss = reduce(weighted_losses, reductionType) If reductionType is MPSCNNReductionTypeMean, the accumulated loss value is divided by width * height instead of width * height * featureChannels.
Sigmoid Cross Entropy loss filter. This loss filter is applied element-wise. This loss function is computed according to the following formulas: Compute losses: losses = max(y, 0) - y * t + log(1 + exp(-|y|)) Compute weighted losses: weighted_losses = weight(s) * losses Compute overall loss: loss = reduce(weighted_losses, reductionType)
Categorical Cross Entropy loss filter. This loss filter is applied element-wise. This loss function is computed according to the following formulas: Compute losses: losses = -t * log(y) Compute weighted losses: weighted_losses = weight(s) * losses Compute overall loss: loss = reduce(weighted_losses, reductionType)
Hinge loss filter. This loss filter is applied element-wise. The labels are expected to be 0.0 or 1.0. This loss function is computed according to the following formulas: Compute losses: losses = max(1 - (t * y), 0.0f) Compute weighted losses: weighted_losses = weight(s) * losses Compute overall loss: loss = reduce(weighted_losses, reductionType)
Huber loss filter. This loss filter is applied element-wise. This loss function is computed according to the following formulas: Compute losses: if (|y - t| <= delta, losses = 0.5 * y^2 if (|y - t| > delta, losses = 0.5 * delta^2 + delta * (|y - t| - delta) Compute weighted losses: weighted_losses = weight(s) * losses Compute overall loss: loss = reduce(weighted_losses, reductionType)
Cosine Distance loss filter. This loss filter is applied element-wise. The only valid reduction type for this loss filter is MPSCNNReductionTypeSum. This loss function is computed according to the following formulas: Compute losses: loss = 1 - reduce_sum(y * t) Compute overall loss: weighted_loss = weight * loss
Log loss filter. This loss filter is applied element-wise. This loss function is computed according to the following formulas: Compute losses: losses = -(t * log(y + epsilon)) - ((1 - t) * log(1 - y + epsilon)) Compute weighted losses: weighted_losses = weight(s) * losses Compute overall loss: loss = reduce(weighted_losses, reductionType)
Kullback-Leibler Divergence loss filter. This loss filter is applied element-wise. The input (predictions) is expected to contain log-probabilities. This loss function is computed according to the following formulas: Compute losses: losses = t * (log(t) - y) Compute weighted losses: weighted_losses = weight(s) * losses Compute overall loss: loss = reduce(weighted_losses, reductionType)
For predictions (y) and labels (t), the loss gradient for each available loss filter type is computed as follows:
Mean Absolute Error loss. The loss gradient is computed according to the following formulas: Compute gradient: d/dy = (y - t) / |y - t| Compute weighted gradient: weighted_gradient = weight(s) * gradient
Mean Squared Error loss. The loss gradient is computed according to the following formulas: Compute gradient: d/dy = 2 * (y - t) Compute weighted gradient: weighted_gradient = weight(s) * gradient
SoftMax Cross Entropy loss. The loss gradient is computed according to the following formulas: First, apply the same label smoothing as in the MPSCNNLoss filter. Compute gradient: d/dy = y - t Compute weighted gradient: weighted_gradient = weight(s) * gradient
Sigmoid Cross Entropy loss. The loss gradient is computed according to the following formulas: First, apply the same label smoothing as in the MPSCNNLoss filter. Compute gradient: d/dy = (1 / (1 + exp(-y)) - t Compute weighted gradient: weighted_gradient = weight(s) * gradient
Categorical Cross Entropy loss. The loss gradient is computed according to the following formulas: Compute gradient: d/dy = -t / y Compute weighted gradient: weighted_gradient = weight(s) * gradient
Hinge loss. The loss gradient is computed according to the following formulas: Compute gradient: d/dy = ((1 + ((1 - (2 * t)) * y)) > 0) ? 1 - (2 * t) : 0 Compute weighted gradient: weighted_gradient = weight(s) * gradient
Huber loss. The loss gradient is computed according to the following formulas: Compute gradient: d/dy = |y - t| > delta ? delta : y - t Compute weighted gradient: weighted_gradient = weight(s) * gradient
Cosine Distance loss. The loss gradient is computed according to the following formulas: Compute gradient: d/dy = -t Compute weighted gradient: weighted_gradient = weight(s) * gradient
Log loss. The loss gradient is computed according to the following formulas: Compute gradient: d/dy = (-2 * epsilon * t - t + y + epsilon) / (y * (1 - y) + epsilon * (epsilon + 1)) Compute weighted gradient: weighted_gradient = weight(s) * gradient
Kullback-Leibler Divergence loss. The loss gradient is computed according to the following formulas: Compute gradient: d/dy = -t / y Compute weighted gradient: weighted_gradient = weight(s) * gradient
The number of output feature channels remains the same as the number of input feature channels.
NSObject.Function_instanceMethodForSelector_ret, NSObject.Function_methodForSelector_ret| Modifier | Constructor and Description |
|---|---|
protected |
MPSCNNLoss(org.moe.natj.general.Pointer peer) |
| Modifier and Type | Method and Description |
|---|---|
boolean |
_supportsSecureCoding()
This property must return YES on all classes that allow secure coding.
|
static boolean |
accessInstanceVariablesDirectly() |
static MPSCNNLoss |
alloc() |
static MPSCNNLoss |
allocWithZone(org.moe.natj.general.ptr.VoidPtr zone) |
static boolean |
automaticallyNotifiesObserversForKey(java.lang.String key) |
static void |
cancelPreviousPerformRequestsWithTarget(java.lang.Object aTarget) |
static void |
cancelPreviousPerformRequestsWithTargetSelectorObject(java.lang.Object aTarget,
org.moe.natj.objc.SEL aSelector,
java.lang.Object anArgument) |
static NSArray<java.lang.String> |
classFallbacksForKeyedArchiver() |
static org.moe.natj.objc.Class |
classForKeyedUnarchiver() |
static java.lang.String |
debugDescription_static() |
float |
delta() |
static java.lang.String |
description_static() |
MPSImage |
encodeToCommandBufferSourceImageLabels(MTLCommandBuffer commandBuffer,
MPSImage sourceImage,
MPSCNNLossLabels labels)
Encode a MPSCNNLoss filter and return a gradient.
|
void |
encodeToCommandBufferSourceImageLabelsDestinationImage(MTLCommandBuffer commandBuffer,
MPSImage sourceImage,
MPSCNNLossLabels labels,
MPSImage destinationImage)
Encode a MPSCNNLoss filter and return a gradient in the destinationImage.
|
float |
epsilon() |
static long |
hash_static() |
MPSCNNLoss |
init() |
MPSCNNLoss |
initWithCoder(NSCoder aDecoder)
NS_DESIGNATED_INITIALIZER
|
MPSCNNLoss |
initWithCoderDevice(NSCoder aDecoder,
java.lang.Object device)
|
MPSCNNLoss |
initWithDevice(java.lang.Object device)
Standard init with default properties per filter type
|
MPSCNNLoss |
initWithDeviceLossDescriptor(MTLDevice device,
MPSCNNLossDescriptor lossDescriptor)
Initialize the loss filter with a loss descriptor.
|
static NSObject.Function_instanceMethodForSelector_ret |
instanceMethodForSelector(org.moe.natj.objc.SEL aSelector) |
static NSMethodSignature |
instanceMethodSignatureForSelector(org.moe.natj.objc.SEL aSelector) |
static boolean |
instancesRespondToSelector(org.moe.natj.objc.SEL aSelector) |
static boolean |
isSubclassOfClass(org.moe.natj.objc.Class aClass) |
static NSSet<java.lang.String> |
keyPathsForValuesAffectingValueForKey(java.lang.String key) |
float |
labelSmoothing() |
int |
lossType()
See MPSCNNLossDescriptor for information about the following properties.
|
static MPSCNNLoss |
new_objc() |
long |
numberOfClasses() |
boolean |
reduceAcrossBatch() |
int |
reductionType() |
static boolean |
resolveClassMethod(org.moe.natj.objc.SEL sel) |
static boolean |
resolveInstanceMethod(org.moe.natj.objc.SEL sel) |
static void |
setVersion_static(long aVersion) |
static org.moe.natj.objc.Class |
superclass_static() |
static boolean |
supportsSecureCoding() |
static long |
version_static() |
float |
weight() |
appendBatchBarrier, clipRect, destinationFeatureChannelOffset, destinationImageAllocator, destinationImageDescriptorForSourceImagesSourceStates, dilationRateX, dilationRateY, edgeMode, encodeToCommandBufferSourceImage, encodeToCommandBufferSourceImageDestinationImage, encodeToCommandBufferSourceImageDestinationStateDestinationImage, encodeToCommandBufferSourceImageDestinationStateDestinationStateIsTemporary, encodingStorageSizeForSourceImageSourceStatesDestinationImage, isBackwards, isResultStateReusedAcrossBatch, isStateModified, kernelHeight, kernelWidth, offset, padding, resultStateForSourceImageSourceStatesDestinationImage, setClipRect, setDestinationFeatureChannelOffset, setDestinationImageAllocator, setEdgeMode, setOffset, setPadding, setSourceFeatureChannelMaxCount, setSourceFeatureChannelOffset, sourceFeatureChannelMaxCount, sourceFeatureChannelOffset, strideInPixelsX, strideInPixelsY, temporaryResultStateForCommandBufferSourceImageSourceStatesDestinationImagecopyWithZone, copyWithZoneDevice, device, encodeWithCoder, label, options, setLabel, setOptionsaccessibilityActivate, accessibilityActivationPoint, accessibilityAssistiveTechnologyFocusedIdentifiers, accessibilityAttributedHint, accessibilityAttributedLabel, accessibilityAttributedUserInputLabels, accessibilityAttributedValue, accessibilityContainerType, accessibilityCustomActions, accessibilityCustomRotors, accessibilityDecrement, accessibilityDragSourceDescriptors, accessibilityDropPointDescriptors, accessibilityElementAtIndex, accessibilityElementCount, accessibilityElementDidBecomeFocused, accessibilityElementDidLoseFocus, accessibilityElementIsFocused, accessibilityElements, accessibilityElementsHidden, accessibilityFrame, accessibilityHint, accessibilityIncrement, accessibilityLabel, accessibilityLanguage, accessibilityNavigationStyle, accessibilityPath, accessibilityPerformEscape, accessibilityPerformMagicTap, accessibilityRespondsToUserInteraction, accessibilityScroll, accessibilityTextualContext, accessibilityTraits, accessibilityUserInputLabels, accessibilityValue, accessibilityViewIsModal, addObserverForKeyPathOptionsContext, attemptRecoveryFromErrorOptionIndex, attemptRecoveryFromErrorOptionIndexDelegateDidRecoverSelectorContextInfo, autoContentAccessingProxy, awakeAfterUsingCoder, awakeFromNib, class_objc, classForCoder, classForKeyedArchiver, copy, dealloc, debugDescription, description, dictionaryWithValuesForKeys, didChangeValueForKey, didChangeValueForKeyWithSetMutationUsingObjects, didChangeValuesAtIndexesForKey, doesNotRecognizeSelector, fileManagerShouldProceedAfterError, fileManagerWillProcessPath, finalize_objc, forwardingTargetForSelector, forwardInvocation, hash, indexOfAccessibilityElement, isAccessibilityElement, isEqual, isKindOfClass, isMemberOfClass, isProxy, methodForSelector, methodSignatureForSelector, mutableArrayValueForKey, mutableArrayValueForKeyPath, mutableCopy, mutableOrderedSetValueForKey, mutableOrderedSetValueForKeyPath, mutableSetValueForKey, mutableSetValueForKeyPath, observationInfo, observeValueForKeyPathOfObjectChangeContext, performSelector, performSelectorInBackgroundWithObject, performSelectorOnMainThreadWithObjectWaitUntilDone, performSelectorOnMainThreadWithObjectWaitUntilDoneModes, performSelectorOnThreadWithObjectWaitUntilDone, performSelectorOnThreadWithObjectWaitUntilDoneModes, performSelectorWithObject, performSelectorWithObjectAfterDelay, performSelectorWithObjectAfterDelayInModes, performSelectorWithObjectWithObject, prepareForInterfaceBuilder, provideImageDataBytesPerRowOrigin_Size_UserInfo, removeObserverForKeyPath, removeObserverForKeyPathContext, replacementObjectForCoder, replacementObjectForKeyedArchiver, respondsToSelector, self, setAccessibilityActivationPoint, setAccessibilityAttributedHint, setAccessibilityAttributedLabel, setAccessibilityAttributedUserInputLabels, setAccessibilityAttributedValue, setAccessibilityContainerType, setAccessibilityCustomActions, setAccessibilityCustomRotors, setAccessibilityDragSourceDescriptors, setAccessibilityDropPointDescriptors, setAccessibilityElements, setAccessibilityElementsHidden, setAccessibilityFrame, setAccessibilityHint, setAccessibilityLabel, setAccessibilityLanguage, setAccessibilityNavigationStyle, setAccessibilityPath, setAccessibilityRespondsToUserInteraction, setAccessibilityTextualContext, setAccessibilityTraits, setAccessibilityUserInputLabels, setAccessibilityValue, setAccessibilityViewIsModal, setIsAccessibilityElement, setNilValueForKey, setObservationInfo, setShouldGroupAccessibilityChildren, setValueForKey, setValueForKeyPath, setValueForUndefinedKey, setValuesForKeysWithDictionary, shouldGroupAccessibilityChildren, superclass, validateValueForKeyError, validateValueForKeyPathError, valueForKey, valueForKeyPath, valueForUndefinedKey, willChangeValueForKey, willChangeValueForKeyWithSetMutationUsingObjects, willChangeValuesAtIndexesForKeypublic static boolean accessInstanceVariablesDirectly()
public static MPSCNNLoss alloc()
public static MPSCNNLoss allocWithZone(org.moe.natj.general.ptr.VoidPtr zone)
public static boolean automaticallyNotifiesObserversForKey(java.lang.String key)
public static void cancelPreviousPerformRequestsWithTarget(java.lang.Object aTarget)
public static void cancelPreviousPerformRequestsWithTargetSelectorObject(java.lang.Object aTarget,
org.moe.natj.objc.SEL aSelector,
java.lang.Object anArgument)
public static NSArray<java.lang.String> classFallbacksForKeyedArchiver()
public static org.moe.natj.objc.Class classForKeyedUnarchiver()
public static java.lang.String debugDescription_static()
public float delta()
public static java.lang.String description_static()
public MPSImage encodeToCommandBufferSourceImageLabels(MTLCommandBuffer commandBuffer, MPSImage sourceImage, MPSCNNLossLabels labels)
This -encode call is similar to the encodeToCommandBuffer:sourceImage:labels:destinationImage: above, except that it creates and returns the MPSImage with the loss gradient result.
commandBuffer - The MTLCommandBuffer on which to encode.sourceImage - The source image from the previous filter in the graph (in the inference direction).labels - The object containing the target data (labels) and optionally, weights for the labels.public void encodeToCommandBufferSourceImageLabelsDestinationImage(MTLCommandBuffer commandBuffer, MPSImage sourceImage, MPSCNNLossLabels labels, MPSImage destinationImage)
This filter consumes the output of a previous layer, for example, the SoftMax layer containing predictions, and the MPSCNNLossLabels object containing the target data (labels) and optionally, weights for the labels. The destinationImage contains the computed gradient for the loss layer. It serves as a source gradient input image to the first gradient layer (in the backward direction), in our example, the SoftMax gradient layer.
commandBuffer - The MTLCommandBuffer on which to encode.sourceImage - The source image from the previous filter in the graph (in the inference direction).labels - The object containing the target data (labels) and optionally, weights for the labels.destinationImage - The MPSImage into which to write the gradient result.public float epsilon()
public static long hash_static()
public MPSCNNLoss init()
init in class MPSCNNKernelpublic MPSCNNLoss initWithCoder(NSCoder aDecoder)
NSCodinginitWithCoder in interface NSCodinginitWithCoder in class MPSCNNKernelpublic MPSCNNLoss initWithCoderDevice(NSCoder aDecoder, java.lang.Object device)
initWithCoderDevice in class MPSCNNKernelaDecoder - The NSCoder subclass with your serialized MPSKerneldevice - The MTLDevice on which to make the MPSKernelpublic MPSCNNLoss initWithDevice(java.lang.Object device)
MPSCNNKernelinitWithDevice in class MPSCNNKerneldevice - The device that the filter will be used on. May not be NULL.public MPSCNNLoss initWithDeviceLossDescriptor(MTLDevice device, MPSCNNLossDescriptor lossDescriptor)
device - The device the filter will run on.lossDescriptor - The loss descriptor.public static NSObject.Function_instanceMethodForSelector_ret instanceMethodForSelector(org.moe.natj.objc.SEL aSelector)
public static NSMethodSignature instanceMethodSignatureForSelector(org.moe.natj.objc.SEL aSelector)
public static boolean instancesRespondToSelector(org.moe.natj.objc.SEL aSelector)
public static boolean isSubclassOfClass(org.moe.natj.objc.Class aClass)
public static NSSet<java.lang.String> keyPathsForValuesAffectingValueForKey(java.lang.String key)
public float labelSmoothing()
public int lossType()
public static MPSCNNLoss new_objc()
public long numberOfClasses()
public int reductionType()
public static boolean resolveClassMethod(org.moe.natj.objc.SEL sel)
public static boolean resolveInstanceMethod(org.moe.natj.objc.SEL sel)
public static void setVersion_static(long aVersion)
public static org.moe.natj.objc.Class superclass_static()
public static boolean supportsSecureCoding()
public boolean _supportsSecureCoding()
NSSecureCoding_supportsSecureCoding in interface NSSecureCoding_supportsSecureCoding in class MPSCNNKernelpublic static long version_static()
public float weight()
public boolean reduceAcrossBatch()