IExecutionContext (JavaCPP Presets for TensorRT 8.6-1.5.10 API)

java.lang.Object
- org.bytedeco.javacpp.Pointer
- - org.bytedeco.tensorrt.nvinfer.INoCopy
  - - org.bytedeco.tensorrt.nvinfer.IExecutionContext

All Implemented Interfaces:

AutoCloseable
```
@Namespace(value="nvinfer1")
 @NoOffset
 @Properties(inherit=nvinfer.class)
public class IExecutionContext
extends INoCopy
```
\class IExecutionContext \brief Context for executing inference using an engine, with functionally unsafe features. Multiple execution contexts may exist for one ICudaEngine instance, allowing the same engine to be used for the execution of multiple batches simultaneously. If the engine supports dynamic shapes, each execution context in concurrent use must use a separate optimization profile. \warning Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Nested Class Summary
- Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer
  Pointer.CustomDeallocator, Pointer.Deallocator, Pointer.NativeDeallocator, Pointer.ReferenceCounter

Field Summary
- Fields inherited from class org.bytedeco.javacpp.Pointer
  address, capacity, limit, position

Constructor Summary

Constructors
Constructor and Description
`IExecutionContext()` Default native constructor.
`IExecutionContext(long size)` Native array allocator.
`IExecutionContext(Pointer p)` Pointer cast constructor.

Method Summary

All Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`boolean`	`allInputDimensionsSpecified()` \brief Whether all dynamic dimensions of input tensors have been specified
`boolean`	`allInputShapesSpecified()` \brief Whether all input shape bindings have been specified
`void`	`destroy()` Deprecated. Deprecated in TRT 8.0. Superseded by `delete`. \warning Calling destroy on a managed pointer will result in a double-free error.
`boolean`	`enqueue(int batchSize, Pointer bindings, org.bytedeco.cuda.cudart.CUstream_st stream, org.bytedeco.cuda.cudart.CUevent_st inputConsumed)` Deprecated.
`boolean`	`enqueue(int batchSize, PointerPointer bindings, org.bytedeco.cuda.cudart.CUstream_st stream, org.bytedeco.cuda.cudart.CUevent_st inputConsumed)` Deprecated. Deprecated in TensorRT 8.4. Superseded by enqueueV2() if the network is created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag.
`boolean`	`enqueueV2(Pointer bindings, org.bytedeco.cuda.cudart.CUstream_st stream, org.bytedeco.cuda.cudart.CUevent_st inputConsumed)` Deprecated.
`boolean`	`enqueueV2(PointerPointer bindings, org.bytedeco.cuda.cudart.CUstream_st stream, org.bytedeco.cuda.cudart.CUevent_st inputConsumed)` Deprecated. Superseded by enqueueV3(). Deprecated in TensorRT 8.5
`boolean`	`enqueueV3(org.bytedeco.cuda.cudart.CUstream_st stream)` \brief Enqueue inference on a stream.
`boolean`	`execute(int batchSize, Pointer bindings)` Deprecated.
`boolean`	`execute(int batchSize, PointerPointer bindings)` Deprecated. Deprecated in TensorRT 8.4. Superseded by executeV2() if the network is created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. \warning This function will trigger layer resource updates if hasImplicitBatchDimension() returns true and batchSize changes between subsequent calls, possibly resulting in performance bottlenecks.
`boolean`	`executeV2(Pointer bindings)`
`boolean`	`executeV2(PointerPointer bindings)` \brief Synchronously execute inference a network.
`Dims32`	`getBindingDimensions(int bindingIndex)` Deprecated. Deprecated in TensorRT 8.5. Superseded by getTensorShape().
`boolean`	`getDebugSync()` \brief Get the debug sync flag.
`ICudaEngine`	`getEngine()` \brief Get the associated engine.
`boolean`	`getEnqueueEmitsProfile()` \brief Get the enqueueEmitsProfile state.
`IErrorRecorder`	`getErrorRecorder()` \brief Get the ErrorRecorder assigned to this interface.
`org.bytedeco.cuda.cudart.CUevent_st`	`getInputConsumedEvent()` \brief The event associated with consuming the input.
`long`	`getMaxOutputSize(BytePointer tensorName)`
`long`	`getMaxOutputSize(String tensorName)` \brief Get upper bound on an output tensor's size, in bytes, based on the current optimization profile and input dimensions.
`String`	`getName()` \brief Return the name of the execution context.
`nvinfer.ProfilingVerbosity`	`getNvtxVerbosity()` \brief Get the NVTX verbosity of the execution context.
`int`	`getOptimizationProfile()` \brief Get the index of the currently selected optimization profile.
`IOutputAllocator`	`getOutputAllocator(BytePointer tensorName)`
`IOutputAllocator`	`getOutputAllocator(String tensorName)` \brief Get output allocator associated with output tensor of given name, or nullptr if the provided name does not map to an output tensor.
`Pointer`	`getOutputTensorAddress(BytePointer tensorName)`
`Pointer`	`getOutputTensorAddress(String tensorName)` \brief Get memory address for given output.
`long`	`getPersistentCacheLimit()` \brief Get the maximum size for persistent cache usage.
`IExecutionContext`	`getPointer(long i)`
`IProfiler`	`getProfiler()` \brief Get the profiler.
`boolean`	`getShapeBinding(int bindingIndex, int[] data)` Deprecated.
`boolean`	`getShapeBinding(int bindingIndex, IntBuffer data)` Deprecated.
`boolean`	`getShapeBinding(int bindingIndex, IntPointer data)` Deprecated. Deprecated in TensorRT 8.5. Superseded by getTensorAddress() or getOutputTensorAddress().
`Dims32`	`getStrides(int bindingIndex)` Deprecated. Deprecated in TensorRT 8.5. Superseded by getTensorStrides().
`IGpuAllocator`	`getTemporaryStorageAllocator()` \brief Get allocator set by setTemporaryStorageAllocator.
`Pointer`	`getTensorAddress(BytePointer tensorName)`
`Pointer`	`getTensorAddress(String tensorName)` \brief Get memory address bound to given input or output tensor, or nullptr if the provided name does not map to an input or output tensor.
`Dims32`	`getTensorShape(BytePointer tensorName)`
`Dims32`	`getTensorShape(String tensorName)` \brief Return the shape of the given input or output.
`Dims32`	`getTensorStrides(BytePointer tensorName)`
`Dims32`	`getTensorStrides(String tensorName)` \brief Return the strides of the buffer for the given tensor name.
`int`	`inferShapes(int nbMaxNames, byte[] tensorNames)`
`int`	`inferShapes(int nbMaxNames, ByteBuffer tensorNames)`
`int`	`inferShapes(int nbMaxNames, BytePointer tensorNames)`
`int`	`inferShapes(int nbMaxNames, PointerPointer tensorNames)` \brief Run shape calculations.
`IExecutionContext`	`position(long position)`
`boolean`	`reportToProfiler()` \brief Calculate layer timing info for the current optimization profile in IExecutionContext and update the profiler after one iteration of inference launch.
`void`	`setAuxStreams(org.bytedeco.cuda.cudart.CUstream_st auxStreams, int nbStreams)` \brief Set the auxiliary streams that TensorRT should launch kernels on in the next enqueueV3() call.
`boolean`	`setBindingDimensions(int bindingIndex, Dims32 dimensions)` Deprecated. Deprecated in TensorRT 8.5. Superseded by setInputShape().
`void`	`setDebugSync(boolean sync)` \brief Set the debug sync flag.
`void`	`setDeviceMemory(Pointer memory)` \brief Set the device memory for use by this execution context.
`void`	`setEnqueueEmitsProfile(boolean enqueueEmitsProfile)` \brief Set whether enqueue emits layer timing to the profiler If set to true (default), enqueue is synchronous and does layer timing profiling implicitly if there is a profiler attached.
`void`	`setErrorRecorder(IErrorRecorder recorder)`
`boolean`	`setInputConsumedEvent(org.bytedeco.cuda.cudart.CUevent_st event)` \brief Mark input as consumed.
`boolean`	`setInputShape(BytePointer tensorName, Dims32 dims)`
`boolean`	`setInputShape(String tensorName, Dims32 dims)` \brief Set shape of given input.
`boolean`	`setInputShapeBinding(int bindingIndex, int[] data)` Deprecated.
`boolean`	`setInputShapeBinding(int bindingIndex, IntBuffer data)` Deprecated.
`boolean`	`setInputShapeBinding(int bindingIndex, IntPointer data)` Deprecated. Deprecated in TensorRT 8.5. Superseded by setInputTensorAddress() or setTensorAddress().
`boolean`	`setInputTensorAddress(BytePointer tensorName, Pointer data)`
`boolean`	`setInputTensorAddress(String tensorName, Pointer data)` \brief Set memory address for given input.
`void`	`setName(BytePointer name)`
`void`	`setName(String name)` \brief Set the name of the execution context.
`boolean`	`setNvtxVerbosity(int verbosity)`
`boolean`	`setNvtxVerbosity(nvinfer.ProfilingVerbosity verbosity)` \brief Set the verbosity of the NVTX markers in the execution context.
`boolean`	`setOptimizationProfile(int profileIndex)` Deprecated. Superseded by setOptimizationProfileAsync. Deprecated prior to TensorRT 8.0 and will be removed in 9.0.
`boolean`	`setOptimizationProfileAsync(int profileIndex, org.bytedeco.cuda.cudart.CUstream_st stream)` \brief Select an optimization profile for the current context with async semantics.
`boolean`	`setOutputAllocator(BytePointer tensorName, IOutputAllocator outputAllocator)`
`boolean`	`setOutputAllocator(String tensorName, IOutputAllocator outputAllocator)` \brief Set output allocator to use for output tensor of given name.
`void`	`setPersistentCacheLimit(long size)` \brief Set the maximum size for persistent cache usage.
`void`	`setProfiler(IProfiler profiler)` \brief Set the profiler.
`boolean`	`setTemporaryStorageAllocator(IGpuAllocator allocator)` \brief Specify allocator to use for internal temporary storage.
`boolean`	`setTensorAddress(BytePointer tensorName, Pointer data)`
`boolean`	`setTensorAddress(String tensorName, Pointer data)` \brief Set memory address for given input or output tensor.

Methods inherited from class org.bytedeco.javacpp.Pointer
address, asBuffer, asByteBuffer, availablePhysicalBytes, calloc, capacity, capacity, close, deallocate, deallocate, deallocateReferences, deallocator, deallocator, equals, fill, formatBytes, free, getDirectBufferAddress, getPointer, getPointer, getPointer, hashCode, interruptDeallocatorThread, isNull, isNull, limit, limit, malloc, maxBytes, maxPhysicalBytes, memchr, memcmp, memcpy, memmove, memset, offsetAddress, offsetof, offsetof, parseBytes, physicalBytes, physicalBytesInaccurate, position, put, realloc, referenceCount, releaseReference, retainReference, setNull, sizeof, sizeof, toString, totalBytes, totalCount, totalPhysicalBytes, withDeallocator, zero

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

- Constructor Detail
  - IExecutionContext
```
public IExecutionContext()
```
    Default native constructor.
  - IExecutionContext
```
public IExecutionContext(long size)
```
    Native array allocator. Access with Pointer.position(long).
  - IExecutionContext
```
public IExecutionContext(Pointer p)
```
    Pointer cast constructor. Invokes Pointer(Pointer).
- Method Detail
  - position
```
public IExecutionContext position(long position)
```
    Overrides:
    
    position in class Pointer
  - getPointer
```
public IExecutionContext getPointer(long i)
```
    Overrides:
    
    getPointer in class Pointer
  - execute
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean execute(int batchSize,
                                                                                  @Cast(value="void*const*")
                                                                                  PointerPointer bindings)
```
    Deprecated. Deprecated in TensorRT 8.4. Superseded by executeV2() if the network is created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. \warning This function will trigger layer resource updates if hasImplicitBatchDimension() returns true and batchSize changes between subsequent calls, possibly resulting in performance bottlenecks.
    
    \brief Synchronously execute inference on a batch. This method requires an array of input and output buffers. The mapping from tensor names to indices can be queried using ICudaEngine::getBindingIndex()
    
    Parameters:
    
    batchSize - The batch size. This is at most the max batch size value supplied to the builder when the engine was built. If the network is created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag, please use executeV2() instead, and this batchSize argument has no effect.
    
    bindings - An array of pointers to input and output buffers for the network.
    
    Returns:
    
    True if execution succeeded.
    
    See Also:
    
    ICudaEngine::getMaxBatchSize()
  - execute
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean execute(int batchSize,
                                                                                  @Cast(value="void*const*") @ByPtrPtr
                                                                                  Pointer bindings)
```
    Deprecated.
  - enqueue
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean enqueue(int batchSize,
                                                                                  @Cast(value="void*const*")
                                                                                  PointerPointer bindings,
                                                                                  org.bytedeco.cuda.cudart.CUstream_st stream,
                                                                                  @ByPtrPtr
                                                                                  org.bytedeco.cuda.cudart.CUevent_st inputConsumed)
```
    Deprecated. Deprecated in TensorRT 8.4. Superseded by enqueueV2() if the network is created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag.
    
    \brief Enqueue inference of a batch on a stream. This method requires an array of input and output buffers. The mapping from tensor names to indices can be queried using ICudaEngine::getBindingIndex()
    
    Parameters:
    
    batchSize - The batch size. This is at most the max batch size value supplied to the builder when the engine was built. If the network is created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag, please use enqueueV3() instead, and this batchSize argument has no effect.
    
    bindings - An array of pointers to input and output buffers for the network.
    
    stream - A cuda stream on which the inference kernels will be enqueued.
    
    inputConsumed - An optional event which will be signaled when the input buffers can be refilled with new data.
    
    Returns:
    
    True if the kernels were enqueued successfully.
    
    See Also:
    
    ICudaEngine::getMaxBatchSize() \warning Calling enqueue() in from the same IExecutionContext object with different CUDA streams concurrently results in undefined behavior. To perform inference concurrently in multiple streams, use one execution context per stream. \warning This function will trigger layer resource updates if hasImplicitBatchDimension() returns true and batchSize changes between subsequent calls, possibly resulting in performance bottlenecks.
  - enqueue
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean enqueue(int batchSize,
                                                                                  @Cast(value="void*const*") @ByPtrPtr
                                                                                  Pointer bindings,
                                                                                  org.bytedeco.cuda.cudart.CUstream_st stream,
                                                                                  @ByPtrPtr
                                                                                  org.bytedeco.cuda.cudart.CUevent_st inputConsumed)
```
    Deprecated.
  - setDebugSync
```
@NoException(value=true)
public void setDebugSync(@Cast(value="bool")
                                                  boolean sync)
```
    \brief Set the debug sync flag. If this flag is set to true, the engine will log the successful execution for each kernel during executeV2(). It has no effect when using enqueueV2()/enqueueV3().
    
    See Also:
    
    getDebugSync()
  - getDebugSync
```
@Cast(value="bool")
 @NoException(value=true)
public boolean getDebugSync()
```
    \brief Get the debug sync flag.
    
    See Also:
    
    setDebugSync()
  - setProfiler
```
@NoException(value=true)
public void setProfiler(IProfiler profiler)
```
    \brief Set the profiler.
    
    See Also:
    
    getProfiler()
  - getProfiler
```
@NoException(value=true)
public IProfiler getProfiler()
```
    \brief Get the profiler.
    
    See Also:
    
    setProfiler()
  - getEngine
```
@Const
 @ByRef
 @NoException(value=true)
public ICudaEngine getEngine()
```
    \brief Get the associated engine.
    
    See Also:
    
    ICudaEngine
  - destroy
```
@Deprecated
 @NoException(value=true)
public void destroy()
```
    Deprecated. Deprecated in TRT 8.0. Superseded by delete. \warning Calling destroy on a managed pointer will result in a double-free error.
    
    \brief Destroy this object.
  - setName
```
@NoException(value=true)
public void setName(String name)
```
    \brief Set the name of the execution context. This method copies the name string. \warning The string name must be null-terminated, and be at most 4096 bytes including the terminator.
    
    See Also:
    
    getName()
  - setName
```
@NoException(value=true)
public void setName(@Cast(value="const char*")
                                             BytePointer name)
```
  - getName
```
@NoException(value=true)
public String getName()
```
    \brief Return the name of the execution context.
    
    See Also:
    
    setName()
  - setDeviceMemory
```
@NoException(value=true)
public void setDeviceMemory(Pointer memory)
```
    \brief Set the device memory for use by this execution context. The memory must be aligned with cuda memory alignment property (using cudaGetDeviceProperties()), and its size must be at least that returned by getDeviceMemorySize(). Setting memory to nullptr is acceptable if getDeviceMemorySize() returns 0. If using enqueueV2()/enqueueV3() to run the network, the memory is in use from the invocation of enqueueV2()/enqueueV3() until network execution is complete. If using executeV2(), it is in use until executeV2() returns. Releasing or otherwise using the memory for other purposes during this time will result in undefined behavior.
    
    See Also:
    
    ICudaEngine::createExecutionContextWithoutDeviceMemory()
  - getStrides
```
@Deprecated
 @ByVal
 @Cast(value="nvinfer1::Dims*")
 @NoException(value=true)
public Dims32 getStrides(int bindingIndex)
```
    Deprecated. Deprecated in TensorRT 8.5. Superseded by getTensorStrides().
    
    \brief Return the strides of the buffer for the given binding. The strides are in units of elements, not components or bytes. For example, for TensorFormat::kHWC8, a stride of one spans 8 scalars. Note that strides can be different for different execution contexts with dynamic shapes. If the bindingIndex is invalid or there are dynamic dimensions that have not been set yet, returns Dims with Dims::nbDims = -1.
    
    Parameters:
    
    bindingIndex - The binding index.
    
    See Also:
    
    getTensorStrides()
  - getTensorStrides
```
@ByVal
 @Cast(value="nvinfer1::Dims*")
 @NoException(value=true)
public Dims32 getTensorStrides(String tensorName)
```
    \brief Return the strides of the buffer for the given tensor name. The strides are in units of elements, not components or bytes. For example, for TensorFormat::kHWC8, a stride of one spans 8 scalars. Note that strides can be different for different execution contexts with dynamic shapes. If the provided name does not map to an input or output tensor, or there are dynamic dimensions that have not been set yet, return Dims{-1, {}}
    
    Parameters:
    
    tensorName - The name of an input or output tensor. \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.
  - getTensorStrides
```
@ByVal
 @Cast(value="nvinfer1::Dims*")
 @NoException(value=true)
public Dims32 getTensorStrides(@Cast(value="const char*")
                                                                                                BytePointer tensorName)
```
  - setOptimizationProfile
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean setOptimizationProfile(int profileIndex)
```
    Deprecated. Superseded by setOptimizationProfileAsync. Deprecated prior to TensorRT 8.0 and will be removed in 9.0.
    
    \brief Select an optimization profile for the current context.
    
    Parameters:
    
    profileIndex - Index of the profile. It must lie between 0 and getEngine().getNbOptimizationProfiles() - 1 The selected profile will be used in subsequent calls to executeV2()/enqueueV2()/enqueueV3(). When an optimization profile is switched via this API, TensorRT may enqueue GPU memory copy operations required to set up the new profile during the subsequent enqueueV2()/enqueueV3() operations. To avoid these calls during enqueueV2()/enqueueV3(), use setOptimizationProfileAsync() instead. If the associated CUDA engine does not have inputs with dynamic shapes, this method need not be called, in which case the default profile index of 0 will be used (this is particularly the case for all safe engines). setOptimizationProfile() must be called before calling setBindingDimensions() and setInputShapeBinding() for all dynamic input tensors or input shape tensors, which in turn must be called before executeV2()/enqueueV2()/enqueueV3(). \warning This function will trigger layer resource updates on the next call of enqueueV2()/enqueueV3()/executeV2(), possibly resulting in performance bottlenecks.
    
    Returns:
    
    true if the call succeeded, else false (e.g. input out of range)
    
    See Also:
    
    IExecutionContext::setOptimizationProfileAsync()
  - getOptimizationProfile
```
@NoException(value=true)
public int getOptimizationProfile()
```
    \brief Get the index of the currently selected optimization profile. If the profile index has not been set yet (implicitly to 0 if no other execution context has been set to profile 0, or explicitly for all subsequent contexts), an invalid value of -1 will be returned and all calls to enqueueV2()/enqueueV3()/executeV2() will fail until a valid profile index has been set. This behavior is deprecated in TensorRT 8.6 and in TensorRT 9.0, all profiles will default to optimization profile 0 and -1 will no longer be returned.
  - setBindingDimensions
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean setBindingDimensions(int bindingIndex,
                                                                                               @ByVal @Cast(value="nvinfer1::Dims*")
                                                                                               Dims32 dimensions)
```
    Deprecated. Deprecated in TensorRT 8.5. Superseded by setInputShape().
    
    \brief Set the dynamic dimensions of an input binding.
    
    Parameters:
    
    bindingIndex - index of an input tensor whose dimensions must be compatible with the network definition (i.e. only the wildcard dimension -1 can be replaced with a new dimension >= 0).
    
    dimensions - specifies the dimensions of the input tensor. It must be in the valid range for the currently selected optimization profile, and the corresponding engine must not be safety-certified. This method requires the engine to be built without an implicit batch dimension. This method will fail unless a valid optimization profile is defined for the current execution context (getOptimizationProfile() must not be -1). For all dynamic non-output bindings (which have at least one wildcard dimension of -1), this method needs to be called before either enqueueV2() or executeV2() may be called. This can be checked using the method allInputDimensionsSpecified(). \warning This function will trigger layer resource updates on the next call of enqueueV2()/executeV2(), possibly resulting in performance bottlenecks, if the dimensions are different than the previous set dimensions.
    
    Returns:
    
    false if an error occurs (e.g. bindingIndex is out of range for the currently selected optimization profile or binding dimension is inconsistent with min-max range of the optimization profile), else true. Note that the network can still be invalid for certain combinations of input shapes that lead to invalid output shapes. To confirm the correctness of the network input shapes, check whether the output binding has valid dimensions using getBindingDimensions() on the output bindingIndex.
    
    See Also:
    
    setInputShape()
  - setInputShape
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setInputShape(String tensorName,
                                                                           @Cast(value="const nvinfer1::Dims*") @ByRef
                                                                           Dims32 dims)
```
    \brief Set shape of given input.
    
    Parameters:
    
    tensorName - The name of an input tensor.
    
    dims - The shape of an input tensor.
    
    Returns:
    
    True on success, false if the provided name does not map to an input tensor, or if some other error occurred. Each dimension must agree with the network dimension unless the latter was -1. \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.
  - setInputShape
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setInputShape(@Cast(value="const char*")
                                                                           BytePointer tensorName,
                                                                           @Cast(value="const nvinfer1::Dims*") @ByRef
                                                                           Dims32 dims)
```
  - getBindingDimensions
```
@Deprecated
 @ByVal
 @Cast(value="nvinfer1::Dims*")
 @NoException(value=true)
public Dims32 getBindingDimensions(int bindingIndex)
```
    Deprecated. Deprecated in TensorRT 8.5. Superseded by getTensorShape().
    
    \brief Get the dynamic dimensions of a binding. If the engine was built with an implicit batch dimension, same as ICudaEngine::getBindingDimensions. If setBindingDimensions() has been called on this binding (or if there are no dynamic dimensions), all dimensions will be positive. Otherwise, it is necessary to call setBindingDimensions() before enqueueV2() or executeV2() may be called. If the bindingIndex is out of range, an invalid Dims with nbDims == -1 is returned. The same invalid Dims will be returned if the engine was not built with an implicit batch dimension and if the execution context is not currently associated with a valid optimization profile (i.e. if getOptimizationProfile() returns -1). If ICudaEngine::bindingIsInput(bindingIndex) is false, then both allInputDimensionsSpecified() and allInputShapesSpecified() must be true before calling this method.
    
    Returns:
    
    Currently selected binding dimensions For backwards compatibility with earlier versions of TensorRT, a bindingIndex that does not belong to the current profile is corrected as described for ICudaEngine::getProfileDimensions.
    
    See Also:
    
    ICudaEngine::getProfileDimensions(), getTensorShape()
  - getTensorShape
```
@ByVal
 @Cast(value="nvinfer1::Dims*")
 @NoException(value=true)
public Dims32 getTensorShape(String tensorName)
```
    \brief Return the shape of the given input or output.
    
    Parameters:
    
    tensorName - The name of an input or output tensor. Return Dims{-1, {}} if the provided name does not map to an input or output tensor. Otherwise return the shape of the input or output tensor. A dimension in an input tensor will have a -1 wildcard value if all the following are true: * setInputShape() has not yet been called for this tensor * The dimension is a runtime dimension that is not implicitly constrained to be a single value. A dimension in an output tensor will have a -1 wildcard value if the dimension depends on values of execution tensors OR if all the following are true: * It is a runtime dimension. * setInputShape() has NOT been called for some input tensor(s) with a runtime shape. * setTensorAddress() has NOT been called for some input tensor(s) with isShapeInferenceIO() = true. An output tensor may also have -1 wildcard dimensions if its shape depends on values of tensors supplied to enqueueV3(). If the request is for the shape of an output tensor with runtime dimensions, all input tensors with isShapeInferenceIO() = true should have their value already set, since these values might be needed to compute the output shape. Examples of an input dimension that is implicitly constrained to a single value: * The optimization profile specifies equal min and max values. * The dimension is named and only one value meets the optimization profile requirements for dimensions with that name. \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.
  - getTensorShape
```
@ByVal
 @Cast(value="nvinfer1::Dims*")
 @NoException(value=true)
public Dims32 getTensorShape(@Cast(value="const char*")
                                                                                              BytePointer tensorName)
```
  - setInputShapeBinding
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean setInputShapeBinding(int bindingIndex,
                                                                                               @Const
                                                                                               IntPointer data)
```
    Deprecated. Deprecated in TensorRT 8.5. Superseded by setInputTensorAddress() or setTensorAddress().
    
    \brief Set values of input tensor required by shape calculations.
    
    Parameters:
    
    bindingIndex - index of an input tensor for which ICudaEngine::isShapeBinding(bindingIndex) and ICudaEngine::bindingIsInput(bindingIndex) are both true.
    
    data - pointer to values of the input tensor. The number of values should be the product of the dimensions returned by getBindingDimensions(bindingIndex). If ICudaEngine::isShapeBinding(bindingIndex) and ICudaEngine::bindingIsInput(bindingIndex) are both true, this method must be called before enqueueV2() or executeV2() may be called. This method will fail unless a valid optimization profile is defined for the current execution context (getOptimizationProfile() must not be -1). \warning This function will trigger layer resource updates on the next call of enqueueV2()/executeV2(), possibly resulting in performance bottlenecks, if the shapes are different than the previous set shapes.
    
    Returns:
    
    false if an error occurs (e.g. bindingIndex is out of range for the currently selected optimization profile or shape data is inconsistent with min-max range of the optimization profile), else true. Note that the network can still be invalid for certain combinations of input shapes that lead to invalid output shapes. To confirm the correctness of the network input shapes, check whether the output binding has valid dimensions using getBindingDimensions() on the output bindingIndex.
    
    See Also:
    
    setTensorAddress()
  - setInputShapeBinding
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean setInputShapeBinding(int bindingIndex,
                                                                                               @Const
                                                                                               IntBuffer data)
```
    Deprecated.
  - setInputShapeBinding
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean setInputShapeBinding(int bindingIndex,
                                                                                               @Const
                                                                                               int[] data)
```
    Deprecated.
  - getShapeBinding
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean getShapeBinding(int bindingIndex,
                                                                                          IntPointer data)
```
    Deprecated. Deprecated in TensorRT 8.5. Superseded by getTensorAddress() or getOutputTensorAddress().
    
    \brief Get values of an input tensor required for shape calculations or an output tensor produced by shape calculations.
    
    Parameters:
    
    bindingIndex - index of an input or output tensor for which ICudaEngine::isShapeBinding(bindingIndex) is true.
    
    data - pointer to where values will be written. The number of values written is the product of the dimensions returned by getBindingDimensions(bindingIndex). If ICudaEngine::bindingIsInput(bindingIndex) is false, then both allInputDimensionsSpecified() and allInputShapesSpecified() must be true before calling this method. The method will also fail if no valid optimization profile has been set for the current execution context, i.e. if getOptimizationProfile() returns -1.
    
    See Also:
    
    getTensorAddress() getOutputTensorAddress()
  - getShapeBinding
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean getShapeBinding(int bindingIndex,
                                                                                          IntBuffer data)
```
    Deprecated.
  - getShapeBinding
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean getShapeBinding(int bindingIndex,
                                                                                          int[] data)
```
    Deprecated.
  - allInputDimensionsSpecified
```
@Cast(value="bool")
 @NoException(value=true)
public boolean allInputDimensionsSpecified()
```
    \brief Whether all dynamic dimensions of input tensors have been specified
    
    Returns:
    
    True if all dynamic dimensions of input tensors have been specified by calling setBindingDimensions(). Trivially true if network has no dynamically shaped input tensors. Does not work with name-base interfaces eg. IExecutionContext::setInputShape(). Use IExecutionContext::inferShapes() instead.
    
    See Also:
    
    setBindingDimensions(bindingIndex,dimensions)
  - allInputShapesSpecified
```
@Cast(value="bool")
 @NoException(value=true)
public boolean allInputShapesSpecified()
```
    \brief Whether all input shape bindings have been specified
    
    Returns:
    
    True if all input shape bindings have been specified by setInputShapeBinding(). Trivially true if network has no input shape bindings. Does not work with name-base interfaces eg. IExecutionContext::setInputShape(). Use IExecutionContext::inferShapes() instead.
    
    See Also:
    
    isShapeBinding(bindingIndex)
  - setErrorRecorder
```
@NoException(value=true)
public void setErrorRecorder(IErrorRecorder recorder)
```
    See Also:
    
    /**
  - getErrorRecorder
```
@NoException(value=true)
public IErrorRecorder getErrorRecorder()
```
    \brief Get the ErrorRecorder assigned to this interface. Retrieves the assigned error recorder object for the given class. A nullptr will be returned if an error handler has not been set.
    
    Returns:
    
    A pointer to the IErrorRecorder object that has been registered.
    
    See Also:
    
    setErrorRecorder()
  - executeV2
```
@Cast(value="bool")
 @NoException(value=true)
public boolean executeV2(@Cast(value="void*const*")
                                                                       PointerPointer bindings)
```
    \brief Synchronously execute inference a network. This method requires an array of input and output buffers. The mapping from tensor names to indices can be queried using ICudaEngine::getBindingIndex(). This method only works for execution contexts built with full dimension networks.
    
    Parameters:
    
    bindings - An array of pointers to input and output buffers for the network.
    
    Returns:
    
    True if execution succeeded.
    
    See Also:
    
    ICudaEngine::getMaxBatchSize()
  - executeV2
```
@Cast(value="bool")
 @NoException(value=true)
public boolean executeV2(@Cast(value="void*const*") @ByPtrPtr
                                                                       Pointer bindings)
```
  - enqueueV2
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean enqueueV2(@Cast(value="void*const*")
                                                                                    PointerPointer bindings,
                                                                                    org.bytedeco.cuda.cudart.CUstream_st stream,
                                                                                    @ByPtrPtr
                                                                                    org.bytedeco.cuda.cudart.CUevent_st inputConsumed)
```
    Deprecated. Superseded by enqueueV3(). Deprecated in TensorRT 8.5
    
    \brief Enqueue inference on a stream. This method requires an array of input and output buffers. The mapping from tensor names to indices can be queried using ICudaEngine::getBindingIndex(). This method only works for execution contexts built with full dimension networks.
    
    Parameters:
    
    bindings - An array of pointers to input and output buffers for the network.
    
    stream - A cuda stream on which the inference kernels will be enqueued
    
    inputConsumed - An optional event which will be signaled when the input buffers can be refilled with new data
    
    Returns:
    
    True if the kernels were enqueued successfully.
    
    See Also:
    
    ICudaEngine::getMaxBatchSize() IExecutionContext::enqueueV3() \note Calling enqueueV2() with a stream in CUDA graph capture mode has a known issue. If dynamic shapes are used, the first enqueueV2() call after a setInputShapeBinding() call will cause failure in stream capture due to resource allocation. Please call enqueueV2() once before capturing the graph. \warning Calling enqueueV2() in from the same IExecutionContext object with different CUDA streams concurrently results in undefined behavior. To perform inference concurrently in multiple streams, use one execution context per stream.
  - enqueueV2
```
@Cast(value="bool")
 @Deprecated
 @NoException(value=true)
public boolean enqueueV2(@Cast(value="void*const*") @ByPtrPtr
                                                                                    Pointer bindings,
                                                                                    org.bytedeco.cuda.cudart.CUstream_st stream,
                                                                                    @ByPtrPtr
                                                                                    org.bytedeco.cuda.cudart.CUevent_st inputConsumed)
```
    Deprecated.
  - setOptimizationProfileAsync
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setOptimizationProfileAsync(int profileIndex,
                                                                                         org.bytedeco.cuda.cudart.CUstream_st stream)
```
    \brief Select an optimization profile for the current context with async semantics.
    
    Parameters:
    
    profileIndex - Index of the profile. The value must lie between 0 and getEngine().getNbOptimizationProfiles() - 1
    
    stream - A cuda stream on which the cudaMemcpyAsyncs may be enqueued When an optimization profile is switched via this API, TensorRT may require that data is copied via cudaMemcpyAsync. It is the applicationâs responsibility to guarantee that synchronization between the profile sync stream and the enqueue stream occurs. The selected profile will be used in subsequent calls to executeV2()/enqueueV2()/enqueueV3(). If the associated CUDA engine has inputs with dynamic shapes, the optimization profile must be set with its corresponding profileIndex before calling execute or enqueue. If no execution context is assigned optimization profile 0 and a new context is created for an engine, setOptimizationProfile(0) is called implicitly. This functionality is deprecated in TensorRT 8.6 and will instead default all optimization profiles to 0 starting in TensorRT 9.0. If the associated CUDA engine does not have inputs with dynamic shapes, this method need not be called, in which case the default profile index of 0 will be used. setOptimizationProfileAsync() must be called before calling setBindingDimensions() and setInputShapeBinding() for all dynamic input tensors or input shape tensors, which in turn must be called before executeV2()/enqueueV2()/enqueueV3(). \warning This function will trigger layer resource updates on the next call of enqueueV2()/executeV2()/enqueueV3(), possibly resulting in performance bottlenecks. \warning Not synchronizing the stream used at enqueue with the stream used to set optimization profile asynchronously using this API will result in undefined behavior.
    
    Returns:
    
    true if the call succeeded, else false (e.g. input out of range)
    
    See Also:
    
    ICudaEngine::getNbOptimizationProfiles(), IExecutionContext::setOptimizationProfile()
  - setEnqueueEmitsProfile
```
@NoException(value=true)
public void setEnqueueEmitsProfile(@Cast(value="bool")
                                                            boolean enqueueEmitsProfile)
```
    \brief Set whether enqueue emits layer timing to the profiler If set to true (default), enqueue is synchronous and does layer timing profiling implicitly if there is a profiler attached. If set to false, enqueue will be asynchronous if there is a profiler attached. An extra method reportToProfiler() needs to be called to obtain the profiling data and report to the profiler attached.
    
    See Also:
    
    IExecutionContext::getEnqueueEmitsProfile(), IExecutionContext::reportToProfiler()
  - getEnqueueEmitsProfile
```
@Cast(value="bool")
 @NoException(value=true)
public boolean getEnqueueEmitsProfile()
```
    \brief Get the enqueueEmitsProfile state.
    
    Returns:
    
    The enqueueEmitsProfile state.
    
    See Also:
    
    IExecutionContext::setEnqueueEmitsProfile()
  - reportToProfiler
```
@Cast(value="bool")
 @NoException(value=true)
public boolean reportToProfiler()
```
    \brief Calculate layer timing info for the current optimization profile in IExecutionContext and update the profiler after one iteration of inference launch. If IExecutionContext::getEnqueueEmitsProfile() returns true, the enqueue function will calculate layer timing implicitly if a profiler is provided. This function returns true and does nothing. If IExecutionContext::getEnqueueEmitsProfile() returns false, the enqueue function will record the CUDA event timers if a profiler is provided. But it will not perform the layer timing calculation. IExecutionContext::reportToProfiler() needs to be called explicitly to calculate layer timing for the previous inference launch. In the CUDA graph launch scenario, it will record the same set of CUDA events as in regular enqueue functions if the graph is captured from an IExecutionContext with profiler enabled. This function needs to be called after graph launch to report the layer timing info to the profiler. \warning profiling CUDA graphs is only available from CUDA 11.1 onwards. \warning reportToProfiler uses the stream of the previous enqueue call, so the stream must be live otherwise behavior is undefined.
    
    Returns:
    
    true if the call succeeded, else false (e.g. profiler not provided, in CUDA graph capture mode, etc.)
    
    See Also:
    
    IExecutionContext::setEnqueueEmitsProfile(), IExecutionContext::getEnqueueEmitsProfile()
  - setTensorAddress
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setTensorAddress(String tensorName,
                                                                              Pointer data)
```
    \brief Set memory address for given input or output tensor.
    
    Parameters:
    
    tensorName - The name of an input or output tensor.
    
    data - The pointer (void*) to the data owned by the user.
    
    Returns:
    
    True on success, false if error occurred. An address defaults to nullptr. Pass data=nullptr to reset to the default state. Return false if the provided name does not map to an input or output tensor. If an input pointer has type (void const*), use setInputTensorAddress() instead. Before calling enqueueV3(), each input must have a non-null address and each output must have a non-null address or an IOutputAllocator to set it later. If the TensorLocation of the tensor is kHOST, the pointer must point to a host buffer of sufficient size. For shape tensors, the only supported data type is int32_t. If the TensorLocation of the tensor is kDEVICE, the pointer must point to a device buffer of sufficient size and alignment, or be nullptr if the tensor is an output tensor that will be allocated by IOutputAllocator. If getTensorShape(name) reports a -1 for any dimension of an output after all input shapes have been set, then to find out the dimensions, use setOutputAllocator() to associate an IOutputAllocator to which the dimensions will be reported when known. Calling both setTensorAddress and setOutputAllocator() for the same output is allowed, and can be useful for preallocating memory, and then reallocating if it's not big enough. The pointer must have at least 256-byte alignment. \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.
    
    See Also:
    
    getTensorShape() setOutputAllocator() IOutputAllocator
  - setTensorAddress
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setTensorAddress(@Cast(value="const char*")
                                                                              BytePointer tensorName,
                                                                              Pointer data)
```
  - getTensorAddress
```
@Const
 @NoException(value=true)
public Pointer getTensorAddress(String tensorName)
```
    \brief Get memory address bound to given input or output tensor, or nullptr if the provided name does not map to an input or output tensor.
    
    Parameters:
    
    tensorName - The name of an input or output tensor. Use method getOutputTensorAddress() if a non-const pointer for an output tensor is required. \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.
    
    See Also:
    
    getOutputTensorAddress()
  - getTensorAddress
```
@Const
 @NoException(value=true)
public Pointer getTensorAddress(@Cast(value="const char*")
                                                                 BytePointer tensorName)
```
  - setInputTensorAddress
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setInputTensorAddress(String tensorName,
                                                                                   @Const
                                                                                   Pointer data)
```
    \brief Set memory address for given input.
    
    Parameters:
    
    tensorName - The name of an input tensor.
    
    data - The pointer (void const*) to the const data owned by the user.
    
    Returns:
    
    True on success, false if the provided name does not map to an input tensor, does not meet alignment requirements, or some other error occurred. Input addresses can also be set using method setTensorAddress, which requires a (void*). See description of method setTensorAddress() for alignment and data type constraints. \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.
    
    See Also:
    
    setTensorAddress()
  - setInputTensorAddress
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setInputTensorAddress(@Cast(value="const char*")
                                                                                   BytePointer tensorName,
                                                                                   @Const
                                                                                   Pointer data)
```
  - getOutputTensorAddress
```
@NoException(value=true)
public Pointer getOutputTensorAddress(String tensorName)
```
    \brief Get memory address for given output.
    
    Parameters:
    
    tensorName - The name of an output tensor.
    
    Returns:
    
    Raw output data pointer (void*) for given output tensor, or nullptr if the provided name does not map to an output tensor. If only a (void const*) pointer is needed, an alternative is to call method getTensorAddress(). \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.
    
    See Also:
    
    getTensorAddress()
  - getOutputTensorAddress
```
@NoException(value=true)
public Pointer getOutputTensorAddress(@Cast(value="const char*")
                                                               BytePointer tensorName)
```
  - inferShapes
```
@NoException(value=true)
public int inferShapes(int nbMaxNames,
                                                @Cast(value="const char**")
                                                PointerPointer tensorNames)
```
    \brief Run shape calculations.
    
    Parameters:
    
    nbMaxNames - Maximum number of names to write to tensorNames. When the return value is a positive value n and tensorNames != nullptr, the names of min(n,nbMaxNames) insufficiently specified input tensors are written to tensorNames.
    
    tensorNames - Buffer in which to place names of insufficiently specified input tensors.
    
    Returns:
    
    0 on success. Positive value n if n input tensors were not sufficiently specified. -1 for other errors. An input tensor is insufficiently specified if either of the following is true: * It has dynamic dimensions and its runtime dimensions have not yet been specified via IExecutionContext::setInputShape. * isShapeInferenceIO(t)=true and the tensor's address has not yet been set. If an output tensor has isShapeInferenceIO(t)=true and its address has been specified, then its value is written. Returns -1 if tensorNames == nullptr and nbMaxNames != 0. Returns -1 if nbMaxNames < 0. Returns -1 if a tensor's dimensions are invalid, e.g. a tensor ends up with a negative dimension.
  - inferShapes
```
@NoException(value=true)
public int inferShapes(int nbMaxNames,
                                                @Cast(value="const char**") @ByPtrPtr
                                                BytePointer tensorNames)
```
  - inferShapes
```
@NoException(value=true)
public int inferShapes(int nbMaxNames,
                                                @Cast(value="const char**") @ByPtrPtr
                                                ByteBuffer tensorNames)
```
  - inferShapes
```
@NoException(value=true)
public int inferShapes(int nbMaxNames,
                                                @Cast(value="const char**") @ByPtrPtr
                                                byte[] tensorNames)
```
  - setInputConsumedEvent
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setInputConsumedEvent(org.bytedeco.cuda.cudart.CUevent_st event)
```
    \brief Mark input as consumed.
    
    Parameters:
    
    event - The cuda event that is triggered after all input tensors have been consumed. \warning The set event must be valid during the inferece.
    
    Returns:
    
    True on success, false if error occurred. Passing event==nullptr removes whatever event was set, if any.
  - getInputConsumedEvent
```
@NoException(value=true)
public org.bytedeco.cuda.cudart.CUevent_st getInputConsumedEvent()
```
    \brief The event associated with consuming the input.
    
    Returns:
    
    The cuda event. Nullptr will be returned if the event is not set yet.
  - setOutputAllocator
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setOutputAllocator(String tensorName,
                                                                                IOutputAllocator outputAllocator)
```
    \brief Set output allocator to use for output tensor of given name. Pass nullptr to outputAllocator to unset. The allocator is called by enqueueV3().
    
    Parameters:
    
    tensorName - The name of an output tensor.
    
    outputAllocator - IOutputAllocator for the tensors.
    
    Returns:
    
    True if success, false if the provided name does not map to an output or, if some other error occurred. \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.
    
    See Also:
    
    IOutputAllocator
  - setOutputAllocator
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setOutputAllocator(@Cast(value="const char*")
                                                                                BytePointer tensorName,
                                                                                IOutputAllocator outputAllocator)
```
  - getOutputAllocator
```
@NoException(value=true)
public IOutputAllocator getOutputAllocator(String tensorName)
```
    \brief Get output allocator associated with output tensor of given name, or nullptr if the provided name does not map to an output tensor. \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.
    
    See Also:
    
    IOutputAllocator
  - getOutputAllocator
```
@NoException(value=true)
public IOutputAllocator getOutputAllocator(@Cast(value="const char*")
                                                                    BytePointer tensorName)
```
  - getMaxOutputSize
```
@Cast(value="int64_t")
 @NoException(value=true)
public long getMaxOutputSize(String tensorName)
```
    \brief Get upper bound on an output tensor's size, in bytes, based on the current optimization profile and input dimensions. If the profile or input dimensions are not yet set, or the provided name does not map to an output, returns -1.
    
    Parameters:
    
    tensorName - The name of an output tensor.
    
    Returns:
    
    Upper bound in bytes. \warning The string tensorName must be null-terminated, and be at most 4096 bytes including the terminator.
  - getMaxOutputSize
```
@Cast(value="int64_t")
 @NoException(value=true)
public long getMaxOutputSize(@Cast(value="const char*")
                                                                              BytePointer tensorName)
```
  - setTemporaryStorageAllocator
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setTemporaryStorageAllocator(IGpuAllocator allocator)
```
    \brief Specify allocator to use for internal temporary storage. This allocator is used only by enqueueV3() for temporary storage whose size cannot be predicted ahead of enqueueV3(). It is not used for output tensors, because memory allocation for those is allocated by the allocator set by setOutputAllocator(). All memory allocated is freed by the time enqueueV3() returns.
    
    Parameters:
    
    allocator - pointer to allocator to use. Pass nullptr to revert to using TensorRT's default allocator.
    
    Returns:
    
    True on success, false if error occurred.
    
    See Also:
    
    setOutputAllocator()
  - getTemporaryStorageAllocator
```
@NoException(value=true)
public IGpuAllocator getTemporaryStorageAllocator()
```
    \brief Get allocator set by setTemporaryStorageAllocator. Returns a nullptr if a nullptr was passed with setTemporaryStorageAllocator().
  - enqueueV3
```
@Cast(value="bool")
 @NoException(value=true)
public boolean enqueueV3(org.bytedeco.cuda.cudart.CUstream_st stream)
```
    \brief Enqueue inference on a stream.
    
    Parameters:
    
    stream - A cuda stream on which the inference kernels will be enqueued.
    
    Returns:
    
    True if the kernels were enqueued successfully, false otherwise. Modifying or releasing memory that has been registered for the tensors before stream synchronization or the event passed to setInputConsumedEvent has been being triggered results in undefined behavior. Input tensor can be released after the setInputConsumedEvent whereas output tensors require stream synchronization.
  - setPersistentCacheLimit
```
@NoException(value=true)
public void setPersistentCacheLimit(@Cast(value="size_t")
                                                             long size)
```
    \brief Set the maximum size for persistent cache usage. This function sets the maximum persistent L2 cache that this execution context may use for activation caching. Activation caching is not supported on all architectures - see "How TensorRT uses Memory" in the developer guide for details
    
    Parameters:
    
    size - the size of persistent cache limitation in bytes. The default is 0 Bytes.
    
    See Also:
    
    getPersistentCacheLimit
  - getPersistentCacheLimit
```
@Cast(value="size_t")
 @NoException(value=true)
public long getPersistentCacheLimit()
```
    \brief Get the maximum size for persistent cache usage.
    
    Returns:
    
    The size of the persistent cache limit
    
    See Also:
    
    setPersistentCacheLimit
  - setNvtxVerbosity
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setNvtxVerbosity(nvinfer.ProfilingVerbosity verbosity)
```
    \brief Set the verbosity of the NVTX markers in the execution context. Building with kDETAILED verbosity will generally increase latency in enqueueV2/enqueueV3(). Call this method to select NVTX verbosity in this execution context at runtime. The default is the verbosity with which the engine was built, and the verbosity may not be raised above that level. This function does not affect how IEngineInspector interacts with the engine.
    
    Parameters:
    
    verbosity - The verbosity of the NVTX markers.
    
    Returns:
    
    True if the NVTX verbosity is set successfully. False if the provided verbosity level is higher than the profiling verbosity of the corresponding engine.
    
    See Also:
    
    getNvtxVerbosity(), ICudaEngine::getProfilingVerbosity()
  - setNvtxVerbosity
```
@Cast(value="bool")
 @NoException(value=true)
public boolean setNvtxVerbosity(@Cast(value="nvinfer1::ProfilingVerbosity")
                                                                              int verbosity)
```
  - getNvtxVerbosity
```
@NoException(value=true)
public nvinfer.ProfilingVerbosity getNvtxVerbosity()
```
    \brief Get the NVTX verbosity of the execution context.
    
    Returns:
    
    The current NVTX verbosity of the execution context.
    
    See Also:
    
    setNvtxVerbosity()
  - setAuxStreams
```
@NoException(value=true)
public void setAuxStreams(@ByPtrPtr
                                                   org.bytedeco.cuda.cudart.CUstream_st auxStreams,
                                                   int nbStreams)
```
    \brief Set the auxiliary streams that TensorRT should launch kernels on in the next enqueueV3() call. If set, TensorRT will launch the kernels that are supposed to run on the auxiliary streams using the streams provided by the user with this API. If this API is not called before the enqueueV3() call, then TensorRT will use the auxiliary streams created by TensorRT internally. TensorRT will always insert event synchronizations between the main stream provided via enqueueV3() call and the auxiliary streams: - At the beginning of the enqueueV3() call, TensorRT will make sure that all the auxiliary streams wait on the activities on the main stream. - At the end of the enqueueV3() call, TensorRT will make sure that the main stream wait on the activities on all the auxiliary streams.
    
    Parameters:
    
    auxStreams - The pointer to an array of cudaStream_t with the array length equal to nbStreams.
    
    nbStreams - The number of auxiliary streams provided. If nbStreams is greater than engine->getNbAuxStreams(), then only the first engine->getNbAuxStreams() streams will be used. If nbStreams is less than engine->getNbAuxStreams(), such as setting nbStreams to 0, then TensorRT will use the provided streams for the first nbStreams auxiliary streams, and will create additional streams internally for the rest of the auxiliary streams. \note The provided auxiliary streams must not be the default stream and must all be different to avoid deadlocks.
    
    See Also:
    
    enqueueV3(), IBuilderConfig::setMaxAuxStreams(), ICudaEngine::getNbAuxStreams()

Class IExecutionContext

Nested Class Summary

Nested classes/interfaces inherited from class org.bytedeco.javacpp.Pointer

Field Summary

Fields inherited from class org.bytedeco.javacpp.Pointer

Constructor Summary

Method Summary

Methods inherited from class org.bytedeco.javacpp.Pointer

Methods inherited from class java.lang.Object

Constructor Detail

IExecutionContext

IExecutionContext

IExecutionContext

Method Detail

position

getPointer

execute

execute

enqueue

enqueue

setDebugSync

getDebugSync

setProfiler

getProfiler

getEngine

destroy

setName

setName

getName

setDeviceMemory

getStrides

getTensorStrides

getTensorStrides

setOptimizationProfile

getOptimizationProfile

setBindingDimensions

setInputShape

setInputShape

getBindingDimensions

getTensorShape

getTensorShape

setInputShapeBinding

setInputShapeBinding

setInputShapeBinding

getShapeBinding

getShapeBinding

getShapeBinding

allInputDimensionsSpecified

allInputShapesSpecified

setErrorRecorder

getErrorRecorder

executeV2

executeV2

enqueueV2

enqueueV2

setOptimizationProfileAsync

setEnqueueEmitsProfile

getEnqueueEmitsProfile

reportToProfiler

setTensorAddress

setTensorAddress

getTensorAddress

getTensorAddress

setInputTensorAddress

setInputTensorAddress

getOutputTensorAddress

getOutputTensorAddress

inferShapes

inferShapes

inferShapes

inferShapes

setInputConsumedEvent

getInputConsumedEvent

setOutputAllocator

setOutputAllocator

getOutputAllocator

getOutputAllocator

getMaxOutputSize

getMaxOutputSize

setTemporaryStorageAllocator