Package ai.djl.inference.streaming

Interface StreamingTranslator<I,O>

Type Parameters:

I - the input type

O - the output type

All Superinterfaces:

PostProcessor<O>, PreProcessor<I>, Translator<I,O>

All Known Subinterfaces:

ServingTranslator

public interface StreamingTranslator<I,O>
extends Translator<I,O>

An expansion of the Translator with postProcessing for the StreamingBlock (used by Predictor.streamingPredict(Object).

Method Summary

Modifier and Type	Method	Description
O	processStreamOutput(TranslatorContext ctx, java.util.stream.Stream<NDList> list)	Processes the output NDList to the corresponding output object.

Methods inherited from interface ai.djl.translate.PostProcessor

processOutput

Methods inherited from interface ai.djl.translate.PreProcessor

processInput

Methods inherited from interface ai.djl.translate.Translator

getBatchifier, getExpansions, prepare, toBatchTranslator, toBatchTranslator

Method Detail

processStreamOutput

O processStreamOutput(TranslatorContext ctx,
                      java.util.stream.Stream<NDList> list)
               throws java.lang.Exception

Processes the output NDList to the corresponding output object.

Parameters:

ctx - the toolkit used for post-processing

list - the output NDList after inference, usually immutable in engines like PyTorch. @see Issue 1774

Returns:

the output object of expected type

Throws:

java.lang.Exception - if an error occurs during processing output