Package org.apache.beam.sdk.ml.inference.remote

Interface BaseModelHandler<ParamT extends BaseModelParameters,​InputT extends BaseInput,​OutputT extends BaseResponse>

  • public interface BaseModelHandler<ParamT extends BaseModelParameters,​InputT extends BaseInput,​OutputT extends BaseResponse>
    Interface for model-specific handlers that perform remote inference operations.

    Implementations of this interface encapsulate all logic for communicating with a specific remote inference service. Each handler is responsible for:

    • Initializing and managing client connections
    • Converting Beam inputs to service-specific request formats
    • Making inference API calls
    • Converting service responses to Beam output types
    • Handling errors and retries if applicable

    Lifecycle

    Handler instances follow this lifecycle:

    1. Instantiation via no-argument constructor
    2. createClient(ParamT) called with parameters during setup
    3. request(java.util.List<InputT>) called for each batch of inputs

    Handlers typically contain non-serializable client objects. Mark client fields as transient and initialize them in createClient(ParamT)

    Batching Considerations

    The request(java.util.List<InputT>) method receives a list of inputs. Implementations should:

    • Batch inputs efficiently if the service supports batch inference
    • Return results in the same order as inputs
    • Maintain input-output correspondence in PredictionResult

    • Method Detail

      • createClient

        void createClient​(ParamT parameters)
        Initializes the remote model client with the provided parameters.