Class GkeInferenceQuickstartGrpc.GkeInferenceQuickstartFutureStub

java.lang.Object
io.grpc.stub.AbstractStub<S>
io.grpc.stub.AbstractFutureStub<GkeInferenceQuickstartGrpc.GkeInferenceQuickstartFutureStub>
com.google.cloud.gkerecommender.v1.GkeInferenceQuickstartGrpc.GkeInferenceQuickstartFutureStub
Enclosing class:
GkeInferenceQuickstartGrpc

public static final class GkeInferenceQuickstartGrpc.GkeInferenceQuickstartFutureStub extends io.grpc.stub.AbstractFutureStub<GkeInferenceQuickstartGrpc.GkeInferenceQuickstartFutureStub>
A stub to allow clients to do ListenableFuture-style rpc calls to service GkeInferenceQuickstart.
 GKE Inference Quickstart (GIQ) service provides profiles with performance
 metrics for popular models and model servers across multiple accelerators.
 These profiles help generate optimized best practices for running inference
 on GKE.
 
  • Method Details

    • build

      protected GkeInferenceQuickstartGrpc.GkeInferenceQuickstartFutureStub build(io.grpc.Channel channel, io.grpc.CallOptions callOptions)
      Specified by:
      build in class io.grpc.stub.AbstractStub<GkeInferenceQuickstartGrpc.GkeInferenceQuickstartFutureStub>
    • fetchModels

      public com.google.common.util.concurrent.ListenableFuture<FetchModelsResponse> fetchModels(FetchModelsRequest request)
       Fetches available models. Open-source models follow the Huggingface Hub
       `owner/model_name` format.
       
    • fetchModelServers

      public com.google.common.util.concurrent.ListenableFuture<FetchModelServersResponse> fetchModelServers(FetchModelServersRequest request)
       Fetches available model servers. Open-source model servers use simplified,
       lowercase names (e.g., `vllm`).
       
    • fetchModelServerVersions

      public com.google.common.util.concurrent.ListenableFuture<FetchModelServerVersionsResponse> fetchModelServerVersions(FetchModelServerVersionsRequest request)
       Fetches available model server versions. Open-source servers use their own
       versioning schemas (e.g., `vllm` uses semver like `v1.0.0`).
       Some model servers have different versioning schemas depending on the
       accelerator. For example, `vllm` uses semver on GPUs, but returns nightly
       build tags on TPUs. All available versions will be returned when different
       schemas are present.
       
    • fetchProfiles

      public com.google.common.util.concurrent.ListenableFuture<FetchProfilesResponse> fetchProfiles(FetchProfilesRequest request)
       Fetches available profiles. A profile contains performance metrics and
       cost information for a specific model server setup. Profiles can be
       filtered by parameters. If no filters are provided, all profiles are
       returned.
       Profiles display a single value per performance metric based on the
       provided performance requirements. If no requirements are given, the
       metrics represent the inflection point. See [Run best practice inference
       with GKE Inference Quickstart
       recipes](https://cloud.google.com/kubernetes-engine/docs/how-to/machine-learning/inference/inference-quickstart#how)
       for details.
       
    • generateOptimizedManifest

      public com.google.common.util.concurrent.ListenableFuture<GenerateOptimizedManifestResponse> generateOptimizedManifest(GenerateOptimizedManifestRequest request)
       Generates an optimized deployment manifest for a given model and model
       server, based on the specified accelerator, performance targets, and
       configurations. See [Run best practice inference with GKE Inference
       Quickstart
       recipes](https://cloud.google.com/kubernetes-engine/docs/how-to/machine-learning/inference/inference-quickstart)
       for deployment details.
       
    • fetchBenchmarkingData

      public com.google.common.util.concurrent.ListenableFuture<FetchBenchmarkingDataResponse> fetchBenchmarkingData(FetchBenchmarkingDataRequest request)
       Fetches all of the benchmarking data available for a profile. Benchmarking
       data returns all of the performance metrics available for a given model
       server setup on a given instance type.