Class GkeInferenceQuickstartGrpc.GkeInferenceQuickstartBlockingStub

java.lang.Object
io.grpc.stub.AbstractStub<S>
io.grpc.stub.AbstractBlockingStub<GkeInferenceQuickstartGrpc.GkeInferenceQuickstartBlockingStub>
com.google.cloud.gkerecommender.v1.GkeInferenceQuickstartGrpc.GkeInferenceQuickstartBlockingStub
Enclosing class:
GkeInferenceQuickstartGrpc

public static final class GkeInferenceQuickstartGrpc.GkeInferenceQuickstartBlockingStub extends io.grpc.stub.AbstractBlockingStub<GkeInferenceQuickstartGrpc.GkeInferenceQuickstartBlockingStub>
A stub to allow clients to do limited synchronous rpc calls to service GkeInferenceQuickstart.
 GKE Inference Quickstart (GIQ) service provides profiles with performance
 metrics for popular models and model servers across multiple accelerators.
 These profiles help generate optimized best practices for running inference
 on GKE.
 
  • Method Details

    • build

      protected GkeInferenceQuickstartGrpc.GkeInferenceQuickstartBlockingStub build(io.grpc.Channel channel, io.grpc.CallOptions callOptions)
      Specified by:
      build in class io.grpc.stub.AbstractStub<GkeInferenceQuickstartGrpc.GkeInferenceQuickstartBlockingStub>
    • fetchModels

      public FetchModelsResponse fetchModels(FetchModelsRequest request)
       Fetches available models. Open-source models follow the Huggingface Hub
       `owner/model_name` format.
       
    • fetchModelServers

      public FetchModelServersResponse fetchModelServers(FetchModelServersRequest request)
       Fetches available model servers. Open-source model servers use simplified,
       lowercase names (e.g., `vllm`).
       
    • fetchModelServerVersions

      public FetchModelServerVersionsResponse fetchModelServerVersions(FetchModelServerVersionsRequest request)
       Fetches available model server versions. Open-source servers use their own
       versioning schemas (e.g., `vllm` uses semver like `v1.0.0`).
       Some model servers have different versioning schemas depending on the
       accelerator. For example, `vllm` uses semver on GPUs, but returns nightly
       build tags on TPUs. All available versions will be returned when different
       schemas are present.
       
    • fetchProfiles

      public FetchProfilesResponse fetchProfiles(FetchProfilesRequest request)
       Fetches available profiles. A profile contains performance metrics and
       cost information for a specific model server setup. Profiles can be
       filtered by parameters. If no filters are provided, all profiles are
       returned.
       Profiles display a single value per performance metric based on the
       provided performance requirements. If no requirements are given, the
       metrics represent the inflection point. See [Run best practice inference
       with GKE Inference Quickstart
       recipes](https://cloud.google.com/kubernetes-engine/docs/how-to/machine-learning/inference/inference-quickstart#how)
       for details.
       
    • generateOptimizedManifest

      public GenerateOptimizedManifestResponse generateOptimizedManifest(GenerateOptimizedManifestRequest request)
       Generates an optimized deployment manifest for a given model and model
       server, based on the specified accelerator, performance targets, and
       configurations. See [Run best practice inference with GKE Inference
       Quickstart
       recipes](https://cloud.google.com/kubernetes-engine/docs/how-to/machine-learning/inference/inference-quickstart)
       for deployment details.
       
    • fetchBenchmarkingData

      public FetchBenchmarkingDataResponse fetchBenchmarkingData(FetchBenchmarkingDataRequest request)
       Fetches all of the benchmarking data available for a profile. Benchmarking
       data returns all of the performance metrics available for a given model
       server setup on a given instance type.