Package org.springframework.ai.bedrock.converse

Class BedrockProxyChatModel

java.lang.Object

org.springframework.ai.bedrock.converse.BedrockProxyChatModel

All Implemented Interfaces:

org.springframework.ai.chat.model.ChatModel, org.springframework.ai.chat.model.StreamingChatModel, org.springframework.ai.model.Model<org.springframework.ai.chat.prompt.Prompt,org.springframework.ai.chat.model.ChatResponse>, org.springframework.ai.model.StreamingModel<org.springframework.ai.chat.prompt.Prompt,org.springframework.ai.chat.model.ChatResponse>

public class BedrockProxyChatModel
extends Object
implements org.springframework.ai.chat.model.ChatModel

A ChatModel implementation that uses the Amazon Bedrock Converse API to interact with the Supported models.

The Converse API doesn't support any embedding models (such as Titan Embeddings G1 - Text) or image generation models (such as Stability AI).

https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html

https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html

https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html

https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html

https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html

Since:

1.0.0

Author:

Christian Tzolov, Wei Jiang, Alexandros Pappas, Jihoon Kim, Soby Chacko, Sun Yuhan

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	BedrockProxyChatModel.Builder

Field Summary

Fields

Modifier and Type	Field	Description
static final reactor.core.publisher.Sinks.EmitFailureHandler	DEFAULT_EMIT_FAILURE_HANDLER

Constructor Summary

Constructors

Constructor	Description
BedrockProxyChatModel(software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient bedrockRuntimeClient, software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeAsyncClient bedrockRuntimeAsyncClient, BedrockChatOptions defaultOptions, io.micrometer.observation.ObservationRegistry observationRegistry, org.springframework.ai.model.tool.ToolCallingManager toolCallingManager)
BedrockProxyChatModel(software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient bedrockRuntimeClient, software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeAsyncClient bedrockRuntimeAsyncClient, BedrockChatOptions defaultOptions, io.micrometer.observation.ObservationRegistry observationRegistry, org.springframework.ai.model.tool.ToolCallingManager toolCallingManager, org.springframework.ai.model.tool.ToolExecutionEligibilityPredicate toolExecutionEligibilityPredicate)

Method Summary

Modifier and Type	Method	Description
static BedrockProxyChatModel.Builder	builder()
org.springframework.ai.chat.model.ChatResponse	call(org.springframework.ai.chat.prompt.Prompt prompt)	Invoke the model and return the response.
reactor.core.publisher.Flux<software.amazon.awssdk.services.bedrockruntime.model.ConverseStreamOutput>	converseStream(software.amazon.awssdk.services.bedrockruntime.model.ConverseStreamRequest converseStreamRequest)	Invoke the model and return the response stream.
org.springframework.ai.chat.prompt.ChatOptions	getDefaultOptions()
void	setObservationConvention(org.springframework.ai.chat.observation.ChatModelObservationConvention observationConvention)	Use the provided convention for reporting observation data
reactor.core.publisher.Flux<org.springframework.ai.chat.model.ChatResponse>	stream(org.springframework.ai.chat.prompt.Prompt prompt)	Invoke the model and return the response stream.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.springframework.ai.chat.model.ChatModel

call, call

Methods inherited from interface org.springframework.ai.chat.model.StreamingChatModel

stream, stream

Field Details

DEFAULT_EMIT_FAILURE_HANDLER

public static final reactor.core.publisher.Sinks.EmitFailureHandler DEFAULT_EMIT_FAILURE_HANDLER

Constructor Details

BedrockProxyChatModel

public BedrockProxyChatModel(software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient bedrockRuntimeClient,
 software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeAsyncClient bedrockRuntimeAsyncClient,
 BedrockChatOptions defaultOptions,
 io.micrometer.observation.ObservationRegistry observationRegistry,
 org.springframework.ai.model.tool.ToolCallingManager toolCallingManager)

BedrockProxyChatModel

public BedrockProxyChatModel(software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient bedrockRuntimeClient,
 software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeAsyncClient bedrockRuntimeAsyncClient,
 BedrockChatOptions defaultOptions,
 io.micrometer.observation.ObservationRegistry observationRegistry,
 org.springframework.ai.model.tool.ToolCallingManager toolCallingManager,
 org.springframework.ai.model.tool.ToolExecutionEligibilityPredicate toolExecutionEligibilityPredicate)

Method Details

call

public org.springframework.ai.chat.model.ChatResponse call(org.springframework.ai.chat.prompt.Prompt prompt)

Invoke the model and return the response. https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrockruntime/BedrockRuntimeClient.html#converse

Specified by:

call in interface org.springframework.ai.chat.model.ChatModel

Specified by:

call in interface org.springframework.ai.model.Model<org.springframework.ai.chat.prompt.Prompt,org.springframework.ai.chat.model.ChatResponse>

Returns:

The model invocation response.

getDefaultOptions

public org.springframework.ai.chat.prompt.ChatOptions getDefaultOptions()

Specified by:

getDefaultOptions in interface org.springframework.ai.chat.model.ChatModel

stream

public reactor.core.publisher.Flux<org.springframework.ai.chat.model.ChatResponse> stream(org.springframework.ai.chat.prompt.Prompt prompt)

Invoke the model and return the response stream. https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrockruntime/BedrockRuntimeAsyncClient.html#converseStream

Specified by:

stream in interface org.springframework.ai.chat.model.ChatModel

Specified by:

stream in interface org.springframework.ai.chat.model.StreamingChatModel

Specified by:

stream in interface org.springframework.ai.model.StreamingModel<org.springframework.ai.chat.prompt.Prompt,org.springframework.ai.chat.model.ChatResponse>

Returns:

The model invocation response stream.

converseStream

public reactor.core.publisher.Flux<software.amazon.awssdk.services.bedrockruntime.model.ConverseStreamOutput> converseStream(software.amazon.awssdk.services.bedrockruntime.model.ConverseStreamRequest converseStreamRequest)

Invoke the model and return the response stream. https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrockruntime/BedrockRuntimeAsyncClient.html#converseStream

Parameters:

converseStreamRequest - Model invocation request.

Returns:

The model invocation response stream.

setObservationConvention

public void setObservationConvention(org.springframework.ai.chat.observation.ChatModelObservationConvention observationConvention)

Use the provided convention for reporting observation data

Parameters:

observationConvention - The provided convention

builder

public static BedrockProxyChatModel.Builder builder()