Package com.alibaba.cloud.ai.graph.agent.tool.multimodal

Class ToolMultimodalResult

java.lang.Object

com.alibaba.cloud.ai.graph.agent.tool.multimodal.ToolMultimodalResult

public final class ToolMultimodalResult
extends Object

Generic result type for tools that return multimodal content (image, audio, etc.).

Reuses Media from Spring AI to support both URL references and raw byte data. A custom MultimodalToolCallResultConverter converts this to a structured string for the LLM and for frontend parsing.

Usage example:

 // From URL
 return ToolMultimodalResult.builder()
         .text("Image generated.")
         .media(ToolMultimodalResult.mediaFromUrl(url, MimeTypeUtils.IMAGE_PNG))
         .build();

 // From URL and base64 (when model returns both - converter picks by OutputFormat)
 return ToolMultimodalResult.builder()
         .text("Image generated.")
         .media(ToolMultimodalResult.mediaFromUrlAndBase64(url, b64Json, MimeTypeUtils.IMAGE_PNG))
         .build();

 // From byte data
 return ToolMultimodalResult.builder()
         .text("Audio generated.")
         .media(ToolMultimodalResult.mediaFromBytes(audioBytes, MimeTypeUtils.parseMimeType("audio/mpeg")))
         .build();
 

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ToolMultimodalResult.Builder
static final record	ToolMultimodalResult.MediaFormats	Holds both URL and base64 when available.

Method Summary

Modifier and Type	Method	Description
static ToolMultimodalResult.Builder	builder()
List<org.springframework.ai.content.Media>	media()
static org.springframework.ai.content.Media	mediaFromBase64(String base64, org.springframework.util.MimeType mimeType)	Create Media from base64 string only (when model returns b64_json directly).
static org.springframework.ai.content.Media	mediaFromBytes(byte[] data, org.springframework.util.MimeType mimeType)	Create Media from raw byte data.
static org.springframework.ai.content.Media	mediaFromResource(org.springframework.core.io.Resource resource, org.springframework.util.MimeType mimeType)	Create Media from a Spring Resource.
static org.springframework.ai.content.Media	mediaFromUri(URI uri, org.springframework.util.MimeType mimeType)	Create Media from a URI.
static org.springframework.ai.content.Media	mediaFromUrl(String url, org.springframework.util.MimeType mimeType)	Create Media from a URL/URI string.
static org.springframework.ai.content.Media	mediaFromUrlAndBase64(String url, String base64, org.springframework.util.MimeType mimeType)	Create Media when both URL and base64 are available (e.g.
static ToolMultimodalResult	of(String text)
static ToolMultimodalResult	of(String text, org.springframework.ai.content.Media... mediaItems)
String	text()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

text

public String text()

media

public List<org.springframework.ai.content.Media> media()

builder

public static ToolMultimodalResult.Builder builder()

of

public static ToolMultimodalResult of(String text)

of

public static ToolMultimodalResult of(String text,
 org.springframework.ai.content.Media... mediaItems)

mediaFromUrl

public static org.springframework.ai.content.Media mediaFromUrl(String url,
 org.springframework.util.MimeType mimeType)

Create Media from a URL/URI string.

mediaFromUri

public static org.springframework.ai.content.Media mediaFromUri(URI uri,
 org.springframework.util.MimeType mimeType)

Create Media from a URI.

mediaFromBytes

public static org.springframework.ai.content.Media mediaFromBytes(byte[] data,
 org.springframework.util.MimeType mimeType)

Create Media from raw byte data.

mediaFromResource

public static org.springframework.ai.content.Media mediaFromResource(org.springframework.core.io.Resource resource,
 org.springframework.util.MimeType mimeType)

Create Media from a Spring Resource.

mediaFromUrlAndBase64

public static org.springframework.ai.content.Media mediaFromUrlAndBase64(String url,
 String base64,
 org.springframework.util.MimeType mimeType)

Create Media when both URL and base64 are available (e.g. from model response). The converter will choose the appropriate format based on OutputFormat, avoiding unnecessary fetch when base64 is already present.

Parameters:

url - URL reference (may be null)

base64 - Base64-encoded data (raw string, not data URL; may be null)

mediaFromBase64

public static org.springframework.ai.content.Media mediaFromBase64(String base64,
 org.springframework.util.MimeType mimeType)

Create Media from base64 string only (when model returns b64_json directly).

Parameters:

base64 - Base64-encoded data (raw string, not data URL)