程序包 io.agentscope.core.tool.multimodal

类 DashScopeMultiModalTool

java.lang.Object
io.agentscope.core.tool.multimodal.DashScopeMultiModalTool
public class DashScopeMultiModalTool extends Object
DashScope multimodal tool. convert text to images, convert images to text, convert text to audio, and convert audio to text. Please refer to the `dashscope documentation` for more details.

  • 构造器详细资料

    • DashScopeMultiModalTool

      public DashScopeMultiModalTool(String apiKey)

  • 方法详细资料

    • dashscopeTextToImage

      @Tool(name="dashscope_text_to_image", description="Generate image(s) based on the given prompt, and return image url(s) or base64 data.") public reactor.core.publisher.Mono<ToolResultBlock> dashscopeTextToImage(@ToolParam(name="prompt",description="The text prompt to generate image") String prompt, @ToolParam(name="model",description="The model to use, e.g., \'wanx-v1\', \'qwen-image\', \'wan2.2-t2i-flash\', etc.",required=false) String model, @ToolParam(name="n",description="The number of images to generate",required=false) Integer n, @ToolParam(name="size",description="Size of the image, e.g., \'1024*1024\', \'1280*1280\', \'800*1200\', etc.",required=false) String size, @ToolParam(name="use_base64",description="Whether to use base64 data for images",required=false) Boolean useBase64)
      Generate image(s) based on the given prompt, and return image url(s) or base64 data.
      参数:
      prompt - The text prompt to generate image
      model - The model to use, e.g., 'wanx-v1', 'qwen-image', 'wan2.2-t2i-flash', etc.
      n - The number of images to generate
      size - Size of the image, e.g., '1024*1024', '1280*1280', '800*1200', etc.
      useBase64 - Whether to use base64 data for images
      返回:
      A ToolResultBlock containing the generated image url, base64 data, or error message.

    • dashscopeImageToText

      @Tool(name="dashscope_image_to_text", description="Generate text based on the given images.") public reactor.core.publisher.Mono<ToolResultBlock> dashscopeImageToText(@ToolParam(name="image_urls",description="The URL(s) of image(s) to be converted into text.") List<String> imageUrls, @ToolParam(name="prompt",description="The text prompt.",required=false) String prompt, @ToolParam(name="model",description="The model to use, e.g., \'qwen3-vl-plus\', qwen-vl-plus\' \'qwen-vl-max\', etc.",required=false) String model)
      Generate text based on the given images.
      参数:
      imageUrls - The URL(s) of image(s) to be converted into text.
      prompt - The text prompt.
      model - The model to use, e.g., 'qwen3-vl-plus', qwen-vl-plus' 'qvq-plus', etc.
      返回:
      A ToolResultBlock containing the generated text or error message.

    • dashscopeTextToAudio

      @Tool(name="dashscope_text_to_audio", description="Convert the given text to audio.") public reactor.core.publisher.Mono<ToolResultBlock> dashscopeTextToAudio(@ToolParam(name="text",description="The text to be converted into audio.") String text, @ToolParam(name="model",description="The TTS model to use. For Qwen TTS: \'qwen3-tts-flash\', \'qwen-tts\'. For Sambert: \'sambert-zhinan-v1\', \'sambert-zhiqi-v1\', \'sambert-zhichu-v1\', etc.",required=false) String model, @ToolParam(name="voice",description="Voice name for Qwen TTS models, e.g., \'Cherry\', \'Serena\'. Ignored for Sambert models.",required=false) String voice, @ToolParam(name="language",description="Language type for Qwen TTS, e.g., \'Chinese\', \'English\'. Ignored for Sambert models.",required=false) String language, @ToolParam(name="sample_rate",description="Sample rate of the audio (e.g., 16000, 24000, 48000).",required=false) Integer sampleRate)
      Convert the given text to audio.

      Supports two types of TTS models:

      • Qwen TTS models (qwen3-tts-flash, qwen-tts) - uses multimodal-generation API
      • Sambert models (sambert-*) - uses speech synthesis SDK
      参数:
      text - The text to be converted into audio.
      model - The TTS model to use. For Qwen TTS: 'qwen3-tts-flash', 'qwen-tts'. For Sambert: 'sambert-zhinan-v1', 'sambert-zhiqi-v1', 'sambert-zhichu-v1', etc.
      voice - Voice name for Qwen TTS models, e.g., 'Cherry', 'Serena'. Ignored for Sambert models.
      language - Language type for Qwen TTS, e.g., 'Chinese', 'English'. Ignored for Sambert models.
      sampleRate - Sample rate of the audio (e.g., 16000, 24000, 48000).
      返回:
      A ToolResultBlock containing the base64 data of audio or error message.

    • dashscopeAudioToText

      @Tool(name="dashscope_audio_to_text", description="Convert the given audio to text.") public reactor.core.publisher.Mono<ToolResultBlock> dashscopeAudioToText(@ToolParam(name="audio_url",description="The file path or URL of audio to be converted into text.") String audioUrl, @ToolParam(name="model",description="The TTS model to use, e.g., \'paraformer-realtime-v1\', \'paraformer-realtime-8k-v1\', \'paraformer-realtime-v2\', \'paraformer-realtime-8k-v2\'.",required=false) String model, @ToolParam(name="sample_rate",description="Sample rate of the audio (e.g., 8000, 16000).",required=false) Integer sampleRate)
      Convert the given audio to text.
      参数:
      audioUrl - The file path or URL of audio to be converted into text.
      model - The speech recognition model to use, e.g., 'paraformer-realtime-v1', 'paraformer-realtime-8k-v1', 'paraformer-realtime-v2', 'paraformer-realtime-8k-v2'.
      sampleRate - Sample rate of the audio (e.g., 8000, 16000).
      返回:
      A ToolResultBlock containing the recognition text or error message.

    • sendAudioChunk

      public void sendAudioChunk(String audioUrl, com.alibaba.dashscope.audio.asr.recognition.Recognition recognizer) throws IOException, InterruptedException
      Send audio input stream by chunk to DashScope. marked as public for unit test.
      参数:
      audioUrl - The file path or URL of audio.
      recognizer - DashScope Recognition instance
      抛出:
      IOException - if read failed
      InterruptedException - if interrupted