SimpleTokenizeSplitter (AgentsFlex)

java.lang.Object

com.agentsflex.core.document.splitter.SimpleTokenizeSplitter

所有已实现的接口:: DocumentSplitter

public class SimpleTokenizeSplitter extends Object implements DocumentSplitter

字段概要

字段

修饰符和类型

字段

说明

private int

chunkSize

private com.knuddels.jtokkit.api.EncodingType

encodingType

private int

overlapSize

private com.knuddels.jtokkit.api.EncodingRegistry

registry
构造器概要

构造器

构造器

说明

SimpleTokenizeSplitter(int chunkSize)

SimpleTokenizeSplitter(int chunkSize, int overlapSize)
方法概要

修饰符和类型

方法

说明

int

getChunkSize()

com.knuddels.jtokkit.api.EncodingType

getEncodingType()

int

getOverlapSize()

com.knuddels.jtokkit.api.EncodingRegistry

getRegistry()

void

setChunkSize(int chunkSize)

void

setEncodingType(com.knuddels.jtokkit.api.EncodingType encodingType)

void

setOverlapSize(int overlapSize)

void

setRegistry(com.knuddels.jtokkit.api.EncodingRegistry registry)

List<Document>

split(Document document, DocumentIdGenerator idGenerator)

从类继承的方法 java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 com.agentsflex.core.document.DocumentSplitter
split, splitAll

字段详细资料
- registry
  
  private com.knuddels.jtokkit.api.EncodingRegistry registry
- encodingType
  
  private com.knuddels.jtokkit.api.EncodingType encodingType
- chunkSize
  
  private int chunkSize
- overlapSize
  
  private int overlapSize
构造器详细资料
- SimpleTokenizeSplitter
  
  public SimpleTokenizeSplitter(int chunkSize)
- SimpleTokenizeSplitter
  
  public SimpleTokenizeSplitter(int chunkSize, int overlapSize)
方法详细资料
- getChunkSize
  
  public int getChunkSize()
- setChunkSize
  
  public void setChunkSize(int chunkSize)
- getOverlapSize
  
  public int getOverlapSize()
- setOverlapSize
  
  public void setOverlapSize(int overlapSize)
- getRegistry
  
  public com.knuddels.jtokkit.api.EncodingRegistry getRegistry()
- setRegistry
  
  public void setRegistry(com.knuddels.jtokkit.api.EncodingRegistry registry)
- getEncodingType
  
  public com.knuddels.jtokkit.api.EncodingType getEncodingType()
- setEncodingType
  
  public void setEncodingType(com.knuddels.jtokkit.api.EncodingType encodingType)
- split
  
  public List<Document> split(Document document, DocumentIdGenerator idGenerator)
  
  指定者:
  
  split 在接口中 DocumentSplitter