org.apache.flink.cdc.common.text

接口 TokenStream.Tokenizer

所有已知实现类:

Predicates.RegExSplitterByComma, Predicates.RegExSplitterByDot

封闭类:

TokenStream

public static interface TokenStream.Tokenizer

Interface for a Tokenizer component responsible for processing the characters in a TokenStream.CharacterStream and constructing the appropriate TokenStream.Token objects.

方法概要

限定符和类型	方法和说明
void	tokenize(TokenStream.CharacterStream input, TokenStream.Tokens tokens) Process the supplied characters and construct the appropriate TokenStream.Token objects.

方法详细资料

tokenize

void tokenize(TokenStream.CharacterStream input,
              TokenStream.Tokens tokens)
       throws ParsingException

Process the supplied characters and construct the appropriate TokenStream.Token objects.

参数:

input - the character input stream; never null

tokens - the factory for TokenStream.Token objects, which records the order in which the tokens are created

抛出:

ParsingException - if there is an error while processing the character stream (e.g., a quote is not closed, etc.)