Uses of Class
weka.core.tokenizers.Tokenizer

Packages that use Tokenizer

Package	Description
weka.classifiers.bayes
weka.classifiers.functions
weka.core
weka.core.converters
weka.core.tokenizers
weka.filters.unsupervised.attribute

Uses of Tokenizer in weka.classifiers.bayes

Methods in weka.classifiers.bayes that return Tokenizer

Modifier and Type	Method	Description
Tokenizer	NaiveBayesMultinomialText.getTokenizer()	Returns the current tokenizer algorithm.

Methods in weka.classifiers.bayes with parameters of type Tokenizer

Modifier and Type	Method	Description
void	NaiveBayesMultinomialText.setTokenizer(Tokenizer value)	the tokenizer algorithm to use.

Uses of Tokenizer in weka.classifiers.functions

Methods in weka.classifiers.functions that return Tokenizer

Modifier and Type	Method	Description
Tokenizer	SGDText.getTokenizer()	Returns the current tokenizer algorithm.

Methods in weka.classifiers.functions with parameters of type Tokenizer

Modifier and Type	Method	Description
void	SGDText.setTokenizer(Tokenizer value)	the tokenizer algorithm to use.

Uses of Tokenizer in weka.core

Methods in weka.core that return Tokenizer

Modifier and Type	Method	Description
Tokenizer	DictionaryBuilder.getTokenizer()	Returns the current tokenizer algorithm.

Methods in weka.core with parameters of type Tokenizer

Modifier and Type	Method	Description
void	DictionaryBuilder.setTokenizer(Tokenizer value)	the tokenizer algorithm to use.

Uses of Tokenizer in weka.core.converters

Methods in weka.core.converters that return Tokenizer

Modifier and Type	Method	Description
Tokenizer	DictionarySaver.getTokenizer()	Returns the current tokenizer algorithm.

Methods in weka.core.converters with parameters of type Tokenizer

Modifier and Type	Method	Description
void	DictionarySaver.setTokenizer(Tokenizer value)	the tokenizer algorithm to use.

Uses of Tokenizer in weka.core.tokenizers

Subclasses of Tokenizer in weka.core.tokenizers

Modifier and Type	Class	Description
class	AlphabeticTokenizer	Alphabetic string tokenizer, tokens are to be formed only from contiguous alphabetic sequences.
class	CharacterDelimitedTokenizer	Abstract superclass for tokenizers that take characters as delimiters.
class	CharacterNGramTokenizer	Splits a string into an n-gram with min and max grams.
class	NGramTokenizer	Splits a string into an n-gram with min and max grams.
class	WordTokenizer	A simple tokenizer that is using the java.util.StringTokenizer class to tokenize the strings.

Methods in weka.core.tokenizers with parameters of type Tokenizer

Modifier and Type	Method	Description
static void	Tokenizer.runTokenizer(Tokenizer tokenizer, String[] options)	initializes the given tokenizer with the given options and runs the tokenizer over all the remaining strings in the options array.
static String[]	Tokenizer.tokenize(Tokenizer tokenizer, String[] options)	initializes the given tokenizer with the given options and runs the tokenizer over all the remaining strings in the options array.

Uses of Tokenizer in weka.filters.unsupervised.attribute

Methods in weka.filters.unsupervised.attribute that return Tokenizer

Modifier and Type	Method	Description
Tokenizer	FixedDictionaryStringToWordVector.getTokenizer()	Returns the current tokenizer algorithm.
Tokenizer	StringToWordVector.getTokenizer()	Returns the current tokenizer algorithm.

Methods in weka.filters.unsupervised.attribute with parameters of type Tokenizer

Modifier and Type	Method	Description
void	FixedDictionaryStringToWordVector.setTokenizer(Tokenizer value)	the tokenizer algorithm to use.
void	StringToWordVector.setTokenizer(Tokenizer value)	the tokenizer algorithm to use.