Package org.dizitart.no2.index.fulltext

Class EnglishTextTokenizer

java.lang.Object

org.dizitart.no2.index.fulltext.BaseTextTokenizer

org.dizitart.no2.index.fulltext.EnglishTextTokenizer

All Implemented Interfaces:

TextTokenizer

public class EnglishTextTokenizer
extends BaseTextTokenizer

A TextTokenizer implementation for the English languages.

Since:

1.0

Author:

Anindya Chatterjee.

Constructor Summary

Constructors

Constructor	Description
EnglishTextTokenizer()

Method Summary

Modifier and Type	Method	Description
Languages	getLanguage()	Gets the language for the tokenizer.
Set<String>	stopWords()	Gets all stop-words for a language.

Methods inherited from class org.dizitart.no2.index.fulltext.BaseTextTokenizer

tokenize

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

EnglishTextTokenizer

public EnglishTextTokenizer()

Method Details

getLanguage

public Languages getLanguage()

Description copied from interface: TextTokenizer

Gets the language for the tokenizer.

Returns:

the language for this tokenizer.

stopWords

public Set<String> stopWords()

Description copied from interface: TextTokenizer

Gets all stop-words for a language.

Returns:

the set of all stop-words.