com.metaweb.lessen.tokenizers
Class StringTokenizer

java.lang.Object
  com.metaweb.lessen.tokenizers.TokenizerBase
      com.metaweb.lessen.tokenizers.StringTokenizer

All Implemented Interfaces:

Tokenizer

public class StringTokenizer
extends TokenizerBase

Field Summary

protected int	_end
protected int	_next
protected int	_start
protected String	_text

Fields inherited from class com.metaweb.lessen.tokenizers.TokenizerBase

_token

Constructor Summary

StringTokenizer(String text)

StringTokenizer(String text, int start, int end)

Method Summary

protected void	advance()
protected void	advance(int by, boolean flush)
protected void	flush(int upToIndex)
protected char	getCharRelative(int offset)
protected char	getCurrentChar()
protected int	getCurrentOffset()
protected String	getText(int from, int to)
protected boolean	hasMoreChar()
protected boolean	hasMoreChar(int offset)

Methods inherited from class com.metaweb.lessen.tokenizers.TokenizerBase

advance, getToken, isHexDigit, isNameChar, isNameStartChar, isWhitespace, lookOverIdentifier, lookOverName, lookOverNameChar, lookOverNameStartChar, lookOverUnicodeChar, lookOverUnicodeCode, lookOverWhitespace, next, parseNumber, parseRestOfURI, parseString, swallowWhitespace

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_text

protected final String _text

_start

protected final int _start

_end

protected final int _end

_next

protected int _next

Constructor Detail

StringTokenizer

public StringTokenizer(String text,
                       int start,
                       int end)

StringTokenizer

public StringTokenizer(String text)

Method Detail

hasMoreChar

protected boolean hasMoreChar()

Specified by:

hasMoreChar in class TokenizerBase

advance

protected void advance()

Specified by:

advance in class TokenizerBase

advance

protected void advance(int by,
                       boolean flush)

Specified by:

advance in class TokenizerBase

flush

protected void flush(int upToIndex)

Specified by:

flush in class TokenizerBase

getCharRelative

protected char getCharRelative(int offset)

Specified by:

getCharRelative in class TokenizerBase

getCurrentChar

protected char getCurrentChar()

Specified by:

getCurrentChar in class TokenizerBase

getCurrentOffset

protected int getCurrentOffset()

Specified by:

getCurrentOffset in class TokenizerBase

getText

protected String getText(int from,
                         int to)

Specified by:

getText in class TokenizerBase

hasMoreChar

protected boolean hasMoreChar(int offset)

Specified by:

hasMoreChar in class TokenizerBase