com.metaweb.lessen.tokenizers
Class ReaderTokenizer

java.lang.Object
  com.metaweb.lessen.tokenizers.TokenizerBase
      com.metaweb.lessen.tokenizers.ReaderTokenizer

All Implemented Interfaces:

Tokenizer

public class ReaderTokenizer
extends TokenizerBase

Field Summary

protected char[]	_chars
protected boolean	_eof
protected int	_next
protected Reader	_reader
protected StringBuffer	_sb
protected int	_sbStartIndex
protected static int	s_bufferSize

Fields inherited from class com.metaweb.lessen.tokenizers.TokenizerBase

_token

Constructor Summary

ReaderTokenizer(Reader reader)

Method Summary

protected void	advance()
protected void	advance(int by, boolean flush)
protected void	flush(int upToIndex)
protected char	getCharRelative(int offset)
protected char	getCurrentChar()
protected int	getCurrentOffset()
protected String	getText(int from, int to)
protected boolean	hasMoreChar()
protected boolean	hasMoreChar(int offset)
protected void	tryLoadAhead(int distance)

Methods inherited from class com.metaweb.lessen.tokenizers.TokenizerBase

advance, getToken, isHexDigit, isNameChar, isNameStartChar, isWhitespace, lookOverIdentifier, lookOverName, lookOverNameChar, lookOverNameStartChar, lookOverUnicodeChar, lookOverUnicodeCode, lookOverWhitespace, next, parseNumber, parseRestOfURI, parseString, swallowWhitespace

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

s_bufferSize

protected static final int s_bufferSize

See Also:

Constant Field Values

_reader

protected final Reader _reader

_sb

protected final StringBuffer _sb

_chars

protected final char[] _chars

_sbStartIndex

protected int _sbStartIndex

_next

protected int _next

_eof

protected boolean _eof

Constructor Detail

ReaderTokenizer

public ReaderTokenizer(Reader reader)

Method Detail

hasMoreChar

protected boolean hasMoreChar()

Specified by:

hasMoreChar in class TokenizerBase

advance

protected void advance()

Specified by:

advance in class TokenizerBase

advance

protected void advance(int by,
                       boolean flush)

Specified by:

advance in class TokenizerBase

flush

protected void flush(int upToIndex)

Specified by:

flush in class TokenizerBase

getCharRelative

protected char getCharRelative(int offset)

Specified by:

getCharRelative in class TokenizerBase

getCurrentChar

protected char getCurrentChar()

Specified by:

getCurrentChar in class TokenizerBase

getCurrentOffset

protected int getCurrentOffset()

Specified by:

getCurrentOffset in class TokenizerBase

getText

protected String getText(int from,
                         int to)

Specified by:

getText in class TokenizerBase

hasMoreChar

protected boolean hasMoreChar(int offset)

Specified by:

hasMoreChar in class TokenizerBase

tryLoadAhead

protected void tryLoadAhead(int distance)