com.metaweb.lessen.tokenizers
Class BufferedTokenizer

java.lang.Object
  com.metaweb.lessen.tokenizers.BufferedTokenizer

All Implemented Interfaces:

Tokenizer

public class BufferedTokenizer
extends Object
implements Tokenizer

Field Summary

protected Tokenizer	_tokenizer
protected List<Token>	_tokens

Constructor Summary

BufferedTokenizer(Tokenizer tokenizer)

Method Summary

protected boolean	getMore()
Token	getToken()
Token	getToken(int lookahead)
int	lookOverWhitespaceAndComment(int lookahead)
void	next()
void	next(int count)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_tokenizer

protected final Tokenizer _tokenizer

_tokens

protected final List<Token> _tokens

Constructor Detail

BufferedTokenizer

public BufferedTokenizer(Tokenizer tokenizer)

Method Detail

getToken

public Token getToken(int lookahead)

getToken

public Token getToken()

Specified by:

getToken in interface Tokenizer

next

public void next(int count)

next

public void next()

Specified by:

next in interface Tokenizer

lookOverWhitespaceAndComment

public int lookOverWhitespaceAndComment(int lookahead)

getMore

protected boolean getMore()