Package io.karatelabs.parser

Class Token

java.lang.Object

io.karatelabs.parser.Token

public class Token
extends Object

Lightweight token using flyweight pattern. Core fields (pos, line, col, type, length) are stored directly for performance. Resource and rarely-used data (comments, prev/next) are accessed via TokenBuffer.

Field Summary

Fields

Modifier and Type	Field	Description
final short	col
static final Token	EMPTY
final short	length
final short	line
final int	pos
final TokenType	type

Constructor Summary

Constructors

Constructor	Description
Token(TokenBuffer buffer, TokenType type, int pos, int line, int col, int length)
Token(TokenBuffer buffer, TokenType type, int pos, short line, short col, short length)

Method Summary

Modifier and Type	Method	Description
void	cacheLiteral(Object value)
Object	getCachedLiteral()	Memoized literal value for this token, or null if not computed yet.
List<Token>	getComments()
String	getLineText()
Token	getNext()
Token	getNextPrimary()
String	getPositionDisplay()
Token	getPrev()
Resource	getResource()
String	getText()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

EMPTY

public static final Token EMPTY

pos

public final int pos

line

public final short line

col

public final short col

type

public final TokenType type

length

public final short length

Constructor Details

Token

public Token(TokenBuffer buffer,
 TokenType type,
 int pos,
 short line,
 short col,
 short length)

Token

public Token(TokenBuffer buffer,
 TokenType type,
 int pos,
 int line,
 int col,
 int length)

Method Details

getResource

public Resource getResource()

getText

public String getText()

getCachedLiteral

public Object getCachedLiteral()

Memoized literal value for this token, or null if not computed yet.

cacheLiteral

public void cacheLiteral(Object value)

getNextPrimary

public Token getNextPrimary()

getPrev

public Token getPrev()

getNext

public Token getNext()

getLineText

public String getLineText()

getPositionDisplay

public String getPositionDisplay()

getComments

public List<Token> getComments()

toString

public String toString()

Overrides:

toString in class Object