Package software.amazon.smithy.syntax

Class CapturedToken

java.lang.Object
software.amazon.smithy.syntax.CapturedToken
All Implemented Interfaces:
software.amazon.smithy.model.FromSourceLocation, software.amazon.smithy.utils.ToSmithyBuilder<CapturedToken>
public final class CapturedToken extends Object implements software.amazon.smithy.model.FromSourceLocation, software.amazon.smithy.utils.ToSmithyBuilder<CapturedToken>
A persisted token captured from an IdlTokenizer.

For performance, IdlTokenizer does not create new tokens types for each encountered token. Instead, it updates the current state of the tokenizer and allows the caller to inspect the tokenizer for information about each token. Because smithy-syntax needs to create a token-tree rather than go directly to an AST, it requires arbitrary lookahead of tokens, requiring it to persist tokens in memory.

  • Method Details

    • builder

      public static CapturedToken.Builder builder()

    • from

      public static CapturedToken from(software.amazon.smithy.model.loader.IdlTokenizer tokenizer)
      Persist the current token of an IdlTokenizer.
      Parameters:
      tokenizer - Tokenizer to capture.
      Returns:
      Returns the persisted token.

    • from

      public static CapturedToken from(software.amazon.smithy.model.loader.IdlTokenizer tokenizer, Function<CharSequence,String> stringTable)
      Persist the current token of an IdlTokenizer.
      Parameters:
      tokenizer - Tokenizer to capture.
      stringTable - String table that caches previously created strings.
      Returns:
      Returns the persisted token.

    • toBuilder

      public CapturedToken.Builder toBuilder()
      Specified by:
      toBuilder in interface software.amazon.smithy.utils.ToSmithyBuilder<CapturedToken>

    • getIdlToken

      public software.amazon.smithy.model.loader.IdlToken getIdlToken()
      Get the token IDL token of the captured token.
      Returns:
      Returns the underlying token type.

    • getSourceLocation

      public software.amazon.smithy.model.SourceLocation getSourceLocation()
      Specified by:
      getSourceLocation in interface software.amazon.smithy.model.FromSourceLocation

    • getFilename

      public String getFilename()

    • getPosition

      public int getPosition()

    • getStartLine

      public int getStartLine()

    • getStartColumn

      public int getStartColumn()

    • getEndLine

      public int getEndLine()

    • getEndColumn

      public int getEndColumn()

    • getSpan

      public int getSpan()

    • getLexeme

      public CharSequence getLexeme()
      Get the raw lexeme of the current token.
      Returns:
      Returns the underlying lexeme of the token.

    • getStringContents

      public String getStringContents()
      Get the associated String contents of the token if it's a string, text block, or identifier.
      Returns:
      Returns the string contents of the lexeme, or null if not a string|text block|identifier.

    • getErrorMessage

      public String getErrorMessage()
      Gets the associated error message with the token if it's an error.
      Returns:
      Returns the error message or null if not an error.

    • getNumberValue

      public Number getNumberValue()
      Get the computed Number of the current token if it's a number.
      Returns:
      Returns the computed Number or null if not a number.