Type Members

1.  abstract class AbstractParser extends DataTypeParserInterface with Logging

   Base SQL parsing infrastructure.

2.  class DataTypeAstBuilder extends SqlBaseParserBaseVisitor[AnyRef] with DataTypeErrorsBase

   AST builder for parsing data type definitions and table schemas.

   AST builder for parsing data type definitions and table schemas.

   This is a client-side parser designed specifically for parsing data type strings (e.g., "INT", "STRUCT<name:STRING, age:INT>") and table schemas. It assumes that the input does not contain parameter markers (:name or ?), as parameter substitution should occur before data types are parsed.

   Key characteristics:

   • **Client-side parser**: Used for parsing data type strings provided by users or stored in catalogs, not for parsing full SQL statements.
   • **No parameter markers**: This parser explicitly rejects parameter markers in data type contexts by throwing UNEXPECTED_USE_OF_PARAMETER_MARKER errors.
   • **No string literal coalescing**: This base class does not coalesce consecutive string literals. Coalescing is handled by AstBuilder where SQL configuration is available to determine the correct escape processing mode.

   Examples of valid inputs:

   • Simple types: INT, STRING, DOUBLE
   • Parameterized types: DECIMAL(10,2), VARCHAR(100), CHAR(5)
   • Complex types: ARRAY<INT>, MAP<STRING, INT>, STRUCT<name:STRING, age:INT>
   • Table schemas: id INT, name STRING, created_at TIMESTAMP

   This class extends SqlBaseParserBaseVisitor and provides visitor methods for the grammar rules related to data types.

   See also

   org.apache.spark.sql.catalyst.parser.AstBuilder for the full SQL statement parser

   CRITICAL: Extracting Identifier Names

   When extracting identifier names from parser contexts, you MUST use the helper methods provided by this class instead of calling ctx.getText() directly:

   • getIdentifierText(ctx): For single identifiers (column names, aliases, window names)
   • getIdentifierParts(ctx): For qualified identifiers (table names, schema.table) DO NOT use ctx.getText() or ctx.identifier.getText() directly! These methods do not handle the IDENTIFIER('literal') syntax and will cause incorrect behavior. The IDENTIFIER('literal') syntax allows string literals to be used as identifiers at parse time (e.g., IDENTIFIER('my_col') resolves to the identifier my_col). If you use getText(), you'll get the raw text "IDENTIFIER('my_col')" instead of "my_col", breaking the feature. Example:

   // WRONG - does not handle IDENTIFIER('literal'):
   val name = ctx.identifier.getText
   SubqueryAlias(ctx.name.getText, plan)
   
   // CORRECT - handles both regular identifiers and IDENTIFIER('literal'):
   val name = getIdentifierText(ctx.identifier)
   SubqueryAlias(getIdentifierText(ctx.name), plan)

3.  trait DataTypeParserInterface extends AnyRef

   Interface for DataType parsing functionality.

4.  case class ParameterLocation(start: Int, end: Int) extends Product with Serializable

   Data class representing a parameter location in the source text.

5.  case class ParameterLocationInfo(namedParameterLocations: Map[String, List[ParameterLocation]], positionalParameterLocations: List[ParameterLocation]) extends Product with Serializable

   Data class to hold parameter location information for substitution.

6.  class ParseException extends AnalysisException

   A ParseException is an SparkException that is thrown during the parse process.

   A ParseException is an SparkException that is thrown during the parse process. It contains fields and an extended error message that make reporting and diagnosing errors easier.

7.  case class PositionMapper(originalText: String, substitutedText: String, substitutions: List[Substitution]) extends Product with Serializable

   Maps positions between original SQL text and substituted SQL text using sparse ranges.

   Maps positions between original SQL text and substituted SQL text using sparse ranges.

   This implementation uses O(k) space and O(log k) lookup time where k = number of substitutions.

   originalText

   The original SQL text with parameter markers

   substitutedText

   The SQL text after parameter substitution

   substitutions

   List of substitutions that were applied

8.  case class PositionRange(substitutedStart: Int, substitutedEnd: Int, originalStart: Int, offsetDelta: Int) extends Product with Serializable

   Represents a range mapping from substituted positions to original positions.

   Represents a range mapping from substituted positions to original positions. This is used for efficient O(k) position mapping where k = number of substitutions.

   substitutedStart

   Start position in substituted text (inclusive)

   substitutedEnd

   End position in substituted text (exclusive)

   originalStart

   Start position in original text

   offsetDelta

   Offset difference between original and substituted positions

9.  class SparkParserBailErrorStrategy extends SparkParserErrorStrategy

   Inspired by org.antlr.v4.runtime.BailErrorStrategy, which is used in two-stage parsing: This error strategy allows the first stage of two-stage parsing to immediately terminate if an error is encountered, and immediately fall back to the second stage.

   Inspired by org.antlr.v4.runtime.BailErrorStrategy, which is used in two-stage parsing: This error strategy allows the first stage of two-stage parsing to immediately terminate if an error is encountered, and immediately fall back to the second stage. In addition to avoiding wasted work by attempting to recover from errors here, the empty implementation of sync improves the performance of the first stage.

10.  class SparkParserErrorStrategy extends DefaultErrorStrategy

    A SparkParserErrorStrategy extends the DefaultErrorStrategy, that does special handling on errors.

    A SparkParserErrorStrategy extends the DefaultErrorStrategy, that does special handling on errors.

    The intention of this class is to provide more information of these errors encountered in ANTLR parser to the downstream consumers, to be able to apply the SparkThrowable error message framework to these exceptions.

11.  class SparkRecognitionException extends RecognitionException

    A SparkRecognitionException extends the RecognitionException with more information including the error class and parameters for the error message, which align with the interface of SparkThrowableHelper.

12.  class SqlBaseLexer extends Lexer

    Annotations

    @SuppressWarnings()

13.  class SqlBaseParser extends Parser

    Annotations

    @SuppressWarnings()

14.  class SqlBaseParserBaseListener extends SqlBaseParserListener

    This class provides an empty implementation of SqlBaseParserListener, which can be extended to create a listener which only needs to handle a subset of the available methods.

    This class provides an empty implementation of SqlBaseParserListener, which can be extended to create a listener which only needs to handle a subset of the available methods.

    Annotations

    @SuppressWarnings()

15.  class SqlBaseParserBaseVisitor[T] extends AbstractParseTreeVisitor[T] with SqlBaseParserVisitor[T]

    This class provides an empty implementation of SqlBaseParserVisitor, which can be extended to create a visitor which only needs to handle a subset of the available methods.

    This class provides an empty implementation of SqlBaseParserVisitor, which can be extended to create a visitor which only needs to handle a subset of the available methods.

    Annotations

    @SuppressWarnings()

16.  trait SqlBaseParserListener extends ParseTreeListener

    This interface defines a complete listener for a parse tree produced by SqlBaseParser.

17.  trait SqlBaseParserVisitor[T] extends ParseTreeVisitor[T]

    This interface defines a complete generic visitor for a parse tree produced by SqlBaseParser.

18.  class SubstituteParmsAstBuilder extends SqlBaseParserBaseVisitor[AnyRef]

    AST builder for extracting parameter markers and their locations from SQL parse trees.

    AST builder for extracting parameter markers and their locations from SQL parse trees. This builder traverses the parse tree and collects parameter information for substitution.

19.  case class Substitution(location: ParameterLocation, replacement: String) extends Product with Serializable

    Case class representing a text substitution.

20.  case class UnclosedCommentProcessor(command: String, tokenStream: CommonTokenStream) extends SqlBaseParserBaseListener with Product with Serializable

    The post-processor checks the unclosed bracketed comment.