Package org.apache.druid.segment.virtual

Class NestedFieldVirtualColumn

java.lang.Object

org.apache.druid.segment.virtual.NestedFieldVirtualColumn

All Implemented Interfaces:

Cacheable, VirtualColumn

public class NestedFieldVirtualColumn
extends Object
implements VirtualColumn

Optimized virtual column that can make direct selectors into a NestedDataComplexColumn or any associated nested fields (NestedFieldDictionaryEncodedColumn) including using their indexes.

This virtual column is used for the SQL operators JSON_VALUE (if isProcessFromRaw() is set to false) or JSON_QUERY (if it is true), and accepts 'JSONPath' or 'jq' syntax string representations of paths, or a parsed list of NestedPathPart in order to determine what should be selected from the column.

Type information for nested fields is completely absent in the SQL planner, so it guesses the best it can to set getExpectedType() from the context of how something is being used, e.g. an aggregators default type or an explicit cast, or, if using the 'RETURNING' syntax which explicitly specifies type. This might not be the same as if it had actual type information, but, we try to stick with whatever we chose there to do the best we can for now.

Since capabilities(ColumnInspector, String) is determined by the getExpectedType(), the results will be best effor cast to the expected type if the column is not natively the expected type so that this column can fulfill the contract of the type of selector that is likely to be created to read this column.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	NestedFieldVirtualColumn.FieldDimensionSelector	Create a DimensionSelector for a nested field on top of a ColumnValueSelector
static final class	NestedFieldVirtualColumn.RawFieldVectorObjectSelector	Process the "raw" data to extract vectors of values with NestedPathFinder.find(Object, List), returning a value matching an expected type.

Nested classes/interfaces inherited from interface org.apache.druid.segment.VirtualColumn

VirtualColumn.EquivalenceKey

Constructor Summary

Constructors

Constructor	Description
NestedFieldVirtualColumn(String columnName, String path, String outputName)
NestedFieldVirtualColumn(String columnName, String path, String outputName, ColumnType expectedType)
NestedFieldVirtualColumn(String columnName, String outputName, ColumnType expectedType, List<NestedPathPart> parts, Boolean processFromRaw, String path, Boolean useJqSyntax)

Method Summary

Modifier and Type	Method	Description
boolean	canVectorize(ColumnInspector inspector)
ColumnCapabilities	capabilities(String columnName)	This method is deprecated in favor of VirtualColumn.capabilities(ColumnInspector, String), which should be used whenever possible and can support virtual column implementations that need to inspect other columns as inputs.
ColumnCapabilities	capabilities(ColumnInspector inspector, String columnName)	Return the ColumnCapabilities which best describe the optimal selector to read from this virtual column.
boolean	equals(Object o)
byte[]	getCacheKey()	Get a byte array used as a cache key.
String	getColumnName()
VirtualColumn.EquivalenceKey	getEquivalanceKey()	Returns a key used for "equivalence" comparisons, for checking if some virtual column is equivalent to some other virtual column, regardless of the output name.
ColumnType	getExpectedType()
ColumnIndexSupplier	getIndexSupplier(String columnName, ColumnIndexSelector indexSelector)	Get the ColumnIndexSupplier of the specified virtual column, with the assistance of a ColumnSelector to allow reading things from segments.
String	getOutputName()	Output name of this column.
List<NestedPathPart>	getPathParts()
int	hashCode()
boolean	isProcessFromRaw()
ColumnValueSelector<?>	makeColumnValueSelector(String columnName, ColumnSelectorFactory selectorFactory, ColumnSelector columnSelector, ReadableOffset offset)	Builds a selector corresponding to this virtual column.
DimensionSelector	makeDimensionSelector(DimensionSpec dimensionSpec, ColumnSelectorFactory selectorFactory, ColumnSelector columnSelector, ReadableOffset offset)	Build a selector corresponding to this virtual column.
SingleValueDimensionVectorSelector	makeSingleValueVectorDimensionSelector(DimensionSpec dimensionSpec, VectorColumnSelectorFactory selectorFactory, ColumnSelector columnSelector, ReadableVectorOffset offset)	Build a selector corresponding to this virtual column.
VectorObjectSelector	makeVectorObjectSelector(String columnName, VectorColumnSelectorFactory selectorFactory, ColumnSelector columnSelector, ReadableVectorOffset offset)	Build a selector corresponding to this virtual column.
VectorValueSelector	makeVectorValueSelector(String columnName, VectorColumnSelectorFactory selectorFactory, ColumnSelector columnSelector, ReadableVectorOffset offset)	Build a selector corresponding to this virtual column.
List<String>	requiredColumns()	Returns a list of columns that this virtual column will access.
VirtualColumn	rewriteRequiredColumns(Map<String,String> columnRewrites)	Return a copy of this virtual column that is identical to this virtual column except that it operates on different columns, based on a renaming map where the key is the column to be renamed and the value is the new column.
boolean	supportsRequiredRewrite()	Returns true if the virtual column supports VirtualColumn.rewriteRequiredColumns(Map)
String	toString()
boolean	usesDotNotation()	Indicates that this virtual column can be referenced with dot notation.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.druid.segment.VirtualColumn

makeColumnValueSelector, makeColumnValueSelector, makeDimensionSelector, makeDimensionSelector, makeGroupByVectorColumnSelector, makeMultiValueVectorDimensionSelector, makeMultiValueVectorDimensionSelector, makeMultiValueVectorDimensionSelector, makeSingleValueVectorDimensionSelector, makeSingleValueVectorDimensionSelector, makeVectorObjectSelector, makeVectorObjectSelector, makeVectorValueSelector, makeVectorValueSelector, toSelectableColumn

Constructor Details

NestedFieldVirtualColumn

public NestedFieldVirtualColumn(String columnName,
 String outputName,
 @Nullable
 ColumnType expectedType,
 @Nullable
 List<NestedPathPart> parts,
 @Nullable
 Boolean processFromRaw,
 @Nullable
 String path,
 @Nullable
 Boolean useJqSyntax)

NestedFieldVirtualColumn

public NestedFieldVirtualColumn(String columnName,
 String path,
 String outputName)

NestedFieldVirtualColumn

public NestedFieldVirtualColumn(String columnName,
 String path,
 String outputName,
 @Nullable
 ColumnType expectedType)

Method Details

getCacheKey

@Nullable
public byte[] getCacheKey()

Description copied from interface: Cacheable

Get a byte array used as a cache key.

Specified by:

getCacheKey in interface Cacheable

Returns:

bytes to be used as cache key - or null if this object should not be cached.

getOutputName

public String getOutputName()

Description copied from interface: VirtualColumn

Output name of this column.

Specified by:

getOutputName in interface VirtualColumn

Returns:

name

getColumnName

public String getColumnName()

getPathParts

public List<NestedPathPart> getPathParts()

getExpectedType

@Nullable
public ColumnType getExpectedType()

isProcessFromRaw

public boolean isProcessFromRaw()

makeDimensionSelector

public DimensionSelector makeDimensionSelector(DimensionSpec dimensionSpec,
 ColumnSelectorFactory selectorFactory,
 @Nullable
 ColumnSelector columnSelector,
 @Nullable
 ReadableOffset offset)

Description copied from interface: VirtualColumn

Build a selector corresponding to this virtual column. The virtual column is expected to apply any necessary DimensionSpec.decorate(DimensionSelector) or DimensionSpec.getExtractionFn() from the dimensionSpec.

Specified by:

makeDimensionSelector in interface VirtualColumn

Parameters:

dimensionSpec - spec the column was referenced with. Also provides the name that the virtual column was referenced with, which is useful if this column uses dot notation.

selectorFactory - object for fetching underlying selectors.

columnSelector - object for fetching underlying columns, if available. Generally only available for regular segments.

offset - offset to use with underlying columns. Available only if columnSelector is available.

makeColumnValueSelector

public ColumnValueSelector<?> makeColumnValueSelector(String columnName,
 ColumnSelectorFactory selectorFactory,
 @Nullable
 ColumnSelector columnSelector,
 @Nullable
 ReadableOffset offset)

Description copied from interface: VirtualColumn

Builds a selector corresponding to this virtual column.

Specified by:

makeColumnValueSelector in interface VirtualColumn

Parameters:

columnName - name the column was referenced with, which is useful if this column uses dot notation.

selectorFactory - object for fetching underlying selectors.

columnSelector - object for fetching underlying columns, if available. Generally only available for regular segments.

offset - offset to use with underlying columns. Available only if columnSelector is available.

canVectorize

public boolean canVectorize(ColumnInspector inspector)

Specified by:

canVectorize in interface VirtualColumn

makeSingleValueVectorDimensionSelector

public SingleValueDimensionVectorSelector makeSingleValueVectorDimensionSelector(DimensionSpec dimensionSpec,
 VectorColumnSelectorFactory selectorFactory,
 ColumnSelector columnSelector,
 ReadableVectorOffset offset)

Description copied from interface: VirtualColumn

Build a selector corresponding to this virtual column. The virtual column is expected to apply any necessary DimensionSpec.decorate(DimensionSelector) or DimensionSpec.getExtractionFn() from the dimensionSpec.

Specified by:

makeSingleValueVectorDimensionSelector in interface VirtualColumn

Parameters:

dimensionSpec - spec the column was referenced with. Also provides the name that the virtual column was referenced with, which is useful if this column uses dot notation.

selectorFactory - object for fetching underlying selectors.

columnSelector - object for fetching underlying columns.

offset - offset to use with underlying columns.

makeVectorObjectSelector

public VectorObjectSelector makeVectorObjectSelector(String columnName,
 VectorColumnSelectorFactory selectorFactory,
 ColumnSelector columnSelector,
 ReadableVectorOffset offset)

Description copied from interface: VirtualColumn

Build a selector corresponding to this virtual column. The virtual column is expected to apply any necessary DimensionSpec.decorate(DimensionSelector) or DimensionSpec.getExtractionFn() from the dimensionSpec.

Specified by:

makeVectorObjectSelector in interface VirtualColumn

Parameters:

columnName - name the column was referenced with, which is useful if this column uses dot notation.

selectorFactory - object for fetching underlying selectors.

columnSelector - object for fetching underlying columns, if available. Generally only available for regular segments.

offset - offset to use with underlying columns. Available only if columnSelector is available.

makeVectorValueSelector

public VectorValueSelector makeVectorValueSelector(String columnName,
 VectorColumnSelectorFactory selectorFactory,
 ColumnSelector columnSelector,
 ReadableVectorOffset offset)

Description copied from interface: VirtualColumn

Build a selector corresponding to this virtual column. The virtual column is expected to apply any necessary DimensionSpec.decorate(DimensionSelector) or DimensionSpec.getExtractionFn() from the dimensionSpec.

Specified by:

makeVectorValueSelector in interface VirtualColumn

Parameters:

columnName - name the column was referenced with, which is useful if this column uses dot notation.

selectorFactory - object for fetching underlying selectors.

columnSelector - object for fetching underlying columns.

offset - offset to use with underlying columns.

getIndexSupplier

@Nullable
public ColumnIndexSupplier getIndexSupplier(String columnName,
 ColumnIndexSelector indexSelector)

Description copied from interface: VirtualColumn

Get the ColumnIndexSupplier of the specified virtual column, with the assistance of a ColumnSelector to allow reading things from segments. Returns null if the virtual column wants to act like a missing column. Returns NoIndexesColumnIndexSupplier.getInstance() if the virtual column does not support indexes and wants cursor-based filtering.

Specified by:

getIndexSupplier in interface VirtualColumn

capabilities

public ColumnCapabilities capabilities(String columnName)

Description copied from interface: VirtualColumn

This method is deprecated in favor of VirtualColumn.capabilities(ColumnInspector, String), which should be used whenever possible and can support virtual column implementations that need to inspect other columns as inputs. This is a fallback implementation to return the capabilities of this virtual column, which includes a type that corresponds to the best performing base selector supertype (e. g. BaseLongColumnValueSelector) of the object, returned from VirtualColumn.makeColumnValueSelector(String, ColumnSelectorFactory). May vary based on columnName if this column uses dot notation.

Specified by:

capabilities in interface VirtualColumn

Parameters:

columnName - the name this virtual column was referenced with

Returns:

capabilities, must not be null

capabilities

@Nullable
public ColumnCapabilities capabilities(ColumnInspector inspector,
 String columnName)

Description copied from interface: VirtualColumn

Return the ColumnCapabilities which best describe the optimal selector to read from this virtual column.

The ColumnInspector (most likely corresponding to an underlying ColumnSelectorFactory of a query) allows the virtual column to consider this information if necessary to compute its output type details.

Examples of this include the ExpressionVirtualColumn, which takes input from other columns and uses the ColumnInspector to infer the output type of expressions based on the types of the inputs.

Specified by:

capabilities in interface VirtualColumn

Parameters:

inspector - column inspector to provide additional information of other available columns

columnName - the name this virtual column was referenced with

Returns:

capabilities, or null if the column should be treated as if it doesn't exist.

requiredColumns

public List<String> requiredColumns()

Description copied from interface: VirtualColumn

Returns a list of columns that this virtual column will access. This may include the names of other virtual columns. May be empty if a virtual column doesn't access any underlying columns. Does not pass columnName because there is an assumption that the list of columns needed by a dot-notation supporting virtual column will not vary based on the columnName.

Specified by:

requiredColumns in interface VirtualColumn

Returns:

column names

supportsRequiredRewrite

public boolean supportsRequiredRewrite()

Description copied from interface: VirtualColumn

Returns true if the virtual column supports VirtualColumn.rewriteRequiredColumns(Map)

Specified by:

supportsRequiredRewrite in interface VirtualColumn

rewriteRequiredColumns

public VirtualColumn rewriteRequiredColumns(Map<String,String> columnRewrites)

Description copied from interface: VirtualColumn

Return a copy of this virtual column that is identical to this virtual column except that it operates on different columns, based on a renaming map where the key is the column to be renamed and the value is the new column. Callers should check VirtualColumn.supportsRequiredRewrite() first to ensure that this method is supported.

Specified by:

rewriteRequiredColumns in interface VirtualColumn

Parameters:

columnRewrites - Column rewrite map

Returns:

Copy of this virtual column that operates on new columns based on the rewrite map

usesDotNotation

public boolean usesDotNotation()

Description copied from interface: VirtualColumn

Indicates that this virtual column can be referenced with dot notation. For example, a virtual column named "foo" could be referred to as "foo.bar" with the Cursor it is registered with. In that case, init will be called with columnName "foo.bar" rather than "foo".

Specified by:

usesDotNotation in interface VirtualColumn

Returns:

whether to use dot notation

getEquivalanceKey

@Nullable
public VirtualColumn.EquivalenceKey getEquivalanceKey()

Description copied from interface: VirtualColumn

Returns a key used for "equivalence" comparisons, for checking if some virtual column is equivalent to some other virtual column, regardless of the output name. If this method returns null, it does not participate in equivalence comparisons.

Specified by:

getEquivalanceKey in interface VirtualColumn

See Also:

• VirtualColumns.findEquivalent(VirtualColumns, VirtualColumn)

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object