Interface SupportsProjectionPushDown


  • @PublicEvolving
    public interface SupportsProjectionPushDown
    Enables to push down a (possibly nested) projection into a ScanTableSource.

    Given the following SQL:

    
     CREATE TABLE t (i INT, r ROW < d DOUBLE, b BOOLEAN>, s STRING);
     SELECT s, r.d FROM t;
     

    In the above example, r.d and s are required fields. Other fields can be skipped in a projection. Compared to table's schema, fields are reordered.

    By default, if this interface is not implemented, a projection is applied in a subsequent operation after the source.

    For efficiency, a source can push a projection further down in order to be close to the actual data generation. A projection is only selecting fields that are used by a query (possibly in a different field order). It does not contain any computation. A projection can either be performed on the fields of the top-level row only or consider nested fields as well (see supportsNestedProjection()).

    See Also:
    Projection, ProjectedRowData
    • Method Detail

      • supportsNestedProjection

        boolean supportsNestedProjection()
        Returns whether this source supports nested projection.
      • applyProjection

        default void applyProjection​(int[][] projectedFields,
                                     DataType producedDataType)
        Provides the field index paths that should be used for a projection. The indices are 0-based and support fields within (possibly nested) structures if this is enabled via supportsNestedProjection().

        In the example mentioned in SupportsProjectionPushDown, this method would receive:

        Note: Use the passed data type instead of ResolvedSchema.toPhysicalRowDataType() for describing the final output data type when creating TypeInformation.

        Parameters:
        projectedFields - field index paths of all fields that must be present in the physically produced data
        producedDataType - the final output type of the source, with the projection applied