Interface SupportsLookupCustomShuffle.InputDataPartitioner

  • All Superinterfaces:
    Serializable
    Enclosing interface:
    SupportsLookupCustomShuffle

    @PublicEvolving
    public static interface SupportsLookupCustomShuffle.InputDataPartitioner
    extends Serializable
    This interface is responsible for providing custom partitioning logic for the RowData records. We didn't use Partitioner directly because the input data is always RowData type, and we need to extract all join keys from the input data before send it to partitioner.
    • Method Detail

      • partition

        int partition​(RowData joinKeys,
                      int numPartitions)
        Determining the partition id for each input data.

        This data is projected to only including all join keys before emit to this partitioner.

        Parameters:
        joinKeys - The extracted join key for each input record.
        numPartitions - The total number of partition.
        Returns:
        An integer representing the partition id to which the record should be sent.
      • isDeterministic

        default boolean isDeterministic()
        Returns information about the determinism of this partitioner.

        It returns true if and only if a call to the partition(RowData, int) method is guaranteed to always return the same result given the same joinKeyRow. If the partitioning logic depends on not purely functional like random(), date(), now(), ... this method must return false.

        If this method return false, planner may not apply this partitioner in upsert mode to avoid out-of-order of the changelog events.