Package org.apache.druid.data.input

Class SegmentsSplitHintSpec

java.lang.Object
org.apache.druid.data.input.SegmentsSplitHintSpec
All Implemented Interfaces:
SplitHintSpec
public class SegmentsSplitHintSpec extends Object implements SplitHintSpec
SplitHintSpec for DruidInputSource. In DruidInputSource, this spec is converted into MaxSizeSplitHintSpec. As a result, its split(java.util.Iterator<T>, java.util.function.Function<T, org.apache.druid.data.input.InputFileAttribute>) method is never called. This doesn't necessarily mean this class is deprecated in favor of the MaxSizeSplitHintSpec. We may want to create more optimized splits in the future. For example, segments can be split to maximize the rollup ratio if the segments have different sets of columns or even different value ranges of columns.

  • Field Details

  • Constructor Details

    • SegmentsSplitHintSpec

      public SegmentsSplitHintSpec(@Nullable HumanReadableBytes maxInputSegmentBytesPerTask, @Nullable Integer maxNumSegments)

  • Method Details

    • getMaxInputSegmentBytesPerTask

      public HumanReadableBytes getMaxInputSegmentBytesPerTask()

    • getMaxNumSegments

      public int getMaxNumSegments()

    • split

      public <T> Iterator<List<T>> split(Iterator<T> inputIterator, Function<T,InputFileAttribute> inputAttributeExtractor)
      Description copied from interface: SplitHintSpec
      Returns an iterator of splits. A split has a list of files of the type SplitHintSpec.
      Specified by:
      split in interface SplitHintSpec
      Parameters:
      inputIterator - that returns input files.
      inputAttributeExtractor - to create InputFileAttribute for each input file. This may involve a network call, so implementations of SplitHintSpec should use it only if needed, and reuse results if appropriate.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object