Package org.apache.druid.timeline.partition

Class NumberedOverwriteShardSpec

java.lang.Object
org.apache.druid.timeline.partition.NumberedOverwriteShardSpec
All Implemented Interfaces:
OverwriteShardSpec, ShardSpec
public class NumberedOverwriteShardSpec extends Object implements OverwriteShardSpec
This shardSpec is used only for the segments created by overwriting tasks with segment lock enabled. When the segment lock is used, there is a concept of atomic update group which is a set of segments atomically becoming queryable together in Brokers. It is a similar concept to the core partition set (explained NumberedShardSpec), but different in a sense that there is only one core partition set per time chunk while there could be multiple atomic update groups in one time chunk. The atomic update group has the root partition range and the minor version to determine the visibility between atomic update groups; the group of the highest minor version in the same root partition range becomes queryable when they have the same major version (DataSegment.getVersion()). Note that this shardSpec is used only when you overwrite existing segments with segment lock enabled. If the task doesn't overwrite segments, it will use NumberedShardSpec instead even when segment lock is used. Similar to NumberedShardSpec, the size of the atomic update group is determined when the task publishes segments at the end of ingestion. As a result, atomicUpdateGroupSize is set to PartitionIds.UNKNOWN_ATOMIC_UPDATE_GROUP_SIZE first, and updated when publishing segments in SegmentPublisherHelper#annotateShardSpec.
See Also:
  • AtomicUpdateGroup

  • Constructor Details

    • NumberedOverwriteShardSpec

      public NumberedOverwriteShardSpec(int partitionId, int startRootPartitionId, int endRootPartitionId, short minorVersion, short atomicUpdateGroupSize)

    • NumberedOverwriteShardSpec

      public NumberedOverwriteShardSpec(int partitionId, int startRootPartitionId, int endRootPartitionId, short minorVersion)

  • Method Details

    • withAtomicUpdateGroupSize

      public OverwriteShardSpec withAtomicUpdateGroupSize(short atomicUpdateGroupSize)
      Specified by:
      withAtomicUpdateGroupSize in interface OverwriteShardSpec

    • createChunk

      public <T> PartitionChunk<T> createChunk(T obj)
      Specified by:
      createChunk in interface ShardSpec

    • getPartitionNum

      public int getPartitionNum()
      Description copied from interface: ShardSpec
      Returns the partition ID of this segment.
      Specified by:
      getPartitionNum in interface ShardSpec

    • getStartRootPartitionId

      public int getStartRootPartitionId()
      Description copied from interface: ShardSpec
      Returns the start root partition ID of the atomic update group which this segment belongs to.
      Specified by:
      getStartRootPartitionId in interface ShardSpec
      See Also:
      • AtomicUpdateGroup

    • getEndRootPartitionId

      public int getEndRootPartitionId()
      Description copied from interface: ShardSpec
      Returns the end root partition ID of the atomic update group which this segment belongs to.
      Specified by:
      getEndRootPartitionId in interface ShardSpec
      See Also:
      • AtomicUpdateGroup

    • getMinorVersion

      public short getMinorVersion()
      Description copied from interface: ShardSpec
      Returns the minor version associated to the atomic update group which this segment belongs to.
      Specified by:
      getMinorVersion in interface ShardSpec
      See Also:
      • AtomicUpdateGroup

    • getAtomicUpdateGroupSize

      public short getAtomicUpdateGroupSize()
      Description copied from interface: ShardSpec
      Returns the atomic update group size which this segment belongs to.
      Specified by:
      getAtomicUpdateGroupSize in interface ShardSpec
      See Also:
      • AtomicUpdateGroup

    • getLookup

      public ShardSpecLookup getLookup(List<? extends ShardSpec> shardSpecs)
      Specified by:
      getLookup in interface ShardSpec

    • getDomainDimensions

      public List<String> getDomainDimensions()
      Description copied from interface: ShardSpec
      Get dimensions who have possible range for the rows this shard contains.
      Specified by:
      getDomainDimensions in interface ShardSpec
      Returns:
      list of dimensions who has its possible range. Dimensions with unknown possible range are not listed

    • possibleInDomain

      public boolean possibleInDomain(Map<String,com.google.common.collect.RangeSet<String>> domain)
      Description copied from interface: ShardSpec
      if given domain ranges are not possible in this shard, return false; otherwise return true;
      Specified by:
      possibleInDomain in interface ShardSpec
      Returns:
      possibility of in domain

    • getType

      public String getType()
      Description copied from interface: ShardSpec
      Get the type name of this ShardSpec.
      Specified by:
      getType in interface ShardSpec

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object