Package org.apache.druid.frame.processor

Class SuperSorterProgressTracker

java.lang.Object
org.apache.druid.frame.processor.SuperSorterProgressTracker
public class SuperSorterProgressTracker extends Object
A thread-safe class that keeps track of the progress of an n-way, multilevel merge sort. The logic for this class is coupled to the implementation of SuperSorter, which it is designed to track. It keeps a track of the following things: 1. Total levels to merge (Cannnot be modified once set) 2. Total mergers to merge in each level 3. Total merged batches in each level There are a few states in which this progress tracker can be in (depending on the SuperSorter's progres): 1. Unknown everything | totalMergingLevels = -1, mergersForUltimateLevel = UNKNOWN_TOTAL 2. Known total levels, but unknown output partitions | totalMergingLevels = set, mergersForUltimateLevelSet = UNKNOWN_TOTAL 3. Unknown total levels, but known output partitions | totalMergingLevels = -1, mergersForUltimateLevelSet != UNKNOWN_TOTAL 4. Known everything | totalMergingLevels = set, mergersForUltimateLevelSet != UNKNOWN_TOTAL Progress can only be quantified if the total levels are known snapshot function does the housekeeping to hide this operational complexity away from classes other than SuperSorter and SuperSorterProgressTracker

  • Constructor Details

    • SuperSorterProgressTracker

      public SuperSorterProgressTracker()

  • Method Details

    • setTotalMergingLevels

      public void setTotalMergingLevels(int totalMergingLevels)
      Set total merging levels for the SuperSorter it is tracking. Can be set only once

    • setTotalMergersForLevel

      public void setTotalMergersForLevel(int level, long totalMergers)
      Sets the total mergers for a level. Can be set only once, except for the ultimate level (if total levels are known) because they get overridden by totalMergersForUltimateLevel

    • setTotalMergersForUltimateLevel

      public void setTotalMergersForUltimateLevel(long totalMergersForUltimateLevel)
      Sets the number of mergers in the ultimate level (number of mergers = number of output partitions). Can only be set once

    • addMergedBatchesForLevel

      public void addMergedBatchesForLevel(int level, long additionalMergedBatches)
      This method is designed to be called during the course of the sorting. The batches once merged for a particular level can be marked as such through this.

    • markTriviallyComplete

      public void markTriviallyComplete()
      If the SuperSorter is trivially done without doing any work (for eg - empty input), the tracker can be marked as trivially complete. Once a tracker is marked as complete, the snapshots will always report back the progress digest as 1. Any modification to the state of the tracker (eg: calling setTotalMergersForLevel()) would proceed as regular, but

    • snapshot

      public SuperSorterProgressSnapshot snapshot()
      Returns:
      Aggregates the information present in the tracker object and returns a SuperSorterProgressSnapshot representing the same information