Package org.apache.druid.segment.loading

Interface DataSegmentKiller

public interface DataSegmentKiller
DataSegmentKiller knows how to kill segments from the Druid system. Since any implementation of DataSegmentKiller is initialized when an ingestion job starts if a deep storage extension is loaded even when that deep storage is actually not used, implementations should avoid initializing the deep storage client immediately but defer it until the deep storage client is actually used.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final Logger
    log
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static String
    descriptorPath(String path)
     
    default void
    kill(List<DataSegment> segments)
    Kills a list of segments from deep storage.
    void
    kill(DataSegment segment)
    Removes segment files (index and metadata) from deep storage.
    void
    killAll()
    Like a nuke.
    default void
    killQuietly(DataSegment segment)
    A more stoic killer who doesn't throw a tantrum if things get messy.
    default void
    killRecursively(String relativePath)
    Recursively removes a directory (or object-store prefix) under the configured deep storage root.

  • Field Details

    • log

      static final Logger log

  • Method Details

    • descriptorPath

      static String descriptorPath(String path)

    • kill

      void kill(DataSegment segment) throws SegmentLoadingException
      Removes segment files (index and metadata) from deep storage.
      Parameters:
      segment - the segment to kill
      Throws:
      SegmentLoadingException - if the segment could not be completely removed

    • kill

      default void kill(List<DataSegment> segments) throws SegmentLoadingException
      Kills a list of segments from deep storage. The default implementation calls kill on the segments in a loop. Implementers of this interface can leverage batch / bulk deletes to be more efficient. It is preferable to attempt to delete all segments even if there is an issue with deleting a single one. This is up to implementers to implement as putting a try catch around the default kill via iteration can be problematic if the client of the deep storage is unable to authenticate itself and segment loading exception doesn't encode enough information in it to \ understand why it failed.

      If a segment or segments does not exist in deep storage, the method should not throw an exception.

      This version of kill must **NOT** require additional permissions on the deep storage beyond what kill(DataSegment) requires.

      Parameters:
      segments - The list of segments to kill.
      Throws:
      SegmentLoadingException - If there is an exception during deletion such as a segment in the list could not be completely removed.

    • killQuietly

      default void killQuietly(DataSegment segment)
      A more stoic killer who doesn't throw a tantrum if things get messy. Use when killing segments for best-effort cleanup.
      Parameters:
      segment - the segment to kill

    • killAll

      void killAll() throws IOException
      Like a nuke. Use wisely. Used by the 'reset-cluster' command, and of the built-in deep storage implementations, it is only implemented by local and HDFS.
      Throws:
      IOException

    • killRecursively

      default void killRecursively(String relativePath) throws IOException
      Recursively removes a directory (or object-store prefix) under the configured deep storage root. The path is relative to that root: no leading slash, no .. segments, no backslashes. If the path does not exist, this is a no-op. The default implementation does nothing; extensions that cannot recurse should keep the default. HDFS currently only implements this method.
      Throws:
      IOException - if deletion fails