Class RemoteStorageScanner

  • All Implemented Interfaces:
    Runnable

    public class RemoteStorageScanner
    extends Object
    implements Runnable
    The RemoteStorageScanner is introduced to notify asynchronously for file reading on remote storage. Asynchronous notifications will prevent RemoteTierConsumerAgent from repeatedly attempting to read remote files and reduce CPU consumption.

    It will be invoked by RemoteTierConsumerAgent to watch the required segments and scan the existence status of the segments. If the segment file is found, it will notify the availability of segment file.

    • Constructor Detail

      • RemoteStorageScanner

        public RemoteStorageScanner​(String baseRemoteStoragePath)
    • Method Detail

      • start

        public void start()
        Start the executor.
      • watchSegment

        public void watchSegment​(TieredStoragePartitionId partitionId,
                                 TieredStorageSubpartitionId subpartitionId,
                                 int segmentId)
        Watch the segment for a specific subpartition in the RemoteStorageScanner.

        If a segment with a larger or equal id already exists, the current segment won't be watched.

        If a segment with a smaller segment id is still being watched, the current segment will replace it because the smaller segment should have been consumed. This method ensures that only one segment file can be watched for each subpartition.

        Parameters:
        partitionId - is the id of partition.
        subpartitionId - is the id of subpartition.
        segmentId - is the id of segment.
      • close

        public void close()
        Close the executor.
      • run

        public void run()
        Iterate the watched segment ids and check related file status.
        Specified by:
        run in interface Runnable