Package org.apache.druid.metadata

Interface SegmentsMetadataManager

  • All Known Implementing Classes:
    SqlSegmentsMetadataManager
    public interface SegmentsMetadataManager
    The difference between this class and org.apache.druid.sql.calcite.schema.MetadataSegmentView is that this class resides in Coordinator's memory, while org.apache.druid.sql.calcite.schema.MetadataSegmentView resides in Broker's memory.

    • Method Detail

      • startPollingDatabasePeriodically

        void startPollingDatabasePeriodically()

      • stopPollingDatabasePeriodically

        void stopPollingDatabasePeriodically()

      • isPollingDatabasePeriodically

        boolean isPollingDatabasePeriodically()

      • markAsUsedAllNonOvershadowedSegmentsInDataSource

        int markAsUsedAllNonOvershadowedSegmentsInDataSource​(String dataSource)
        Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as used). If the call results in a database error, an exception is relayed to the caller.

      • markAsUsedNonOvershadowedSegmentsInInterval

        int markAsUsedNonOvershadowedSegmentsInInterval​(String dataSource,
                                                org.joda.time.Interval interval)

      • markSegmentAsUsed

        boolean markSegmentAsUsed​(String segmentId)
        Returns true if the state of the segment entry is changed in the database as the result of this call (that is, the segment was marked as used), false otherwise. If the call results in a database error, an exception is relayed to the caller.

      • markAsUnusedAllSegmentsInDataSource

        int markAsUnusedAllSegmentsInDataSource​(String dataSource)
        Returns the number of segment entries in the database whose state was changed as the result of this call (that is, the segments were marked as unused). If the call results in a database error, an exception is relayed to the caller.

      • markAsUnusedSegmentsInInterval

        int markAsUnusedSegmentsInInterval​(String dataSource,
                                   org.joda.time.Interval interval)

      • markSegmentsAsUnused

        int markSegmentsAsUnused​(Set<org.apache.druid.timeline.SegmentId> segmentIds)

      • markSegmentAsUnused

        boolean markSegmentAsUnused​(org.apache.druid.timeline.SegmentId segmentId)
        Returns true if the state of the segment entry is changed in the database as the result of this call (that is, the segment was marked as unused), false otherwise. If the call results in a database error, an exception is relayed to the caller.

      • getImmutableDataSourceWithUsedSegments

        @Nullable
ImmutableDruidDataSource getImmutableDataSourceWithUsedSegments​(String dataSource)
        If there are used segments belonging to the given data source this method returns them as an ImmutableDruidDataSource object. If there are no used segments belonging to the given data source this method returns null.

      • getSnapshotOfDataSourcesWithAllUsedSegments

        DataSourcesSnapshot getSnapshotOfDataSourcesWithAllUsedSegments()
        Returns a snapshot of DruidDataSources and overshadowed segments

      • iterateAllUsedSegments

        Iterable<org.apache.druid.timeline.DataSegment> iterateAllUsedSegments()
        Returns an iterable to go over all segments in all data sources. The order in which segments are iterated is unspecified. Note: the iteration may not be as trivially cheap as, for example, iteration over an ArrayList. Try (to some reasonable extent) to organize the code so that it iterates the returned iterable only once rather than several times.

      • iterateAllUsedNonOvershadowedSegmentsForDatasourceInterval

        com.google.common.base.Optional<Iterable<org.apache.druid.timeline.DataSegment>> iterateAllUsedNonOvershadowedSegmentsForDatasourceInterval​(String datasource,
                                                                                                                                            org.joda.time.Interval interval,
                                                                                                                                            boolean requiresLatest)
        Returns an iterable to go over all used and non-overshadowed segments of given data sources over given interval. The order in which segments are iterated is unspecified. Note: the iteration may not be as trivially cheap as, for example, iteration over an ArrayList. Try (to some reasonable extent) to organize the code so that it iterates the returned iterable only once rather than several times. If {@param requiresLatest} is true then a force metadatastore poll will be triggered. This can cause a longer response time but will ensure that the latest segment information (at the time this method is called) is returned. If {@param requiresLatest} is false then segment information from stale snapshot of up to the last periodic poll period SqlSegmentsMetadataManager.periodicPollDelay will be used.

      • retrieveAllDataSourceNames

        Set<String> retrieveAllDataSourceNames()
        Retrieves all data source names for which there are segment in the database, regardless of whether those segments are used or not. If there are no segments in the database, returns an empty set. Performance warning: this method makes a query into the database. This method might return a different set of data source names than may be observed via getImmutableDataSourcesWithAllUsedSegments() method. This method will include a data source name even if there are no used segments belonging to it, while getImmutableDataSourcesWithAllUsedSegments() won't return such a data source.

      • getUnusedSegmentIntervals

        List<org.joda.time.Interval> getUnusedSegmentIntervals​(String dataSource,
                                                       org.joda.time.DateTime minStartTime,
                                                       org.joda.time.DateTime maxEndTime,
                                                       int limit,
                                                       org.joda.time.DateTime maxUsedFlagLastUpdatedTime)
        Returns top N unused segment intervals with the start time no earlier than the specified start time (if not null) and with the end time no later than the specified maxEndTime and with sed_status_last_updated time no later than maxLastUsedTime when ordered by segment start time, end time. Any segment having no used_status_last_updated time due to upgrade from legacy Druid means maxUsedFlagLastUpdatedTime is ignored for that segment.

      • poll

        void poll()

      • populateUsedFlagLastUpdatedAsync

        void populateUsedFlagLastUpdatedAsync()
        Populates used_status_last_updated column in the segments table iteratively until there are no segments with a NULL value for that column.

      • stopAsyncUsedFlagLastUpdatedUpdate

        void stopAsyncUsedFlagLastUpdatedUpdate()