Package org.apache.druid.segment

Interface IndexMerger

All Known Implementing Classes:

IndexMergerBase, IndexMergerV10, IndexMergerV9

public interface IndexMerger

Field Summary

Fields

Modifier and Type	Field	Description
static final int	INVALID_ROW
static final Logger	log
static final SerializerUtils	SERIALIZER_UTILS
static final int	UNLIMITED_MAX_COLUMNS_TO_MERGE

Method Summary

Modifier and Type	Method	Description
static List<String>	getLexicographicMergedDimensions(List<IndexableAdapter> indexes)	Returns ColumnHolder.TIME_COLUMN_NAME followed by a merged list of dimensions in lexicographic order.
static List<String>	getLongestSharedDimOrder(List<IndexableAdapter> indexes, DimensionsSpec dimensionsSpec)	Gets the longest shared dimension order, including both ColumnHolder.TIME_COLUMN_NAME and dimensions.
static List<String>	getMergedDimensionsFromQueryableIndexes(List<QueryableIndex> indexes, DimensionsSpec dimensionsSpec)	Returns merged dimensions, not including ColumnHolder.TIME_COLUMN_NAME, from a set of indexes.
static List<String>	getMergedDimensionsWithTime(List<IndexableAdapter> indexes, DimensionsSpec dimensionsSpec)	Returns a merged list of dimensions, including ColumnHolder.TIME_COLUMN_NAME, that will be used to build TimeAndDimsPointer.
static boolean	isDimensionOrderingValid(List<IndexableAdapter> indexes, Iterable<String> orderingCandidate)
static TransformableRowIterator	makeRowIteratorWithConvertedDimensionColumns(TransformableRowIterator sourceRowIterator, ColumnValueSelector[] convertedDimensionSelectors, ColumnValueSelector[] convertedMarkedDimensionSelectors)
File	merge(List<IndexableAdapter> indexes, boolean rollup, AggregatorFactory[] metricAggs, File outDir, DimensionsSpec dimensionsSpec, IndexSpec indexSpec, int maxColumnsToMerge)	Only used as a convenience method in tests.
static <T extends Comparable<? super T>> ArrayList<T>	mergeIndexed(List<Iterable<T>> indexedLists)
default File	mergeQueryableIndex(List<QueryableIndex> indexes, boolean rollup, AggregatorFactory[] metricAggs, File outDir, IndexSpec indexSpec, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory, int maxColumnsToMerge)	Merge a collection of QueryableIndex.
File	mergeQueryableIndex(List<QueryableIndex> indexes, boolean rollup, AggregatorFactory[] metricAggs, DimensionsSpec dimensionsSpec, File outDir, IndexSpec indexSpec, IndexSpec indexSpecForIntermediatePersists, ProgressIndicator progress, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory, int maxColumnsToMerge)	Merge a collection of QueryableIndex.
default File	mergeQueryableIndex(List<QueryableIndex> indexes, boolean rollup, AggregatorFactory[] metricAggs, DimensionsSpec dimensionsSpec, File outDir, IndexSpec indexSpec, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory, int maxColumnsToMerge)	Merge a collection of QueryableIndex.
default File	persist(IncrementalIndex index, File outDir, IndexSpec indexSpec, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory)	Equivalent to persist(IncrementalIndex, Interval, File, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory) without a progress indicator and with interval set to IncrementalIndex.getInterval().
File	persist(IncrementalIndex index, org.joda.time.Interval dataInterval, File outDir, IndexSpec indexSpec, ProgressIndicator progress, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory)	Persist an IncrementalIndex to disk in such a way that it can be loaded back up as a QueryableIndex.
default File	persist(IncrementalIndex index, org.joda.time.Interval dataInterval, File outDir, IndexSpec indexSpec, SegmentWriteOutMediumFactory segmentWriteOutMediumFactory)	Equivalent to persist(IncrementalIndex, Interval, File, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory) without a progress indicator.
static List<IndexableAdapter>	toIndexableAdapters(List<QueryableIndex> indexes)
static TransformableRowIterator	toMergedIndexRowIterator(TransformableRowIterator sourceRowIterator, int indexNumber, List<DimensionMergerV9> mergers)	This method applies DimensionMerger.convertSortedSegmentRowValuesToMergedRowValues(int, ColumnValueSelector) to all dimension column selectors of the given sourceRowIterator, using the given index number.

Field Details

log

static final Logger log

SERIALIZER_UTILS

static final SerializerUtils SERIALIZER_UTILS

INVALID_ROW

static final int INVALID_ROW

See Also:

• Constant Field Values

UNLIMITED_MAX_COLUMNS_TO_MERGE

static final int UNLIMITED_MAX_COLUMNS_TO_MERGE

See Also:

• Constant Field Values

Method Details

getMergedDimensionsFromQueryableIndexes

static List<String> getMergedDimensionsFromQueryableIndexes(List<QueryableIndex> indexes,
 @Nullable
 DimensionsSpec dimensionsSpec)

Returns merged dimensions, not including ColumnHolder.TIME_COLUMN_NAME, from a set of indexes.

toIndexableAdapters

static List<IndexableAdapter> toIndexableAdapters(List<QueryableIndex> indexes)

getMergedDimensionsWithTime

static List<String> getMergedDimensionsWithTime(List<IndexableAdapter> indexes,
 @Nullable
 DimensionsSpec dimensionsSpec)

Returns a merged list of dimensions, including ColumnHolder.TIME_COLUMN_NAME, that will be used to build TimeAndDimsPointer.

getLongestSharedDimOrder

@Nullable
static List<String> getLongestSharedDimOrder(List<IndexableAdapter> indexes,
 @Nullable
 DimensionsSpec dimensionsSpec)

Gets the longest shared dimension order, including both ColumnHolder.TIME_COLUMN_NAME and dimensions.

isDimensionOrderingValid

static boolean isDimensionOrderingValid(List<IndexableAdapter> indexes,
 Iterable<String> orderingCandidate)

getLexicographicMergedDimensions

static List<String> getLexicographicMergedDimensions(List<IndexableAdapter> indexes)

Returns ColumnHolder.TIME_COLUMN_NAME followed by a merged list of dimensions in lexicographic order.

mergeIndexed

static <T extends Comparable<? super T>>
ArrayList<T> mergeIndexed(List<Iterable<T>> indexedLists)

persist

default File persist(IncrementalIndex index,
 File outDir,
 IndexSpec indexSpec,
 @Nullable
 SegmentWriteOutMediumFactory segmentWriteOutMediumFactory)
              throws IOException

Equivalent to persist(IncrementalIndex, Interval, File, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory) without a progress indicator and with interval set to IncrementalIndex.getInterval().

Throws:

IOException

persist

default File persist(IncrementalIndex index,
 org.joda.time.Interval dataInterval,
 File outDir,
 IndexSpec indexSpec,
 @Nullable
 SegmentWriteOutMediumFactory segmentWriteOutMediumFactory)
              throws IOException

Equivalent to persist(IncrementalIndex, Interval, File, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory) without a progress indicator.

Throws:

IOException

persist

File persist(IncrementalIndex index,
 org.joda.time.Interval dataInterval,
 File outDir,
 IndexSpec indexSpec,
 ProgressIndicator progress,
 @Nullable
 SegmentWriteOutMediumFactory segmentWriteOutMediumFactory)
      throws IOException

Persist an IncrementalIndex to disk in such a way that it can be loaded back up as a QueryableIndex. This is *not* thread-safe and havoc will ensue if this is called and writes are still occurring on the IncrementalIndex object.

Parameters:

index - the IncrementalIndex to persist

dataInterval - the Interval that the data represents. Typically, this is the same as the interval from the corresponding SegmentId.

outDir - the directory to persist the data to

indexSpec - storage and compression options

progress - an object that will receive progress updates

segmentWriteOutMediumFactory - controls allocation of temporary data structures

Returns:

the index output directory

Throws:

IOException - if an IO error occurs persisting the index

mergeQueryableIndex

default File mergeQueryableIndex(List<QueryableIndex> indexes,
 boolean rollup,
 AggregatorFactory[] metricAggs,
 File outDir,
 IndexSpec indexSpec,
 @Nullable
 SegmentWriteOutMediumFactory segmentWriteOutMediumFactory,
 int maxColumnsToMerge)
                          throws IOException

Merge a collection of QueryableIndex. Only used as a convenience method in tests. In production code, use the full version mergeQueryableIndex(List, boolean, AggregatorFactory[], DimensionsSpec, File, IndexSpec, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory, int).

Throws:

IOException

mergeQueryableIndex

default File mergeQueryableIndex(List<QueryableIndex> indexes,
 boolean rollup,
 AggregatorFactory[] metricAggs,
 DimensionsSpec dimensionsSpec,
 File outDir,
 IndexSpec indexSpec,
 @Nullable
 SegmentWriteOutMediumFactory segmentWriteOutMediumFactory,
 int maxColumnsToMerge)
                          throws IOException

Merge a collection of QueryableIndex. Only used as a convenience method in tests. In production code, use the full version mergeQueryableIndex(List, boolean, AggregatorFactory[], DimensionsSpec, File, IndexSpec, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory, int).

Throws:

IOException

mergeQueryableIndex

File mergeQueryableIndex(List<QueryableIndex> indexes,
 boolean rollup,
 AggregatorFactory[] metricAggs,
 @Nullable
 DimensionsSpec dimensionsSpec,
 File outDir,
 IndexSpec indexSpec,
 IndexSpec indexSpecForIntermediatePersists,
 ProgressIndicator progress,
 @Nullable
 SegmentWriteOutMediumFactory segmentWriteOutMediumFactory,
 int maxColumnsToMerge)
                  throws IOException

Merge a collection of QueryableIndex.

Throws:

IOException

merge

File merge(List<IndexableAdapter> indexes,
 boolean rollup,
 AggregatorFactory[] metricAggs,
 File outDir,
 DimensionsSpec dimensionsSpec,
 IndexSpec indexSpec,
 int maxColumnsToMerge)
    throws IOException

Only used as a convenience method in tests. In production code, to merge multiple QueryableIndex, use mergeQueryableIndex(List, boolean, AggregatorFactory[], DimensionsSpec, File, IndexSpec, IndexSpec, ProgressIndicator, SegmentWriteOutMediumFactory, int). To merge multiple IncrementalIndex, call one of the persist(org.apache.druid.segment.incremental.IncrementalIndex, java.io.File, org.apache.druid.segment.IndexSpec, org.apache.druid.segment.writeout.SegmentWriteOutMediumFactory) methods and then merge the resulting QueryableIndex.

Throws:

IOException

toMergedIndexRowIterator

static TransformableRowIterator toMergedIndexRowIterator(TransformableRowIterator sourceRowIterator,
 int indexNumber,
 List<DimensionMergerV9> mergers)

This method applies DimensionMerger.convertSortedSegmentRowValuesToMergedRowValues(int, ColumnValueSelector) to all dimension column selectors of the given sourceRowIterator, using the given index number.

makeRowIteratorWithConvertedDimensionColumns

static TransformableRowIterator makeRowIteratorWithConvertedDimensionColumns(TransformableRowIterator sourceRowIterator,
 ColumnValueSelector[] convertedDimensionSelectors,
 ColumnValueSelector[] convertedMarkedDimensionSelectors)