All Classes and Interfaces
Class
Description
An abstract class that provides an implementation of
AbstractDimensionSelector.getObject().Abstract class for
InputSource.This class provides a skeletal implementation of the
IntSet
interface to minimize the effort required to implement this interface.Base class for most
DimFilter.This helper class helps us distinguish whether we need to continue processing or not.
A helper class to avoid boilerplate for creating
PrioritizedQueryRunnerCallable objects.contains serialized data that is compressed and delta-encoded (Long)
It's ready to be transferred to a
WritableByteChannelvalid call sequence
serializes a Long/Object(Number) pair in the context of a column/segment.
serializes a Long/Object pair as
Long:Byte:Object
Base class for sort operators.
Processor manager that wraps another
ProcessorManager and accumulates a result.This class represents an intent to acquire a reference to a
Segment and then use it to do stuff, and finally
closing when done.AggregateCombiner is used to fold rollup aggregation results from serveral "rows" of different indexes during index
merging (see
IndexMerger).Aggregate projection schema and row count information to store in
Metadata which itself is stored inside a
segment, defining which projections exist for the segment.API type to specify an aggregating projection on
IncrementalIndexSchemaThis
TopNAlgorithm is tailored to processing aggregates on high cardility columns which are likely to have
larger result sets.An Aggregator is an object that can aggregate metrics.
A class that helps query engines use Buffer- or VectorAggregators in a consistent way.
Encapsulates an
Aggregator and the initial size in bytes required by
the Aggregator.AggregatorFactory is a strategy (in the terms of Design Patterns) that represents column aggregation, e.g.
AllGranularty buckets everything into a single bucket
Like
AllFalseBitmapColumnIndex during normal operation, except if called with 'includeUnknowns' becomes like
AllTrueBitmapColumnIndex.A "having" spec that always evaluates to true
Logical AND filter operation
The logical "and" operator for the "having" clause.
Describes the in-memory indexing method for data ingestion.
A class that allows writing to a series of Memory blocks as if they are one big coherent chunk of memory.
A RowsAndColumns that supports appending columns.
Base interface describing the mechanism used to evaluate an
ApplyFunctionExpr, which 'applies' a
LambdaExpr to one or more array Expr.Evaluates to true if all element of the array input
Expr causes the LambdaExpr to evaluate to a
'truthy' valueEvaluates to true if any element of the array input
Expr causes the LambdaExpr to evaluate to a
'truthy' valueBase class for family of
ApplyFunction which aggregate a scalar or array value given one or more array
input Expr arguments and an array or scalar "accumulator" argument with an initial valueBase class for "map" functions, which are a class of
ApplyFunction which take a lambda function that is
mapped to the values of an ApplyFunction.IndexableMapLambdaObjectBinding which is created from the outer
Expr.ObjectBinding and the values of the array Expr argument(s)Accumulate a value for the cartesian product of 'n' array inputs arguments with an 'n + 1' argument
LambdaExpr.Map the cartesian product of 'n' array input arguments to an 'n' argument
LambdaExprFilter an array to all elements that evaluate to a 'truthy' value for a
LambdaExprAccumulate a value for a single array input with a 2 argument
LambdaExpr.Expr.ObjectBinding which can be iterated by an integer index position for ApplyFunction.BaseFoldFunction.Expr.ObjectBinding which can be iterated by an integer index position for ApplyFunction.BaseMapFunction.Helper that can wrap another
Expr.InputBindingInspector to use to supply the type information of a
LambdaExpr when evaluating Expr.getOutputType(org.apache.druid.math.expr.Expr.InputBindingInspector).Map the scalar values of a single array input
Expr to a single argument LambdaExprBase class for family of
ApplyFunction which evaluate elements elements of a single array input against
a LambdaExpr to evaluate to a final 'truthy' valueSimple, mutable,
Expr.ObjectBinding for a LambdaExpr which provides a Map for storing
arbitrary values to use as values for IdentifierExpr in the body of the lambda that are arguments to the
lambdaAllocator that uses a reusable
WritableMemory arena.Creates
ArenaMemoryAllocator on each call to ArenaMemoryAllocatorFactory.newAllocator().Check to see if an array contains a specific element.
Construct a
BitmapColumnIndex for any array element which might be present in an array contained in the
column.ArrayListRowsAndColumns is a RowsAndColumns implementation that believes it has all of its data on-heap.
A
Segment that is based on a stream of objects.Computes a particular quantile from a numeric array.
Serializable record of an audit event that can be persisted, logged or sent
over REST APIs.
Payload of an
AuditEntry that may be specified AuditEntry.Payload.raw() or AuditEntry.Payload.serialized().Contains information about the author who performed an audited operation.
Column merger for
AutoTypeColumnIndexer to eventually produce some form of
NestedCommonFormatColumn.Common
DimensionSchema for ingestion of 'standard' Druid built-in ColumnType datatypes.An abstract class for all query exceptions that should return a bad request status code (400).
Specialized holder for
BaseColumn (physical column).Double value selecting polymorphic "part" of the
ColumnValueSelector interface.Base class for many (although not all)
ColumnValueSelector<ExprEval>.DruidException.Failure for a specific errorCode, category and persona.Float value selecting polymorphic "part" of the
ColumnValueSelector interface.This is a helper class to facilitate sharing the code for sortedBucketIntervals among
the various GranularitySpec implementations.
Utilities for building a Guice injector.
Long value selecting polymorphic "part" of the
ColumnValueSelector interface.Null value checking polymorphic "part" of the
ColumnValueSelector interface for primitive values.Object value selecting polymorphic "part" of the
ColumnValueSelector interface.A
MonitorScheduler implementation based on ScheduledExecutorService.The implementations of this interface will not validate that things are sorted for the binary search, it assumes that
they must be.
Common interface for bitmap indexes for use by
Filter for cursor creation, to
allow fast row skipping during query processing.BitmapResultFactory is an abstraction that allows to record something along with preFilter bitmap construction, and
emit this information as dimension(s) of query metrics.
Doesn't extend
IntIterable to avoid accidental for-each iteration with boxing.BitSetBitmapFactory implements BitmapFactory as a wrapper for java.util.BitSet
Basic vector processor that processes 2 inputs and works for both primitive value vectors and object vectors.
Base
ExprVectorProcessor for expressions and functions with 2 'object' typed inputs (strings, arrays).In-memory channel backed by a limited-capacity
Deque.An
OutputChannelFactory that generates BlockingQueueFrameChannel.Serializer that produces
BlockLayoutColumnarDoublesSupplier.BlockLayoutColumnarDoubles.Serializer that produces
.
invalid reference
BlockLayoutColumnarFloatsSupplier.BlockLayoutColumnarFloatsSerializer that produces
BlockLayoutColumnarLongsSupplier.BlockLayoutColumnarLongs.Limiter for access to some resource.
Recommended to use
RangeFilter insteadThis is one of the special shardSpecs which are temporarily used during batch ingestion.
A BufferAggregator is an object that can aggregate metrics into a ByteBuffer.
A buffer grouper for array-based aggregation.
See
BuildingShardSpec for how this class is used.See
BuildingShardSpec for how this class is used.See
BuildingShardSpec for how this class is used.This is one of the special shardSpecs which are temporarily used during batch ingestion.
See
BuildingShardSpec for how this class is used.this is used as a vehicle to register the correct version of the system default nested column handler and multi
value handling mode by side effect with the help of binding to
Lifecycle so that
BuiltInTypesModule.initDimensionHandlerAndMvHandlingMode(DefaultColumnFormatConfig) can be called with the injected
DefaultColumnFormatConfig.Query runner that wraps a base single-segment query runner, and wraps its results in a
BySegmentResultValueClass object if the "bySegment" query context parameter is set.Streams bytes from the
ByteBuffer's position to the limit as InputStream.ByteBuffer-based implementation of the min-max heap developed by Atkinson, et al.
Comparator for byte arrays from
RowKey.key instances.Wrapper around an
InputEntity that counts the number of bytes read.HttpResponseHandler for stream data of byte array type.Tracks the byte usage with an upper bound bytes limit.
CacheKeyBuilder is a tool for easily generating cache keys of
Cacheable objects.Handles caching-related tasks for a particular query type.
All functions in this class were basically adopted from Apache Calcite and modified to use them in Druid.
Annotation for methods, which are called from hot loops during query processing.
Legacy fallback column format used by columns which do not directly implement
ColumnFormat.Processor for
CardinalityVectorAggregator.CartesianList computes the cartesian product of n lists.This class implements a function that checks if one string contains another string.
CellReader is intended to read the data written byte
CellWriter.usage:
Monitor that reports cpu usage stats by reading `cpu.stat` reported by cgroupv2
Monitor that reports disk usage stats by reading `io.stat` reported by cgroupv2
Monitor that reports memory usage stats by reading `memory.*` files reported by cgroupv2
A QueryRunner that combines a list of other QueryRunners and executes them in parallel on an executor.
Exception thrown by
ReadableByteChunksFrameChannel.addChunk(byte[]) when the channel has been closed
for writes, i.e., after ReadableByteChunksFrameChannel.doneWriting() or
ReadableByteChunksFrameChannel.close() has been called.Various helper methods useful for checking the validity of arguments to spec constructors.
An abstract implementation of the storage connectors that download the file from the remote storage in chunks
and presents the downloaded chunks as a single
InputStream for the consumers of the connector.POJO for storing the parameters required to support chunking of the downloads by
ChunkingStorageConnector.Builder for
ChunkingStorageConnectorParameters.A circular buffer that supports random bidirectional access.
A circular list that is backed by an ordered list of elements containing no duplicates.
An extension point to create a custom Druid service.
Client representation of
.
invalid reference
org.apache.druid.sql.http.SqlParameterClient representation of
.
invalid reference
org.apache.druid.sql.http.SqlQueryA
MonitorScheduler implementation based on CronScheduler.Enum used in the query context to determine if clone queries should be used by native queries.
Like
CloseableIterator, but has a currentMetadata() method, which returns "metadata", which is effectively a Mapinvalid input: '<'String, Object>
about the source of last value returned by next()
The returned metadata is read-only and cannot be modified.A CloseableShapeshifter is an interface created to allow Segments to be used from
CloseableShapeshifter.as(Class), but also to
be able to ensure that any resource used by the object returned from the CloseableShapeshifter.as(Class) method have proper
management of their lifecycle.Methods in this class could have belonged to
Closer, but not editing
that class to keep its source close to Guava source.Common type for 'bucket' and 'path' concept of cloud objects to allow code sharing between cloud specific
implementations.
Describes a key used for sorting or partitioning.
Boundaries of a partition marked by start and end keys.
Holder object for a set of
ClusterByPartition.A semantic interface used to partition a data set based on a given set of columns.
An interface representing a Column of data.
Allows for accessing a column, provides methods to enable row-by-row access of a specific column.
Resource that provides random access to a packed array of primitive doubles.
Serializer that produces
ColumnarDoubles.Resource that provides random access to a packed array of primitive floats.
Serializer that produces
ColumnarFloats.A
CursorFactory implementation based on a single columnar Frame.Resource that provides random access to a packed array of primitive int values.
Serializer that produces either
ColumnarInts or ColumnarMultiInts.Resource that provides random access to a packed array of primitive longs.
Serializer that produces
ColumnarLongs.Resource that provides random access to a packed array of short arrays of ints (
IndexedInts).Serializer that produces
ColumnarMultiInts.Wraps
QueryableIndex and VirtualColumns, providing a unified view of physical and virtual columns,
as well as lifecycle management for physical columns.This interface is used to expose information about columns that is interesting to know for all matters dealing with
reading from columns, including query planning and optimization, creating readers to merge segments at ingestion
time, and probably nearly anything else you can imagine.
This interface defines the shape of a mechanism to allow for bespoke coercion of
ColumnCapabilities.Capable.UNKNOWN into
ColumnCapabilities.Capable.TRUE or ColumnCapabilities.Capable.FALSE for each ColumnCapabilities.Capable of a ColumnCapabilities, as is
appropriate for the situation of the caller.Compares values between columns, first converting them all to strings.
Abstraction for the physical storage components of a column, allowing us to decouple a columns logical type from
how it is actually stored in a segment.
Implements a RowsAndColumns column based on a ColumnHolder.
Sort of like
ColumnCapabilities, except for indexes supplied by ColumnIndexSelector, provides
information for how query processing may use indexes.Provides indexes and information about them (
ColumnIndexCapabilities) for a column.Class that encapsulates knowledge about how to create "column processors", which are...
Creates "column processors", which are objects that wrap a single input column and provide some functionality on
top of it.
Factory class for MetricSelectors
A grouping of various related objects used during query processing for a single dimension, used for convenience.
Base type for strategy objects that handle value type operations pertaining to a specific query type
Used by some
InputSourceReader implementations in order to know what columns will need to be read out
of the InputRow objects they create.Native Druid types.
Base type for interfaces that manage column value selection, e.g.
A semantic interface for use with
Column objects.InputSource that combines data from multiple inputSources.
A custom Jackson serializer that converts a list of objects into a comma-separated string.
Committer includes a Runnable and a Jackson-serialized metadata object containing the offset
Encapsulates the Engine to be used for a compaction task.
This class describes what compaction task spec was used to create a given segment.
Spec containing transform configs for Compaction Task.
Comparator based DimensionDictionary
there are a lot of unused methods in here for now since the only thing this is used for is to build up
the unsorted dictionary and then it is converted to a ComparatorSortedDimensionDictionary, but
leaving the unused methods in place for now to be basically compatible with the other implementation.Comparator based SortedDimensionDictionary
There are a number of unused methods, because nested columns don't merge bitmap indexes during
the merge phase, rather they are created when serializing the column, but leaving for now for
compatibility with the other implementationContains the
DataSegment and Segment.This interface represents a complex column and can be implemented by druid extension writer of a custom column
with arbitrary serialization instead of a custom column that serializes rows of objects serialized using
GenericIndexed class which is default implementation of "writeToXXX" methods in
ComplexColumnSerializer.Reads values written by
ComplexFieldWriter.Wraps a
BaseObjectColumnValueSelector and uses ComplexMetricSerde.toBytes(java.lang.Object) to write complex objects.Column writer for complex columns.
Column writer for complex columns.
ComplexMetrics houses a mapping of serde names to affiliated ComplexMetricSerde objects.
A channel factory which provides ordered composed channels.
A composed readable channel to read frames.
A composed writable channel to write frames.
Reader for a virtual contiguous address range backed by compressed blocks of data.
Streams array of integers out in the binary format described by
CompressedColumnarIntsSupplierImplementation of
NestedDataComplexColumn which uses a CompressedVariableSizedBlobColumn for the
'raw' StructuredData values and provides selectors for nested field columns specified by ordered lists of
NestedPathPart.Streams array of integers out in the binary format described by
CompressedVSizeColumnarIntsSupplierFormat -
byte 1 - version
offsets -
ColumnarInts of length num of rows + 1 representing offsets of starting index of first element of
each row in values index and last element equal to length of values column, the last element in the offsets
represents the total length of values column.Compression of metrics is done by using a combination of
CompressionStrategy
and Encoding(such as CompressionFactory.LongEncodingStrategy for type Long).The compression of decompression of encodings are separated into different enums.
This writer output encoded values to the given ByteBuffer or OutputStream.
Compression strategy is used to compress block of bytes without knowledge of what data the bytes represents.
A RowsAndColumns implementation that effectively concatenates multiple RowsAndColumns objects together.
As the name suggests, this class instantiates bitmaps of the types
WrappedConciseBitmap and WrappedImmutableConciseBitmap.
This is CONCISE: COmpressed 'N' Composable Integer SEt.
Manager that limits the number of processors that may run concurrently.
This synchronization object allows to
ConcurrentAwaitableCounter.increment() a counter without blocking, potentially from multiple
threads (although in some use cases there is just one incrementer thread), and block in other thread(s), awaiting
when the count reaches the provided value: see ConcurrentAwaitableCounter.awaitCount(long), or the specified number of events since the
call: see ConcurrentAwaitableCounter.awaitNextIncrements(long).Grouper based around a set of underlying
SpillingGrouper instances.The implementation of
ResponseContext with a ConcurrentHashMap as a delegateA class to accumulate simple stats of some time points.
Provides the ability to conditionally bind an item to a set.
Utility class for common config operations.
This class implements a function that checks if one string contains another string.
Collect CPU share and quota information from cpu cgroup files.
Collect CPU and memory data from cpuset cgroup files.
Utility class for symmetric key encryption (i.e.
Cursor is an interface for iteration over a range of data points, used during query execution.
Defines the plan for how the reader will scan, filter, transform, group and aggregate, and or order the data from a
CursorFactory so that an appropriate CursorHolder can be constructed.Provides
RowsAndColumns on top of a CursorFactory.Row pointer based on a
ReadableOffset containing a row number.Class that helps non-vectorized query engines handle "granularity" parameters.
Provides
Cursor and if available, VectorCursor which readers can use to scan a set of rows defined
originally from a CursorBuildSpec which describes how data is to be scanned, transformed, filter, grouped and
aggregated, and/or ordered.Metadata of Druid's data segment.
This class is needed for optional injection of pruneLoadSpec and pruneLastCompactionState, see
github.com/google/guice/wiki/FrequentlyAskedQuestions#how-can-i-inject-optional-parameters-into-a-constructor
DataSegmentArchiver knows how to archive segments.
DataSegmentKiller knows how to kill segments from the Druid system.
DataSegmentMover knows how to move the segment location from one to another.
This immutable class encapsulates segments metadata and corresponding schema.
Immutable wrapper class for segment and schema.
Represents a source...
Simple wrapper class to enforce UTC Chronology in formatter.
Pool that pre-generates objects up to a limit, then permits possibly-blocking "take" operations.
JsonInclude filter for DefaultLimitSpec.getLimit().A user configuration holder for all query types.
DefaultQueryMetrics is unsafe for use from multiple threads.
The implementation of
ResponseContext with a HashMap as a delegateThis class is implemented with delegation to another QueryMetrics for compatibility, see "Making subinterfaces of
QueryMetrics for emitting custom dimensions and/or metrics for specific query types" section in
QueryMetrics
javadoc.The DefaultSortedMatrixMaker is a SortedMatrixMaker that works on the generic RowsAndColumns interface.
JsonInclude filter for boolean values that default to true.Controls deferral of
ExpressionVirtualColumn in GroupByQuery.This utility class provides a thin runtime abstraction to pick between
- sun.misc.Unsafe.defineClass in Java 8,
- and MethodHandles.Lookup.defineClass in Java 9 and above,
while still providing compile-time support for both Java 8 and Java 9+.
An Aggregator that delegates everything.
A BufferAggregator that delegates everything.
Utility function for
DelimitedInputFormat.InputFormat for customized Delimiter Separate Value format of input data (default is TSV).
DelimitedValueReader is the reader for Delimitor Separate Value format input data(CSV/TSV).
Strategy for grouping dimensions which can have variable-width objects, and aren't backed by prebuilt dictionaries.
A
GroupByVectorColumnSelector that builds an internal Stringinvalid input: '<'->Integer dictionary, used for grouping
single-valued STRING columns which are not natively dictionary encoded, e.g.Utilities for parts of the groupBy engine that need to build dictionaries.
Basic structure for indexing dictionary encoded columns
Base structure for merging dictionary encoded columns
Specifies any additional per value indexes which should be constructed when
DictionaryEncodedColumnMerger.writeIndexes(List) is called, on top of the standard bitmap index created
with DictionaryEncodedColumnMerger.mergeBitmaps(java.util.List<java.nio.IntBuffer>, org.apache.druid.collections.bitmap.BitmapFactory, org.apache.druid.segment.DictionaryEncodedColumnMerger.IndexSeeker[], int)Get old dictId from new dictId, and only support access in order
Peristent dictionary id conversion mappings, artifacts created during segment merge which map old dictionary ids
to new dictionary ids.
Closer of
DictionaryEncodedColumnMerger.PersistedIdConversion and a parent path which they are stored in for easy cleanup when the
segment is closed.This exposes a 'raw' view into bitmap value indexes of a string
DictionaryEncodedColumn.This exposes a 'raw' view into bitmap value indexes for
DictionaryEncodedColumn.Value to dictionary id lookup, backed with memory mapped dictionaries populated lazily by the supplied
DictionaryWriter.Iterator for merging dictionaries for some comparable type into a single sorted dictionary, useful when merging
dictionary encoded columns
Common
BitmapColumnIndex implementation for indexes which need to scan a range of values.Common
BitmapColumnIndex implementation for indexes which are computed by scanning the entire value
dictionary of the underlying column to check if the value index matches the filter.PartitionsSpec based on dimension values.
Utility class to compare dimensions
Creates a list comparator with custom comparator for the elements.
Array comparator that converts the elements to their string representation, before comparing the values using
the provided
StringComparator.Buildable dictionary for some comparable type.
Interface containing dictionary-related methods common to
DimensionSelector,
SingleValueDimensionVectorSelector, and
MultiValueDimensionVectorSelector.Processing related interface
A DimensionHandler is an object that encapsulates indexing, column merging/building, and querying operations
for a given dimension type (e.g., dict-encoded String, Long).
Dimension to integer id encoder - decoder i.e.
Processing related interface
A DimensionIndexer is a per-dimension stateful object that encapsulates type-specific operations and data structures
used during the in-memory ingestion process (i.e., work done by
IncrementalIndex).Processing related interface
A DimensionMerger is a per-dimension stateful object that encapsulates type-specific operations and data structures
used during the segment merging process (i.e., work done by
IndexMerger).Processing related interface
DimensionMerger subclass to be used with IndexMergerV9.
See
BucketNumberedShardSpec for how this class is used.Spec to create partitions based on value ranges of multiple dimensions.
ShardSpec for partitioning based on ranges of one or more dimensions.Defines the schema of a single dimension in a dataset.
Selector for a string-typed column, either single- or multi-valued.
This class not a public API.
Provides information about a dimension for a grouping query, like topN or groupBy.
Wrapper for
implementations
StringBuilder to re-use common patterns in custom
invalid reference
DimFilter#toString()Creates an executor service that runs each task in the thread that invokes
execute/submit, as in ThreadPoolExecutor.CallerRunsPolicy This applies both to individually submitted
tasks and to collections of tasks submitted via invokeAll or invokeAny.QueryProcessingPool wrapper over Execs.directExecutor()Specialization of
AggregateCombiner for primitive double aggregations.Vectorized implementation of the
DoubleAnyBufferAggregatorReader for columns written by
DoubleArrayFrameColumnWriterColumnar frame writer for
ColumnType.DOUBLE_ARRAY columnsspecialized
DoubleBivariateFunctionVectorProcessor for processing (double[], long[]) -> double[]specialized
DoubleBivariateFunctionVectorProcessor for processing (double[], double[]) -> double[]common machinery for processing two input operators and functions, which should always treat null inputs as null
output, and are backed by a primitive values instead of an object values (and need to use the null vectors instead of
checking the vector themselves for nulls)
this one is specialized for producing double[], see
LongBivariateFunctionVectorProcessor for
long[] primitives.specialized
DoubleBivariateFunctionVectorProcessor for processing (long[], double[]) -> double[]specialized
DoubleBivariateFunctionVectorProcessor for processing (long[], long[]) -> double[]This interface is convenient for implementation of "double-sourcing"
ColumnValueSelectors, it provides
default implementations for all ColumnValueSelector's methods except BaseDoubleColumnValueSelector.getDouble().Column Serializer for double column.
Reads the values produced by
DoubleFieldWriterWraps a
BaseDoubleColumnValueSelector and writes field values.Vectorized version of on heap aggregator for column selectors with type DOUBLE.
specialized
DoubleUnivariateFunctionVectorProcessor for processing (double[]) -> double[]common machinery for processing single input operators and functions, which should always treat null input as null
output, and are backed by a primitive value instead of an object value (and need to use the null vector instead of
checking the vector itself for nulls)
this one is specialized for producing double[], see
LongUnivariateFunctionVectorProcessor for
long[] primitives.specialized
DoubleUnivariateFunctionVectorProcessor for processing (long[]) -> double[]Note: this is not a
PublicApi or an
ExtensionPoint of Druid.Represents an error condition exposed to the user and/or operator of Druid.
Category of error.
The persona that the message on a DruidException is targeting
Note: this is not a
PublicApi or an
ExtensionPoint of Druid.Note: this is not a
PublicApi or an
ExtensionPoint of Druid.Contains dimension names used while emitting metrics.
A Guice module which also provides Jackson modules.
Uses a
DruidPredicateFactory to construct a BitmapColumnIndex containing rows for all values which
satisfy the predicate.Three-value logic result for matching values with predicates produced by
DruidPredicateFactoryA Builder for DataSourceMetadataQuery.
A Builder for ScanQuery.
A Builder for SearchQuery.
A Builder for SegmentMetadataQuery.
A Builder for TimeBoundaryQuery.
A Builder for TimeseriesQuery.
BlockingPool with 0 maxSize, all take*() methods immediately throw
UnsupportedOperationException.NonBlockingPool which is not able to allocate objects,
DummyNonBlockingPool.take() throws UnsupportedOperationException.Helper class that fetches the directory and file names corresponding to file location
DurationGranularity buckets data based on the length of a duration
This is used to get [secure] configuration in various places in an extensible way.
Dynamically determine partitions in the middle of indexing.
Encapsulates either an "error" or a "value".
Logger which also has an ServiceEmitter.Represents the encoded component of a row key corresponding to a single dimension.
Serializer that produces
.
invalid reference
EntireLayoutColumnarDoublesSupplier.EntireLayoutColumnarDoublesSerializer that produces
.
invalid reference
EntireLayoutColumnarFloatsSupplier.EntireLayoutColumnarFloatsSerializer that produces
.
invalid reference
EntireLayoutColumnarLongsSupplier.EntireLayoutColumnarLongsA failure type used to make
DruidExceptions of category
DruidException.Category.INVALID_INPUT for persona DruidException.Persona.USER,
denoting that a certain entry already exists.EnumeratedDistrubtion's sample() method does a linear scan through the array of probabilities.
Represents a join condition between a left-hand-side expression (leftExpr) and a right-hand-side direct
column access (rightColumn).
The "=" operator in a "having" clause.
A Response Object that represents an error to be returned over the wire.
EventMap is a hash map implementation.
This annotation can be applied to a package, class or method to indicate that all
class fields and method parameters and return values in that element are nonnull
by default unless overridden.
Represents the native engine's execution vertex - the execution unit it may execute in one execution cycle.
ExplainAttributes holds the attributes of a SQL statement that is used in the EXPLAIN PLAN result.
Class that encapsulates the information of a single plan for an
EXPLAIN PLAN FOR query.Wraps an underlying sequence and allows us to force it to explode at various points.
Base interface of Druid expression language abstract syntax tree nodes.
Information about the context in which
IdentifierExpr are used in a greater Expr, listing
the 'free variables' (total set of required input columns or values) and distinguishing between which identifiers
are used as scalar inputs and which are used as array inputs.Mechanism to supply input types for the bindings which will back
IdentifierExpr, to use in the aid of
inferring the output type of an expression with Expr.getOutputType(org.apache.druid.math.expr.Expr.InputBindingInspector).Mechanism to supply values to back
IdentifierExpr during expression evaluationMechanism to rewrite an
Expr, implementing a Expr.Shuttle allows visiting all children of an
Expr, and replacing them as desired.Mechanism to supply batches of input values to a
ExprVectorProcessor for optimized processing.Expr.InputBindingInspector + vectorizations stuff for Expr.asVectorProcessor(org.apache.druid.math.expr.Expr.VectorInputBindingInspector)This class provides an empty implementation of
ExprListener,
which can be extended to create a listener which only needs to handle a subset
of the available methods.Basic expression
ColumnValueSelector.Implementation of
GroupByVectorColumnSelector that uses a wide key representing all expression inputs
to enable deferring expression evaluation to ExpressionDeferredGroupByVectorColumnSelector.writeKeyToResultRow(MemoryPointer, int, ResultRow, int).DruidPredicateFactory whose predicates evaluate an Expr on the input match value, turning the result
of that evaluation into a DruidPredicateMatch.UNKNOWN if the value is null, and if not,
using ExprEval.asBoolean() to produce DruidPredicateMatch.TRUE or DruidPredicateMatch.FALSE.Special
Expr.ObjectBinding for use with ExpressionLambdaAggregatorFactory.Module that binds
ExprMacroTable to GuiceExprMacroTable and configures a starter set of
ExprMacroTable.ExprMacro for all macros defined in the "druid-processing" module.Basic multi-value dimension selector for an
Expr evaluating
ColumnValueSelector.Uses the underlying dictionary values of a column to provide
DruidPredicateIndexes that apply an Expr
to the values of an input column before matching with a DruidPredicateFactory.Expressions processing configs
The type system used to process Druid expressions.
ExprEvalVector backed directly by an underlying Expr.VectorInputBindingResult of
ExprVectorProcessor.evalVector(org.apache.druid.math.expr.Expr.VectorInputBinding) which wraps the actual evaluated results of the operation over the
input vector(s).This interface defines a complete listener for a parse tree produced by
ExprParser.Mechanism by which Druid expressions can define new functions for the Druid expression language.
Base class for
Expr from ExprMacroTable.ExprMacro.Base class for
Expr from ExprMacroTable.ExprMacro that accepts all-scalar arguments.stub interface to allow
Parser.flatten(Expr) a way to recognize macro functions that exend thisBase 'value' types of Druid expression language, all
Expr must evaluate to one of these types.Interface describing vectorized expression processors, which can be specialized using input type information to
produce optimized expression evaluators, which can operate on batches of primitive data with minimal object overhead
The ClassLoader that gets used when druid.extensions.useExtensionClassloaderFirst = true.
Signifies that the annotated type is an extension point.
Manages the loading of Druid extensions.
Module for the extensions loader.
Transfers the now-populated extension loader instance from the
startup to the main injector.
Deprecated.
An ExtractionFn is a function that can be used to transform the values of a column (typically a dimension).
IncrementalIndexRow storage interface, a mutable data structure for building up a set or rows to eventually
persist into an immutable segmentPredicate that wraps another predicate and can catch ClassCastException from some
Comparator and degrades into casting input to an expected ExpressionType once an exception
is encountered.Implementation of
ExprVectorProcessor that adapts non-vectorized Expr.eval(Expr.ObjectBinding).A virtual column that picks one column or another based on whether they exist.
Custom json-path JsonProvider override to circumvent slow performance when encountering null paths as described in
https://github.com/json-path/JsonPath/issues/396
Note that this only avoids errors for map properties, avoiding the exception on array paths is not possible without
patching json-path itself
Like the Apache Commons LineIterator, but faster.
Selectable column that does not support anything.
Helps compute the field position for a frame from the different regions in the frame.
Embeds the logic to read a specific field from row-based frames or from
RowKey.Helper used to read field values from row-based frames or
RowKey.Binary serialization for nested field type info, translated into this compact format for storage in segments.
Helper used to write field values to row-based frames or
RowKey.Helper used to write field values to row-based frames or
RowKey.Assigns each input file to its own split.
A class that concatenates files together into configurable sized chunks,
works in conjunction with the SmooshedFileMapper to provide access to the
individual files.
Keeps results of a file copy, including children and total size of the resultant files.
Like
ByteSource, but this is an interface, which allows use of lambdas.FilterBundle is a container for all the goodies used for producing filtered cursors, a
ImmutableBitmap if
the filter can use an index, and/or a FilterBundle.MatcherBundle which contains functions to build ValueMatcher
and VectorValueMatcher for any filters which must be evaluated row by row during the cursor scan.Wraps info needed to build a
FilterBundle, and provides an estimated compute cost for
BitmapColumnIndex.computeBitmapResult(org.apache.druid.query.BitmapResultFactory<T>, boolean).Builder of
ValueMatcher and VectorValueMatcher.This class models a data source to be unnested which is present along with a filter.
This class provides a mechanism to influence whether or not indexes are used for a
Filter during processing
by Filter.makeFilterBundle(FilterBundle.Builder, BitmapResultFactory, int, int, boolean)
(i.e.Query runner that applies
QueryToolChest.makePostComputeManipulatorFn(Query, MetricManipulationFn) to the
result stream.Base type for vectorized version of on heap 'last' aggregator for primitive numeric column selectors..
Fixed width value implementation of
Indexed, stored simply as a header that contains the number of values,
and then the values stored sequentially.Specialized version of
FixedIndexedWriter for writing ints, with no support for null values, and no
verification that data is actually sorted, it just trusts you and takes your word for itWriter for a
FixedIndexedStrategy for grouping dimensions which have fixed-width objects.
Vectorized implementation of the
FloatAnyBufferAggregatorReader for columns written by
FloatArrayFrameColumnWriterColumnar frame writer for
ColumnType.FLOAT_ARRAY columnsThis interface is convenient for implementation of "float-sourcing"
ColumnValueSelectors, it provides default
implementations for all ColumnValueSelector's methods except BaseFloatColumnValueSelector.getFloat().Column Serializer for float column.
Reads values written by
FloatFieldWriter.Wraps a
BaseFloatColumnValueSelector and writes field values.Vectorized version of on heap 'last' aggregator for column selectors with type FLOAT.
Default implementation of
QueryProcessingPool that just forwards operations, including query execution tasks,
to an underlying ExecutorService
Exposes a method ForwardingQueryProcessingPool.submitRunnerTask(PrioritizedQueryRunnerCallable, long, TimeUnit) which allows execution tasks to be serviced a custom timeout.Implementation of
TransformableRowIterator that just delegates all methods to some baseIterator (adapter
pattern).A data frame.
Represents an inline datasource where the rows are embedded within the DataSource object itself.
Serializes
FrameBasedInlineDataSource to the representation of InlineDataSource
so that the servers' on wire transfer data doesn't change.Creates a joinable from the
FrameBasedInlineDataSource.Processor that reads up to "maxFrames" frames from some input channels and combines them into a batch.
Processor that hash-partitions rows from any number of input channels, and writes partitioned frames to output
channels.
Processor that merges already-sorted inputChannels and writes a fully-sorted stream to a single outputChannel.
Processor that merges frames from inputChannels into a single outputChannel.
Embeds the logic to read a specific column from frames with a specific
RowSignature.Creates
FrameColumnReader corresponding to a given column type and number.Represent writers for the columnar frames.
Wraps a
Frame and provides ways to compare rows of that frame to various other things.Implementation of
FrameComparisonWidget for pairs of FrameType.isRowBased() frames.A semantic interface used to aggregate a list of AggregatorFactories across a given set of data
A file containing
Frame data.Encapsulation for ƒrame file footer related operations.
An
HttpResponseHandler that streams data into a ReadableByteChunksFrameChannel.Response object for
FrameFileHttpResponseHandler.Writer for
FrameFile.Copy of
FilteredOffset that does not require bitmap indexes.A FrameProcessor is like an incremental version of Runnable that operates on
ReadableFrameChannel and
WritableFrameChannel.Passed to
SuperSorter to decorate the processors it launches.Manages execution of
FrameProcessor in an ExecutorService.A
QueryableIndex implementation based on a single columnar Frame.Embeds the logic to read frames with a given
RowSignature.Utility methods used by various entities that read data from
Frame objects.Exception that is conventionally thrown by workers when they call
FrameWriter.addSelection() and it returns false on an empty frame, or in
a situation where allocating a new frame is impractical.Encapsulates a
Frame and the RowSignature of the rows that are encapsulated in itUtility for sorting frames in-place.
A
VectorCursor that is based on a Frame.Frame plus partition number.
Writer for
Frame.Interface for creating
FrameWriter.Outward-facing utility methods for
FrameWriterFactory and FrameWriter users.Utility methods used by
FrameWriter implementations.Indexed specialized for storing variable-width binary values (such as utf8 encoded strings), which must be
sorted and unique, using 'front coding'.Indexed specialized for storing int arrays, which must be sorted and unique, using 'front coding'.This class is to hold data while receiving stream data via HTTP.
Base interface describing the mechanism used to evaluate a
FunctionExpr.Scaffolding for a 2 argument
Function which accepts one array and one scalar input and adds the scalar
input to the array in some way.Function that takes 1 array operand and 1 scalar operandFunction that takes 2 array operandsBase scaffolding for functions which accept 2 array arguments and combine them in some way
Base class for a 2 variable input
Function implementationBase class for a 2 variable input mathematical
Function, with specialized 'eval' implementations that
operate on primitive number types"Searched CASE" function, similar to
CASE WHEN boolean_expr THEN result [ELSE else_result] END in SQL."Simple CASE" function, similar to
CASE expr WHEN value THEN result [ELSE else_result] END in SQL.Many math functions always output a
Double primitive, regardless of input type.Many math functions always output a
Double primitive, regardless of input type.SQL function "x IS DISTINCT FROM y".
SQL function "IS FALSE".
SQL function "x IS NOT DISTINCT FROM y".
SQL function "IS NOT FALSE".
SQL function "IS NOT TRUE".
SQL function "IS TRUE".
Primarily internal helper function used to coerce null, [], and [null] into [null], similar to the logic done
by
ExpressionSelectors.supplierFromDimensionSelector(org.apache.druid.segment.DimensionSelector, boolean, boolean) when the 3rd
argument is true, which is done when implicitly mapping scalar functions over mvd values.nvl is like coalesce, but accepts exactly two arguments.
Base class for a 2 variable input
Function whose first argument is a ExprType.STRING and second
argument is ExprType.LONG.Base class for a single variable input
Function implementationBase class for a single variable input mathematical
Function, with specialized 'eval' implementations that
that operate on primitive number typesFunctional extraction uses a function to find the new value.
Wrapper around
ExprVectorProcessor that reports errors in the same style as FunctionExpr.Box for tracking pending futures.
SEQUENTIAL: Generate integer or enumerated values in sequence.
GenericColumnSerializer can be implemented in custom aggregator extensions that would like to take full control of column
serialization.
A generic, flat storage mechanism.
Implementation of
ComplexColumn to be used when complex column serialization is done by using
GenericIndexed by using default implementations of "writeToXXX" methods in
ComplexColumnSerializerStreams arrays of objects out in the binary format described by
GenericIndexed
The version EncodedStringDictionaryWriter.VERSION is reserved and must never be specified as the
GenericIndexed version byte, else it will interfere with string column deserialization.This factory is used for DI of custom
QueryMetrics implementations for all query types, which don't (yet)
need to emit custom dimensions and/or metrics, i.Base class for writer of global dictionary encoded nested field columns for
NestedDataColumnSerializer.TableDataSource variant for globally available 'broadcast' segments.This glueing partitioning operator is supposed to continuously receive data, and output batches of partitioned RACs.
This class was created b/c sometimes static initializers of a class that use a subclass can deadlock.
Tells the indexer how to group events based on timestamp.
Only to create a mapping of the granularity and all the supported file patterns
namely: default, lowerDefault and hive.
The ">" operator in a "having" clause.
Contains a collection of query processing methods for type-specific operations used exclusively by
GroupByQueryEngineV2.
Creates
ColumnSelectorStrategys for grouping dimensions
If the type is STRING, then it delegates the group by handling to KeyMappingMultiValueGroupByColumnSelectorStrategy
which is specialized for DimensionSelectors and multi-value dimensions.Class that knows how to merge a collection of groupBy
QueryRunner objects, called queryables,
using a buffer provided by mergeBufferPool and a parallel executor provided by exec.Contains logic to process a groupBy query on a single
CursorFactory in a
non-vectorized manner.Specialization of
QueryMetrics for GroupByQuery.Implementations could be injected using
PolyBind
.optionBinder(binder, Key.get(GroupByQueryMetricsFactory.class))
.addBinding("myCustomGroupByQueryMetricsFactory")
.to(MyCustomGroupByQueryMetricsFactory.class);
And then setting property:
druid.query.groupBy.queryMetricsFactory=myCustomGroupByQueryMetricsFactory
Toolchest for GroupBy queries
This class contains resources required for a groupBy query execution.
Reserves the
GroupByQueryResources for a given group by query and maps them to the query's resource ID.Class that knows how to merge aggregator data from two groupBy
ResultRow objects that have the same time
and dimensions.Utility class that knows how to do higher-level groupBys: i.e.
Metrics collector for groupBy queries like spilled bytes, merge buffer acquistion time, dictionary size.
Column processor for groupBy dimensions.
Groupers aggregate metrics from rows that they typically get from a ColumnSelectorFactory, under
grouping keys that some outside driver is passing in.
Possibly-stateful object responsible for serde and comparison of keys.
Set of utility methods to faciliate implementation of
Grouper.KeySerde.bufferComparator() and
Grouper.KeySerde.bufferComparatorWithAggregators(AggregatorFactory[], int[])This class implements
grouping function to determine the grouping that a row is part of.Common code for processing
GroupByQuery.A semantic interface used to partition a data set based on a given set of dimensions.
Creates the startup injector.
Decorated
CloseableIterator<InputRow> that can process rows with HandlingInputRowIterator.InputRowHandlers.See
BucketNumberedShardSpec for how this class is used.Represents a deep, left-heavy join of a left-hand side baseSegment onto a series of right-hand side clauses.
This class is used for hash partitioning during ingestion.
An enum of supported hash partition functions.
An implementation of
VectorGrouper backed by a growable MemoryOpenHashTable.A "having" clause that filters aggregated/dimension value.
Heap based topn algorithm that handles aggregates on dimension extractions and numeric typed dimension columns.
Allocator that uses
ByteBuffer.allocate(int) to create chunks in the JVM heap.Specialization for
DimensionSelector queryable via offsets from HistoricalCursor.All functions in this class were basically adopted from Apache Hive and modified to use them in Druid.
Marker interface for abstractions, which are called from hot loops during query processing.
Interface for Async HTTP client libraries.
A handler for an HTTP request.
This annotation is an extension of java validation framework to ensure the validity of value of
HumanReadableBytes.This serializer provides the basic support of JSON serialization of
HumanReadableBytes
to demonstrate serialization and deserialization work
GivenImplements the HyperLogLog cardinality estimator described in:
http://algo.inria.fr/flajolet/Publications/FlFuGaMe07.pdf
Run this code to see a simple indication of expected errors based on different m values:
for (int i = 1; i < 20; ++i) {
System.out.printf("i[%,d], val[%,d] => error[%f%%]%n", i, 2 << i, 104 / Math.sqrt(2 << i));
}
This class is *not* multi-threaded.Contains data hashing methods used before applying Hyper Log Log.
"Mixin" for
DimensionSelector.Map some set of dictionary id to a smaller set of dictionaryIds (or...
This class is meant to represent a simple wrapper around an immutable bitmap
class.
Byte layout:
Header
0 to 1 : the MSB is a boolean flag for isLeaf, the next 15 bits represent the number of children of a node
Body
2 to 2 + numDims * Float.BYTES : minCoordinates
2 + numDims * Float.BYTES to 2 + 2 * numDims * Float.BYTES : maxCoordinates
concise set
rest (children) : Every 4 bytes is storing an offset representing the position of a child.
Similar to
MapLookupExtractor, but immutable, and also reversible without iterating the entire map.An immutable representation of an
RTree for spatial indexing.In-memory, row-based data structure used to hold data during ingestion.
IncrementalIndexRowHolder is a simple
IncrementalIndexRowHolder.get()/IncrementalIndexRowHolder.set(org.apache.druid.segment.incremental.IncrementalIndexRow) holder of IncrementalIndexRow.Interface that abstracts selecting data from a
FactsHolderAn adapter to an index
Indexed is a fixed-size, immutable, indexed set of values which allows
locating a specific index via an exact match, the semantics of which are defined
by the implementation.
Get a int an index (array or list lookup abstraction without boxing).
An interface to a table where some columns (the 'key columns') have indexes that enable fast lookups.
Indexes support fast lookups on key columns.
Readers support reading values out of any column.
IndexSpec defines segment storage format options to be used at indexing time,
such as bitmap type, and column compression formats.
Approximately like the SQL 'IN' filter, with the main difference being that this will match NULL values if contained
in the values list instead of ignoring them.
Represents an inline datasource, where the rows are embedded within the DataSource object itself.
A
JoinableFactory for InlineDataSource.Expr.ObjectBinding backed by a cache populated by ExprEval.bestEffortOf(Object) for when the input
type information is totally unknown, for a single row worth of values.InputEntity abstracts an input entity and knows how to read bytes from the given entity.
CleanableFile is the result type of
InputEntity.fetch(java.io.File, byte[]).InputSourceReader iterating multiple
InputEntitys.InputEntityReader knows how to parse data into
InputRow.A class storing some attributes of an input file.
InputFormat abstracts the file format of input data.
An InputRow is the interface definition of an event being input into the data ingestion layer.
Deprecated.
Schema of
InputRow.InputSource abstracts the storage system where input data is stored.
An interface to generate a
SplittableInputSource objects on the fly.Input unit for distributed batch ingestion.
Tracks bytes read from an input source.
This is a clone of
InputStreamResponseHandler except that it retains HTTP status/response object in the
response holder result.Serializer that chooses optimal
ColumnarLongs format dymamically, based on the values being written.InputEntityReader that parses bytes into some intermediate rows first, and then into InputRows.Produce a stream of intervals generated by a given set of intervals as input and a given
granularity.
Grouper specialized for the primitive int typeAn
Iterator-like interface that allows to "skip" some elements of
the setA failure type used to make
DruidExceptions of category
DruidException.Category.INVALID_INPUT for persona DruidException.Persona.USER.Exception thrown by
FrameWriterUtils.copyByteBufferToMemory(java.nio.ByteBuffer, org.apache.datasketches.memory.WritableMemory, long, int, boolean, boolean) if configured to check for null bytes
and a null byte is encountered.An Iterator-like interface that is intentionally not extending Iterator.
Implements an expression that checks if an IPv4 address belongs to a particular subnet.
Implements an expression that parses a string or long into an IPv4 address stored (as an unsigned
int) in a long.
Implements an expression that converts a long or a string into an IPv4 address dotted-decimal string.
Implements an expression that checks if an IPv6 address belongs to a subnet.
Abstract SQL three-value logic wrapper for some child
DimFilter to implement '{filter} IS TRUE' and
'{filter} IS FALSE'.SQL three-value logic wrapper for some child
Filter to implement '{filter} IS TRUE' and
'{filter} IS FALSE'.Helper methods to create cursor from iterable of rows
Should be used by extension filters, aggregators, etc, that use JavaScript to determine if JavaScript is enabled
or not.
Represents something that can be the right-hand side of a join.
Represents everything about a join clause except for the left-hand datasource.
Utility for creating
Joinable objects.A wrapper class over
JoinableFactory for working with Joinable related classes.Represents analysis of a join condition.
Represents a join of two datasources.
Analysis of a datasource for purposes of deciding how to execute a particular query.
Holds information about:
- whether a filter can be pushed down
- if it needs to be retained after the join,
- a reference to the original filter
- a potentially rewritten filter to be pushed down to the base table
- a list of virtual columns that need to be created on the base table to support the pushed down filter
When there is a filter in a join query, we can sometimes improve performance by applying parts of the filter
when we first read from the base table instead of after the join.
Represents an analysis of what base table columns, if any, can be correlated with a column that will
be filtered on.
A wrapper class for correlation analyses of different filters involved in the query.
A JoinFilterPreAnalysis contains filter push down/rewrite information that does not have per-segment dependencies.
A Builder class to build
JoinFilterPreAnalysisAll the information that is required to generate a
JoinFilterPreAnalysis.A config class that holds properties that control how join filter rewrites behave.
Holds the result of splitting a filter into:
- a portion that can be pushed down to the base table
- a portion that will be applied post-join
- additional virtual columns that need to be created on the base table to support the pushed down filters.
An interface returned by
Joinable.makeJoinMatcher(org.apache.druid.segment.ColumnSelectorFactory, org.apache.druid.segment.join.JoinConditionAnalysis, boolean, org.apache.druid.java.util.common.io.Closer) and used by HashJoinEngine to implement a join.Utility class for working with prefixes in join operations
Provides a singleton value of type
<T> from Properties bound in guice.An iterator over an array of JSON objects.
JsonLineReader reads input text line by line and tries to convert each text line to an JSON object.Deprecated.
In contrast to
JsonLineReader which processes input text line by line independently,
this class tries to split the input into a list of JsonNode objects, and then parses each JsonNode independently
into an InputRow.The ObjectMapper of this annotation will skip serialization of any field with null value.
JSON parser class that uses the JsonPath library to access fields via path expressions.
In contrast to
JsonLineReader which processes input text line by line independently,
this class tries to parse the input text as a whole to an array of objects.Deprecated.
For systems that for whatever reason cannot use Sigar (through org.apache.druid.java.util.metrics.SigarPidDiscoverer ),
this attempts to get the PID from the JVM "name".
Represents a component of a hash or sorting key.
Like
KeyMappingGroupByColumnSelectorStrategy, but for multi-value dimensions, i.e.Ordering associated with a
KeyColumn.Lazily decides to use a tmpBuffer to act as WriteOutBytes, till more than certain threshold is reached.
A replacement for Guice's Singleton scope.
Leaf
DataSource-s have no inputs.The "<" operator in a "having" clause.
An optimized column value
BitmapColumnIndex provider for columns which are stored in 'lexicographical' order,
allowing short-circuit processing of string value ranges.A manager of object Lifecycles.
A synchronization tool for lifecycled objects (see
Lifecycle,
that need happens-before between start() and other methods and/or to check that the object was successfully started
in other methods.A Module to add lifecycle management to the injector.
A scope that adds objects to the Lifecycle.
An
OutputStream that limits how many bytes can be written.An area for limited temporary storage on disk.
An extendable linear shard spec.
Input format that breaks the input on newlines, and returns a single column named
LinesReader.LINE_COLUMN.Reader for
LinesInputFormat.Input row backed by a
List.Adapter for reading
ListBasedInputRow.A means of pulling segment files into a destination directory
Implementation that uses local filesystem.
LocalTmpStorageConfig is a provider for temporary directories.A Logger for usage inside of Druid.
A logical segment can represent an entire segment or a part of a segment.
Specialization of
AggregateCombiner for primitive long aggregations.Vectorized implementation of the
LongAnyBufferAggregatorReader for fields written by
NumericArrayFieldWriter.getLongArrayFieldWriter(org.apache.druid.segment.ColumnValueSelector)Reader for columns written by
LongArrayFrameColumnWriterColumnar frame writer for
ColumnType.LONG_ARRAY columnsspecialized
LongBivariateFunctionVectorProcessor for processing (double[], long[]) -> long[]specialized
LongBivariateFunctionVectorProcessor for processing (double[], double[]) -> long[]common machinery for processing two input operators and functions, which should always treat null inputs as null
output, and are backed by a primitive values instead of an object values (and need to use the null vectors instead of
checking the vector themselves for nulls)
this one is specialized for producing long[], see
DoubleBivariateFunctionVectorProcessor for
double[] primitives.specialized
LongBivariateFunctionVectorProcessor for processing (long[], double[]) -> long[]specialized
LongBivariateFunctionVectorProcessor for processing (long[], long[]) -> long[]This interface is convenient for implementation of "long-sourcing"
ColumnValueSelectors, it provides default
implementations for all ColumnValueSelector's methods except BaseLongColumnValueSelector.getLong().Unsafe for concurrent use from multiple threads.
Column Serializer for long column.
This aggregator is a no-op aggregator with a fixed non-null output value.
BufferAggregator variant of LongConstantAggregatorVectorAggregator variant of LongConstantAggregatorReads values written by
LongFieldWriter.Wraps a
BaseLongColumnValueSelector and writes individual values into frame rows.Vectorized version of on heap 'last' aggregator for column selectors with type LONG..
A null-aware aggregator factory.
specialized
LongUnivariateFunctionVectorProcessor for processing (long[]) -> long[]common machinery for processing single input operators and functions, which should always treat null input as null
output, and are backed by a primitive value instead of an object value (and need to use the null vector instead of
checking the vector itself for nulls)
this one is specialized for producing long[], see
DoubleUnivariateFunctionVectorProcessor for
double[] primitives.specialized
LongUnivariateFunctionVectorProcessor for processing (long[]) -> long[]specialized
UnivariateObjectFunctionVectorProcessor for processing (Object[]) -> long[]Represents a lookup.
Users of Lookup Extraction need to implement a
LookupExtractorFactory supplier of type LookupExtractor.Provides
LookupExtractorFactoryContainer to query and indexing time dimension transformations.This interface is empty because it only exists to signal intent.
A
Segment that is based on a LookupExtractor.Marks the object to be managed by
Lifecycle and set to be on
Lifecycle.Stage.NORMAL stage.Marks the object to be managed by
Lifecycle and set to be on
Lifecycle.Stage.ANNOUNCEMENTS stage.Marks the object to be managed by
Lifecycle and set to be on
Lifecycle.Stage.INIT stage.Marks the object to be managed by
Lifecycle and set to be on
Lifecycle.Stage.SERVER stage.An
IndexedTable.Index backed by a Map.A
JoinableFactory that delegates to the appropriate factory based on the datasource.Lookup extractor backed by any kind of map.
Facilitates using try-with-resources with
MappedByteBuffer.Inspector used by
DataSourceMetadataQuery to populate
DataSourceMetadataResultValue.getMaxIngestedEventTime().A SplitHintSpec that can create splits of multiple files.
Allocator of WritableMemory.
Factory for
MemoryAllocator.Similar to LinkedBlockingQueue but can be bounded by the total byte size of the items present in the queue
rather than number of items.
Holder for a value and the memory increase in the internal dictionary associated with the increase
An open-addressed hash table with linear probing backed by
WritableMemory.Reusable pointer to a location in
Memory.Reference to a particular region of some
Memory.Iterator that merges a collection of sorted iterators using a comparator.
Used to perform an n-way merge on n ordered sequences
Expresses a single compare-and-swap update for MetadataStorageConnector's compareAndSwap method
A
ColumnSelectorFactory that wraps multiple ColumnSelectorFactory and delegates to one of
them at any given time.Vectorized selector for a multi-valued string-typed column.
This class is meant to represent a simple wrapper around a bitmap class.
This naive partitioning operator assumes that it's child operator always gives it RowsAndColumns objects that are
a superset of the partitions that it needs to provide.
A NaiveSorter sorts a stream of data in-place.
A naive sort operator is an operation that sorts a stream of data in-place.
Common stuff for "named" functions of "functional" expressions, such as
FunctionExpr,
ApplyFunctionExpr, and ExprMacroTable.ExprMacroFunctionExpr.supplies direct, 64k, clear()'d ByteBuffers wrapped in a ResourceHolder.
Native I/O operations in order to minimize cache impact.
Binding annotation for implements of interfaces that are focused on running native queries.
Base implementation for columns created with
AutoTypeColumnSchema and handled with
NestedCommonFormatColumnHandler to allow ease of merge via
AutoTypeColumnMerger by providing a common implementation.Defines storage format for 'auto' and json columns.
ColumnPartSerde shared by all NestedCommonFormatColumn.Overrides
NestedCommonFormatColumnFormatSpec so that NestedCommonFormatColumnPartSerde.FormatSpec.getBitmapEncoding() participates in serde
so that it can store the complete object in the column metadataBasic serializer implementation for the
NestedCommonFormatColumn family of columns.Nested column
DimensionSchema.Serializer for
NestedCommonFormatColumn which can store nested data.Nested data column with optimized support for simple arrays.
Describes the basic shape for any 'nested data' (
StructuredData) ComplexColumn implementation along
with basic facilities for caching any columns created and methods for retrieving selectors for nested field columns.Supplies indexes for nested field columns
NestedFieldDictionaryEncodedColumn of
NestedDataComplexColumn.Optimized virtual column that can make direct selectors into a
NestedDataComplexColumn or any associated
nested fields (NestedFieldDictionaryEncodedColumn) including using
their indexes.Create a
DimensionSelector for a nested field on top of a ColumnValueSelectorProcess the "raw" data to extract vectors of values with
NestedPathFinder.find(Object, List), returning
a value matching an expected type.Abstract class for nested file formats such as JSON, ORC, etc.
Specialized virtual column for
NestedDataExpressions.JsonMergeExprMacro.Specialized virtual column for
NestedDataExpressions.JsonObjectExprMacro.Represents an expression and a target type.
Implementation of
HttpClient built using Netty.A "having" spec that always evaluates to false
NOTE:
This class should be deprecated after Druid supports configurable index types on dimensions.
Represents "absent" column.
Treats all rows as null.
A vector aggregator that returns the default numeric value.
Error response transform strategy that does nothing and simply return the same Exception back without any change
Default implementation of
ColumnIndexSupplier for columns which do not
have any indexes.NoneGranularity does not bucket data
Deprecated.
Since Druid 0.15.0.
Mostly used for test purpose.
Allows all data sources (no restrictions).
A no-op implementation of QueryableIndex.
Implementation of
QueryProcessingPool that throws when any query execution task unit is submitted to it.This class is used only in
RealtimeIndexTask which is deprecated now.Represents a special kind of policy restriction, indicating that this table is restricted, but doesn't impose any restriction
to a user.
Nice filter you have there...
The logical "not" operator for the "having" clause.
A failure class that is used to indicate that something is just not implemented yet.
Null-aware numeric
AggregateCombiner.Null-aware numeric
Aggregator.Abstract superclass for null-aware numeric aggregators.
Null-aware numeric
BufferAggregator.A wrapper around a non-null-aware VectorAggregator that makes it null-aware.
Wrapper of
TypeStrategy for nullable types, which stores TypeStrategies.IS_NULL_BYTE or
TypeStrategies.IS_NOT_NULL_BYTE in the leading byte of any value, as appropriate.A ColumnPartSerde to read and write null-only columns.
Provides
BitmapColumnIndex for all null rows in a column, to use with IS/IS NOT NULL filtersColumnIndexSupplier for columns which only have an ImmutableBitmap to indicate which rows only have
null values, such as LongNumericColumnPartSerdeV2, DoubleNumericColumnPartSerdeV2, and
FloatNumericColumnPartSerdeV2.This shardSpec is used only for the segments created by overwriting tasks with segment lock enabled.
PartitionChunk corresponding to
NumberedOverwriteShardSpecAn extendable linear shard spec containing the information of core partitions.
Base type for on heap 'any' aggregator for primitive numeric column selectors
Base type for buffer based 'any' aggregator for primitive numeric column selectors
Reader class for the fields written by
NumericArrayFieldWriter.Base implementation of the column value selector that the concrete numeric field reader implementations inherit from.
Writes the values of the type ARRAY where X is a numeric type to row based frames.
Implementations of this class reads columns written by the corresponding implementations of
NumericArrayFrameColumnWriter.Parent class for the family of writers writing numeric arrays in columnar frames.
Reads the fields created by the
NumericFieldWriter.Helper class which allows the inheritors to fetch the nullity of the field located at fieldPointer's position in
the dataRegion.
FieldWriter for numeric datatypes.
Factory for
NumericFieldWriterBase type for on heap 'first' aggregator for primitive numeric column selectors
Base type for buffer based 'first' aggregator for primitive numeric column selectors
Base type for on heap 'last' aggregator for primitive numeric column selectors..
Base type for buffer based 'last' aggregator for primitive numeric column selectors
This could probably share a base type with
NumericFirstBufferAggregator ...An optimized column value
BitmapColumnIndex provider for specialized processing of numeric value ranges.Specialization of
AggregateCombiner for object aggregations.Object based column selector.
This class can be used to wrap Double Aggregator that consume double type columns to handle Object type.
This class can be used to wrap Double BufferAggregator that consume double type columns to handle Object type.
This class can be used to wrap Float Aggregator that consume float type columns to handle Object type.
This class can be used to wrap Float BufferAggregator that consume float type columns to handle Object type.
This class can be used to wrap Long Aggregator that consume long type columns to handle Object type.
This class can be used to wrap Long BufferAggregator that consume long type columns to handle Object type.
Restricts selector usage to only allow
BaseObjectColumnValueSelector.getObject().many objects enter, one object leaves...
Response handler that delegates successful responses (2xx response codes) to some other handler, but returns
errors (non-2xx response codes) as Strings.
Default implementation of
TypeStrategy for all ComplexMetricSerde
implementations that just wraps the ObjectStrategy they are required to implement.Wraps a
ColumnValueSelector, calls BaseObjectColumnValueSelector.getObject(), interprets that value using
ExprEval.ofType(org.apache.druid.math.expr.ExpressionType, java.lang.Object), and casts it using ExprEval.castTo(org.apache.druid.math.expr.ExpressionType).Wraps a
ColumnValueSelector, calls BaseObjectColumnValueSelector.getObject() and provides primitive numeric
accessors based on that object value.Generic object matcher on top of a
VectorObjectSelector.The "mutable" version of a ReadableOffset.
Projection of
OnheapIncrementalIndex for AggregateProjectionSpecAn Operator interface that intends to have implementations that align relatively closely with the Operators that
other databases would also tend to be implemented using.
This is the return object from a receiver.
A factory for Operators.
Provides a sequence on top of Operators.
Logical OR filter operation
The logical "or" operator for the "having" clause.
SysMonitor implemented using
invalid reference
oshiRepresents an output channel for some frame processor.
Factory for generating channel pairs for output data from processors.
A list of
OutputChannel.Interface to represent a class which can have overshadow relation between its instances.
ShardSpec for non-first-generation segments.
ParallelCombiner builds a combining tree which asynchronously aggregates input entries.
Artisanal, locally-sourced, hand-crafted, gluten and GMO free, bespoke, free-range, organic, small-batch parallel
merge combining sequence.
ListenableFuture that allows ParallelMergeCombiningSequence to be
registered with QueryWatcher.registerQueryFuture(org.apache.druid.query.Query<?>, com.google.common.util.concurrent.ListenableFuture<?>) to participate in query
cancellation or anything else that has a need to watch the activity on the merge pool.Token used to stop internal parallel processing across all tasks in the merge pool.
Metrics for the execution of a
ParallelMergeCombiningSequence on the ForkJoinPoolBinding annotation for org.apache.druid.server.DruidNode.
ParseException can be thrown on both ingestion side and query side.
A handler for
ParseExceptions thrown during ingestion.Class that can parse Strings into Maps.
Deprecated.
This interface is used in the segment allocation protocol when it is coordinated by the Overlord; when appending
segments to an existing datasource (either streaming ingestion or batch append) or any case when segment
lock is used.
List of range partition boundaries.
A PartitionChunk represents a chunk of a partitioned(sharded) space.
A channel which can contain multiple partitions of data.
Provides an interface to read a partitioned frame channel.
An object that clumps together multiple other objects which each represent a shard of some space.
This enum is used a parameter for several methods in
VersionedIntervalTimeline, specifying whether only
complete partitions should be
considered, or incomplete partitions as well.This operator sorts rows inside partitioned RACs, on the sort columns.
PartitionsSpec describes the secondary partitioning method for data ingestion.
Deprecated.
This Interface is used as a MixIn for ObjectMapper objects when there is a desire to avoid serializing a Password
from a PasswordProvider to JSON in plaintext when that JSON is going to be used for purposes that don't require the
password to be present, such as logging to a file.
PeriodGranularity buckets data based on any custom time period
This runner optimizes queries made on a single segment, using per-segment information,
before submitting the queries to the base runner.
Holds information about a single segment that Query objects can use to optimize themselves
when they are run on that single segment.
ErrorResponseTransformStrategy that modifies the error message of a DruidException based on the
persona.Interface for methods describing physical segments such as
QueryableIndexSegment and
IncrementalIndexSegment that is not typically used at query time (outside of metadata queries).Extensible interface for a granular-level (e.x.
Interface for enforcing policies on data sources and segments in Druid queries.
Provides the ability to create "polymorphic" bindings where the polymorphism is actually just making a decision
based on a value in Properties.
This
TopNAlgorithm is highly specialized for processing aggregates on string columns that are
ColumnCapabilities.isDictionaryEncoded() and ColumnCapabilities.areDictionaryValuesUnique().A
ColumnValueSelector that wraps a base selector but might also generate null values on demand.Functionally similar to an Aggregator.
A Cursor decorator used by
HashJoinSegmentCursorFactory.makeCursorHolder(CursorBuildSpec) to add post-join
virtual columns and filters.Creates
ValueMatcher that apply a predicate to each value.Virtual column form of
PrefixFilteredDimensionSpecAn implementation of
PrioritizedCallable that also lets caller get access to associated QueryRunner
It is used in implementations of QueryRunnerFactoryA Processor is a bit of logic that processes a single RowsAndColumns object to produce a new RowsAndColumns
object.
Processor and success callback returned by
ProcessorManager.next().Used by
FrameProcessorExecutor.runAllFully(org.apache.druid.frame.processor.manager.ProcessorManager<T, R>, int, org.apache.druid.frame.processor.Bouncer, java.lang.String) to manage the launching of processors.Utility functions for creating
ProcessorManager.Fetches data from top-level procfs files for metrics.
Terminal marker indicating that a
Filter can be dropped because it matches the filter of a projectionTransformed
CursorBuildSpec to run against a projection and remapping ofReturns true if column is defined in
AggregateProjectionSpec.getGroupingColumns() OR if the column does not
exist in the base table.Convenience class for holding a pair of string key and templated value.
A Provider of a Supplier that uses a Provider to implement the Supplier.
This annnotation is used to inject a boolean parameter into a
DataSegment constructor, which prescribes to
drop deserialized "lastCompactionState" and don't store it in a field of a DataSegment.This annnotation is used to inject a boolean parameter into a
DataSegment constructor, which prescribes to
drop deserialized "loadSpec" and don't store it in a field of a DataSegment.Signifies that the annotated entity is a public API for extension authors.
Direct interface to memory mapped segments.
The basic implementation of
ColumnSelectorFactory over a historical segment (i.Represents a projection of some base table available to use to build a
CursorHolder
by a CursorFactory.This exception is for QueryResource and SqlResource to surface when a query is cast away after
it hits a resource limit.
Immutable holder for query context parameters with typed access methods.
Base serializable error response.
A
DruidException.Failure that serves to cover conversions from QueryException.Exception representing a failed query.
Provides facilities to executes the given query type by utilizing the
QuerySegmentWalker to run subqueries if necessary.Abstraction wrapping
ServiceMetricEvent.Builder and allowing to
control what metrics are actually emitted, what dimensions do they have, etc.An immutable composite object of
Query + extra stuff needed in QueryRunners.This class implements the logic of how units of query execution run concurrently.
Wrapper class on the queryResourceId string.
An interface that defines the nitty gritty implementation details of a Query on a Segment
An interface for query-handling entry points.
This exception is thrown when a query does not finish before the configured query timeout.
The broker-side (also used by server in some cases) API for a specific Query type.
This exception is for the query engine to surface when a query cannot be run.
This interface is in a very early stage and should not be considered stable.
Implementation of
NonBlockingPool based on a pre-created BlockingQueue that never actually blocks.Reusable IndexedInts that returns sequences [0, 1, ..., N].
Set from start (inclusive) to end (exclusive).
Channel backed by a byte stream that is continuously streamed in using
ReadableByteChunksFrameChannel.addChunk(byte[]).Channel that concatenates a sequence of other channels that are provided by an iterator.
Pointer to a field position in some memory.
Channel backed by a
FrameFile.Interface for reading a sequence of frames.
Pointer to a row position in some memory.
Channel backed by an
InputStream.Empty channel.
A ReadableOffset is an object that provides an integer offset, ostensibly as an index into an array.
Vector inspector that can supply a unique identifier of the vector to use with caching in addition to
sizing information
The result of calling
VectorValueMatcher.match(org.apache.druid.query.filter.vector.ReadableVectorMatch, boolean).Provides a batch of offsets, ostensibly as indexes into an array.
Rounds the output values from the sample() function of an AbstractRealDistribution.
This class exists to "decorate" a rows and columns such that it pretends to exist in a new ordering.
Interface to capture the pattern of providing tracked usage of some resource.
Helper object to assist in managing
Segment storage lifecycle.Base type to mark
Segment returned by ReferenceCountedObjectProvider<Segment> as 'leaf' segments,
to distinguish from other transformations which can be done on top of this segment, such as by
SegmentMapFunctionReferenceCountingCloseableObject implements something like automatic reference count-based resource management,
backed by a
Phaser.Virtual column form of
RegexFilteredDimensionSpecContains information about a REST API request that was audited.
Exception indicating that an operation failed because it exceeded some configured resource limit.
A resource pool based on
LoadingCache.The context for storing and passing data between chains of
QueryRunners.Abstract key class which provides most functionality except the
type-specific merge logic.
Boolean valued attribute with the semantics that once the flag is
set true, it stays true.
Long valued attribute that holds the accumulation of values assigned.
The base interface of a response context key.
Global registry of response context keys.
Long valued attribute that holds the latest value assigned.
Serialization result of
ResponseContext.String valued attribute that holds the latest value assigned.
Deserialize a response context.
All tables must be restricted by a policy.
A factory class for creating
Cursor instances with strict adherence to Policy restrictions.Reperesents a TableDataSource with policy restriction.
Used by
GroupByQuery for results.Utility class for conditional serde of
ResultRow objects.Serialization medium of the query results on the broker.
This Exception class can be use with
RetryUtils.This class is responsible for re-opening the underlying input
stream for the input object on the given
RetryingInputStream.retryCondition.Instances of this class are returned by
FrameProcessor.runIncrementally(it.unimi.dsi.fastutil.ints.IntSet), and are used by
FrameProcessorExecutor to manage execution.A
Grouper.Entry implementation that is reusable.A candidate is an RHS column that appears in a filter, along with the value being filtered on, plus
the joinable clause associated with the table that the RHS column is from.
As the name suggests, this class instantiates bitmaps of the types
WrappedRoaringBitmap and WrappedImmutableRoaringBitmap.
A Row of data.
An adapter between arbitrary types and the needs of callers that want to read specific columns out of those
types (treating them as rows).
Utility class for creating
RowAdapter.A
ColumnSelectorFactory that is based on an object supplier and a RowAdapter for that type of object.A
Cursor that is based on a stream of objects.Expression column value selector that examines a set of 'unknown' type input bindings on a row by row basis,
transforming the expression to handle multi-value list typed inputs as they are encountered.
Write row-based frames, where
FrameType.isRowBased().This class contains shared code between
GroupByMergingQueryRunner and GroupByRowProcessor.Utility class for creating
IndexedTable.Index instances.An IndexedTable composed of a List-based table and Map-based indexes.
A
Segment that is based on a stream of objects.Represents a basic row filter policy restriction.
A
CursorFactory implementation based on a single row-based Frame.Interface for evaluating functions on rows.
Returned by
ColumnSelectorFactory.getRowIdSupplier().A collection of meters for row ingestion stats, with support for moving average calculations.
Represents a specific sorting or hashing key.
Comparator for
RowKey instances.Denotes the ascending-descending run lengths of the fields of the keycolumns that can be compared together.
Embeds the logic to read
RowKey from a particular signature of row-based frame.A
ReadableFieldPointer that is derived from a row-based frame.Could be used as a simple "row number supplier" for
RowPointer implementations.Extension of
TimeAndDimsPointer, that keeps "row number" of the current data point in some collection of data
points, that actually makes this data point to be called "row", and the collection, thus, "collection of rows".Class for reading rows in the same format as used by
FrameType.isRowBased().An interface representing a chunk of RowsAndColumns.
Deserializer for
RowsAndColumns returning as an instance of FrameRowsAndColumnsSerializer for
RowsAndColumns by converting the instance to FrameRowsAndColumnsAn interface for "decorating" a rowsAndColumns.
Type signature for a row in a Druid datasource or query result.
Used by
RowBasedCursorFactory and RowBasedCursor to walk through rows.This RTree has been optimized to work with bitmap inverted indexes.
Information about a continguous run of keys, that has the same sorting order
Frame processor that simply runs a
Runnable, once.This includes the state of a task in the task runner not covered by
TaskState, this state is not stored
in databaseSafety first! Don't trust something whose contents you locations to read and write stuff to, but need a
Buffer or WritableBuffer? use this!Safety first! Don't trust something whose contents you locations to read and write stuff to, but need a
Memory or WritableMemory? use this!Serializer for a
ScalarDoubleColumnNested field writer for double type columns of
NestedDataColumnSerializer.Serializer for a
ScalarLongColumnNested field writer for long type columns of
NestedDataColumnSerializer.Serializer for a string
NestedCommonFormatColumn that can be read with
StringUtf8DictionaryEncodedColumn.Nested field writer for string type columns of
NestedDataColumnSerializerA scan operator is really just a way to push down various things that can be lazily applied when data needs to
actually be read.
This iterator supports iteration through a Sequence returned by a ScanResultValue QueryRunner.
A Sequence that wraps the results of a ScanQuery and skips a given number of rows.
Returns a thread-unsafe iterable, that converts a sequence of
ScanResultValue to an iterable of FrameSignaturePair.Iterates over the scan result sequence and provides an interface to clean up the resources (if any) to close the
underlying sequence.
This comparator class supports comparisons of ScanResultValues based on the timestamp of their first event.
Representation of schema payload, includes information like RowSignature and aggregator factories.
Wrapper over
SchemaPayload to include numRows information.A class which implements SearchableVersionedDataFinder can find a DataDescriptor which describes the most up to
date version of data given a base descriptor and a matching pattern.
Implementations could be injected using
PolyBind
.optionBinder(binder, Key.get(SearchQueryMetricsFactory.class))
.addBinding("myCustomSearchQueryMetricsFactory")
.to(MyCustomSearchQueryMetricsFactory.class);
And then setting property:
druid.query.search.queryMetricsFactory=myCustomSearchQueryMetricsFactory
In Druid, ingested data are primarily partitioned based on time range (@link GranularitySpec#getSegmentGranularity),
and then secondly partitioned based on the given
PartitionsSpec.The difference between this class and
DataSegment is that DataSegment contains the segment metadata only, while this class represents the actual
body of segment data, queryable.Helper class to build
SegmentAnalysis objects.This class is a pointer to a time range of a segment to be queried.
Identifier of
DataSegment.Factory that loads segment files from the disk and creates
Segment objectResult of killing data segments using
DataSegmentKiller.Encapsulates segment level information like numRows, schema fingerprint.
The event that gets generated whenever a segment is committed
Wrapper for a
SegmentDescriptor and Optional<Segment>, the latter being created by a
SegmentMapFunction being applied to a ReferenceCountedSegmentProvider.Compact representation of segment schema for multiple segments.
SplitHintSpec for DruidInputSource.This class represents the current state of a segment in the cluster and encapsulates the following:
the
Objects of this class are used to sync the state of segments from the Coordinator to different services, typically the Broker.
DataSegment object
overshadowed status of the segment
replication factor of the segment
number of rows in the segment
if the segment is realtime
Objects of this class are used to sync the state of segments from the Coordinator to different services, typically the Broker.
VersionedIntervalTimeline for DataSegment objects.Utility methods useful for implementing deep storage extensions.
Utility for creating
Segment objects for concrete datasources.SegmentWriteOutMedium is an umbrella "resource disposer" for temporary buffers (in the form of
WriteOutBytes,
obtained by calling SegmentWriteOutMedium.makeWriteOutBytes() on the SegmentWriteOutMedium instance), that are used during new Druid
segment creation, and other resources (see SegmentWriteOutMedium.getCloser()).Class returned by
ColumnHolder.Differentiating factor between the first and the last aggregator.
Recommended to use
EqualityFilter or NullFilter insteadThis filter is to select the rows where the
SelectorFilter.dimension has the SelectorFilter.value.A
DruidPredicateFactory that checks if input values equal a specific, provided value.Deprecated.
Binding annotation for org.apache.druid.server.DruidNode.
Annotation used to indicate that the method is used as a creator for a semantic interface.
A Sequence represents an iterable sequence of elements.
A HTTP response handler which uses sequence input streams to create a final InputStream.
Processor manager based on a
Sequence.valid call sequence
The SerializablePairLongStringSerde serializes a Long-String pair (SerializablePairLongString).
serializes a Long/String pair in the context of a column/segment.
serializes a Long/String pair as
Long:Integer:bytes
simple utility class useful for when multiple passes of input are needed for encoding (e.g.
Main interface for "serializeable something" in Druid segment serialization.
Immutable metric event emitted by a Druid
ServiceEmitter.Builder for a
ServiceMetricEvent.SettableColumnValueSelectors are used in
QueryableIndexIndexableAdapter.RowIteratorImpl.A column selector factory, that represents the column values from multiple underlying cursors.
Settable implementation of
DimensionSelector.A simple
ReadableFieldPointer that returns the position and the length that was set on its object.Virtual column that returns a changeable value, via
SettableLongVirtualColumn.setValue(long).Simple map backed object binding
A settable Supplier.
A BaseDoubleColumnValueSelector impl to return settable double value on calls to
BaseDoubleColumnValueSelector.getDouble()A BaseFloatColumnValueSelector impl to return settable float value on calls to
BaseFloatColumnValueSelector.getFloat()A BaseLongColumnValueSelector impl to return settable long value on calls to
BaseLongColumnValueSelector.getLong()A Marker interface that exists to combine ShardSpec objects together for Jackson.
ShardSpec type names.
Implementation of
ColumnSelectorFactory for ShimCursor.Adapter from
VectorCursor to Cursor.DimensionSelector that internally uses a VectorObjectSelector.BitmapColumnIndex with Druids "default" ColumnIndexCapabilities.This is an abstract class inherited by various
AggregatorFactory implementations that consume double input
and produce double output on aggregation.SimpleBitmapColumnIndex which wraps a single ImmutableBitmap with no 'unknowns', e.g.SimpleBitmapColumnIndex for anything which can compute an Iterable<ImmutableBitmap> in some mannerThis is an abstract class inherited by various
AggregatorFactory implementations that consume long input
and produce long output on aggregation.Simple Sequence based on an Iterable, created using
Sequences.simple(java.lang.Iterable<T>).Make a 2 argument, math processor with the following type rules
long, long -> double
long, double -> double
double, long -> double
double, double -> double
Make a 2 argument, math processor with the following type rules
long, long -> long
long, double -> long
double, long -> long
double, double -> long
Make a 2 argument, math processor with the following type rules
long, long -> long
long, double -> double
double, long -> double
double, double -> double
Make a 1 argument math processor with the following type rules
long -> double
double -> double
using simple scalar functions
DoubleUnivariateLongFunction and DoubleUnivariateDoubleFunctionMake a 1 argument math processor with the following type rules
long -> long
double -> long
using simple scalar functions
LongUnivariateLongFunction and LongUnivariateDoubleFunctionMake a 1 argument math processor with the following type rules
long -> long
double -> double
using simple scalar functions
LongUnivariateLongFunction and DoubleUnivariateDoubleFunctionPartition a segment by a single dimension.
See
BucketNumberedShardSpec for how this class is used.ShardSpec for range partitioning based on a single dimensionDeprecated.
Reusable IndexedInts that represents a sequence of a solo value [X].
Like
ExpressionColumnValueSelector, but caches the most recently computed value and re-uses it in the case
of runs in the underlying column.Wraps a single
MemoryAllocator.A special
DimensionSelector for projected time columns
- it assumes time values are scanned once and values are grouped together
(i.e.Like
ExpressionColumnValueSelector, but caches results for the first CACHE_SIZE dictionary IDs of
a string column.A
DimensionSelector decorator that directly exposes the underlying dictionary id in SingleStringInputDeferredEvaluationExpressionDimensionSelector.getRow(),
saving expression computation until SingleStringInputDeferredEvaluationExpressionDimensionSelector.lookupName(int) is called.A
SingleValueDimensionVectorSelector decorator that directly exposes the underlying dictionary ids in
SingleStringInputDeferredEvaluationExpressionDimensionVectorSelector.getRowVector(), saving expression computation until SingleStringInputDeferredEvaluationExpressionDimensionVectorSelector.lookupName(int) is called.This AggregatorFactory is meant to wrap the subquery used as an expression into a single value
and is expected to throw an exception when the subquery results in more than one row
Serializer that produces
ColumnarInts.Vectorized selector for a singly-valued string-typed column.
A Sequence that skips the first few elements.
Reusable IndexedInts, that could represent a sub-sequence ("slice") in a larger IndexedInts object.
Class that works in conjunction with FileSmoosher.
Creates a sorted dictionary given some existing dictionary, storing a mapping of both sorted id to unsorted id
and unsorted id to sorted id
An operator that can join the data streams from other operators.
A thing that makes Matrixs in an assumed sorted-fashion.
A matrix thingie
Container to collect a set of sorted
Indexed representing the global value dictionaries of some
NestedDataComplexColumn, to later use with DictionaryMergingIterator
to merge into a new global dictionaryWe throw away all invalid spatial dimensions
Deprecated.
Manages class specialization during query processing.
Base class for virtual columns that are specialized implementations of expressions.
Grouper based around a single underlying
BufferHashGrouper.In native parallel indexing, the supervisor task partitions input data into splits and assigns each of them
to a single sub task.
Splittable InputSource.
Response object for org.apache.druid.msq.sql.resources.SqlTaskResource#doPost.
Simultaneously sorts and limits its input.
StagedSerde is useful when you have objects that have their own internal logic to serialize, but you wish to
compose the results of multiple serialized objects into a single ByteBuffer (or wrapped byte[]).The ClassLoader that gets used when druid.extensions.useExtensionClassloaderFirst = false.
Create the startup injector used to "prime" the modules for the
main injector.
Wrapper over
Stopwatch to provide some utility
methods such as Stopwatch.millisElapsed(), Stopwatch.restart(), Stopwatch.hasElapsed(Duration).It's useful to return this object when multiple serializable objects are to be composed into a single ByteBuffer
or byte[].
Low level interface for interacting with different storage providers like S3, GCS, Azure and local file system.
A streaming grouper which can aggregate sorted inputs.
A strict
TypeIdResolver implementation that validates all incoming type ids.Reads fields written by
StringArrayFieldWriterLike
StringFieldWriter, but reads arrays from a ColumnValueSelector instead of reading from
a DimensionSelector.Reader for
ColumnType.STRING_ARRAY.Comparison using the natural comparator of
String.NaturalComparator refers to the natural ordering of the type that it refers.
Creates
ValueMatcher that match constants.DimensionDictionary for String dimension values.
An enum that provides a way for users to specify what encoding should be used when hashing strings.
JsonInclude filter for StringEncoding that ignores UTF16LE, which is the typical default
for aggregators.Adapter to convert
Indexed<ByteBuffer> with utf8 encoded bytes into Indexed<String> to be friendly
to consumers.Reads fields written by
StringFieldWriter or StringArrayFieldWriter.Wraps a
DimensionSelector and writes to rframe rows.For backwards compatibility; equivalent to a regular StringFirstAggregatorFactory.
Reader for
StringFrameColumnWriter, type ColumnType.STRING.HttpResponseHandler for stream data of string type.For backwards compatibility; equivalent to a regular StringLastAggregatorFactory.
String matcher on top of a
VectorObjectSelectorClass to be used to obtain String representation of runtime shape of one or several
HotLoopCallees.Represents a tuple of String values, typically used to represent
(single-valued) dimension values for an InputRow.
DictionaryEncodedColumn<String> for a column which has a ByteBuffer based UTF-8 dictionary.Base type for a
VectorObjectSelector for a dictionary encoded ColumnType.STRING
built around a ColumnarMultiInts.Base type for a
MultiValueDimensionVectorSelector for a dictionary encoded ColumnType.STRING
built around a ColumnarMultiInts.Base type for a
SingleValueDimensionVectorSelector for a dictionary encoded ColumnType.STRING
built around a ColumnarInts.Base type for a
VectorObjectSelector for a dictionary encoded ColumnType.STRING
built around a ColumnarInts.Supplier for
StringUtf8DictionaryEncodedColumnAs of OpenJDK / Oracle JDK 8, the JVM is optimized around String charset variable instead of Charset passing, that
is exploited in
StringUtils.toUtf8(String) and StringUtils.fromUtf8(byte[]).Deprecated.
Accumulates the list of literal field paths and a rough size estimation for
StructuredDataProcessorAn annotation that tells all subclasses of the annotated class to not use the default implementation of hashCode
and equals that is provided by
Object.If there's a subquery, run it instead of the outer query
Sorts and partitions a dataset using parallel, possibly-external merge sort.
Like
SuperSorterProgressTracker, but immutable.A thread-safe class that keeps track of the progress of an n-way, multilevel merge sort.
This AggregatorFactory is meant for wrapping delegate aggregators for optimization purposes.
Annotation for suppressing spotbugs checks when necessary.
An emitter than that offers the ability to direct an event to multiple emitters based on the event's feed.
Implements
TaskLogs by delegating to different task log providers based on the functionality required.Deprecated.
System fields that can appear when reading data.
Decorator of
InputRow from an InputEntity.An
InputSource that can generate system fields.Container of
SystemField.Model class containing the id, type and groupId of a task
These fields are extracted from the task payload for the new schema and this model can be used for migration as well.
Contains a
TaskIdentifier and its TaskStatus.Cleans up stale task logs from deep storage.
Something that knows how to persist local task logs to some form of long-term storage.
Something that knows how to stream logs for tasks.
Lookup types and parameters for task lookups in the metadata store.
Task lookup for complete tasks.
Task state in the metadata store.
Something that knows how to push a task payload before it is run to somewhere
a ingestion worker will be able to stream the task payload from when trying to run the task.
TaskReport objects contain additional information about an indexing task, such as row statistics, errors, and
published segments.
Represents an ordered map from report key to a TaskReport that is compatible
for writing out reports to files or serving over HTTP.
Represents the status of a task from the perspective of the coordinator.
Abstract
InputEntityReader for text format readers such as CSV or JSON.TimeAndDimsIterator (in conjunction with
TimeAndDimsPointer) is an Iterator-like
abstraction, designed for allocation-free transformation, merging, combining and iteration over a stream of data
points.TimeAndDimsPointer is used in conjunction with
TimeAndDimsIterator, it's an _immutable_ object that points to
different logical data points, as TimeAndDimsIterator.moveToNext() is called.Returns min/max values of
ColumnHolder.TIME_COLUMN_NAME for a segment.Specialization of
QueryMetrics for TimeseriesQuery.Implementations could be injected using
PolyBind
.optionBinder(binder, Key.get(TimeseriesQueryMetricsFactory.class))
.addBinding("myCustomTimeseriesQueryMetricsFactory")
.to(MyCustomTimeseriesQueryMetricsFactory.class);
And then setting property:
druid.query.timeseries.queryMetricsFactory=myCustomTimeseriesQueryMetricsFactory
TimewarpOperator is an example post-processing operator that maps current time
to the latest period ending withing the specified data interval and truncates
the query interval to discard data that would be mapped to the future.
Builds segment write out medium objects that are based on temporary files.
A shard spec to represent tombstones.
Implements a QueryRunner that takes
ResultSerializationMode into
account.This
ColumnSelectorStrategy is used by all TopNAlgorithm to provide
selector value cardinality to TopNParams (perhaps unecessarily, but that is another matter), but is primarily
used by HeapBasedTopNAlgorithm to serve as its value aggregates store.oddly specific
Segment inspector for topN optimizationsA Builder for TopNQuery.
Specialization of
QueryMetrics for TopNQuery.Implementations could be injected using
PolyBind
.optionBinder(binder, Key.get(TopNQueryMetricsFactory.class))
.addBinding("myCustomTopNQueryMetricsFactory")
.to(MyCustomTopNQueryMetricsFactory.class);
And then setting property:
druid.query.topN.queryMetricsFactory=myCustomTopNQueryMetricsFactory
Simultaneously sorts and limits its input.
Tree-of-losers tournament tree used for K-way merging.
A row transform that is part of a
TransformSpec.TransformableRowIterator tightens
RowIterator.getPointer() contract, that allows to transform iterated
rows without allocations on each iterations, and reuse the mechanics of the underlying iterator.Specifies how input rows should be filtered and transforms.
Utility methods to map the primitive numeric types into an equi-wide byte representation, such that the
given byte sequence preserves the ordering of the original type when done byte comparison.
Based on
BiConsumerThis exception is thrown when
ResponseContext is truncated after serialization
in historicals or realtime tasks.Approximately like the SQL 'IN' filter, with the main difference being that this will match NULL values if contained
in the values list instead of ignoring them.
Create
TypeSignature of a TypeDescriptor.This interface serves as a common foundation for Druids native type system, and provides common methods for reasoning
about and handling type matters.
Read and write a non-null ARRAY which is permitted to have null elements (all elements are always read and written
with a
NullableTypeStrategy wrapper on the TypeStrategy of the
TypeSignature.getElementType().Read and write non-null DOUBLE values.
Read and write non-null FLOAT values.
Read and write non-null LONG values.
Read and write non-null UTF8 encoded String values.
TypeStrategy provides value comparison and binary serialization for Druid types.
Reperesents a UNION ALL of two or more datasources.
An
IndexedTable.Index backed by an int array.A type that has exactly one value.
common machinery for processing single input operators and functions, which are backed by an object value instead of
a primitive value (so do not need to use the null vector, and instead can check the value vector itself for nulls)
The cursor to help unnest MVDs without dictionary encoding and ARRAY type selectors.
The data source for representing an unnest operation.
The cursor to help unnest MVDs with dictionary encoding.
This type of ParseException is meant to be used within ingestion to hold parse exception information for
rows that were partially parseable but had one or more unparseable columns, such as when passing a non-numeric
value to a numeric column.
This utiliy class enables runtime access to sun.misc.Unsafe using reflection,
since Unsafe may not be directly accessible in newer JDK versions
Signifies that the annotated entity is an unstable API for extension authors.
Dummy reader for unsupported types.
A URIDataPuller has handlings for URI based data
Annotating test methods, which names have "parametersFor" prefix, and used by
JUnitParamsRunner, see
https://github.com/Pragmatists/junitparams/wiki/Quickstart.Annotation for members, which are used in generated code (e.
Construct a
BitmapColumnIndex for a set of utf8 byte values which might be present in the column.Streams array of integers out in the binary format described by
V3CompressedVSizeColumnarMultiIntsSupplierThe format is mostly the same with
CompressedVSizeColumnarMultiIntsSupplier (which has version 0x2, so we
call it V2), the only difference is V3's offsets is not VSize encoded, it's just compressed.Used by
AutoTypeColumnIndexer to build the value dictionary, which can be converted into a
SortedValueDictionary to sort and write out the values to a segment with ValueDictionary.getSortedCollector().Construct a
BitmapColumnIndex for a specific value which might be present in the column.An object that returns a boolean indicating if the "current" row should be selected or not.
Utility methods for creating
ValueMatcher instances.Construct a
BitmapColumnIndex for a set of values which might be present in the column.This enumeration defines the high level classification of the Druid type system, used by
ColumnType to
indicate the type of data stored in columns and produced by expressions and aggregations, used to allow query
processing engine algorithms to compute results, used to compute query result row signatures, and all other type
needs.Nested field writer for array type columns of
NestedDataColumnSerializer.NestedCommonFormatColumn for single type array columns, and mixed type columns.Make a
VectorObjectSelector for a dictionary encoded column that coerces mixed types to a common typeSerializer for a
NestedCommonFormatColumn for single type arrays and mixed type columns, but not columns
with nested data.Internal serializer used to serialize a
VariantColumn.Nested field writer for mixed type scalar or array columns of
NestedDataColumnSerializer.An object that can aggregate metrics into a ByteBuffer, from vectorized column selectors.
Class that encapsulates knowledge about how to create vector column processors.
A class that comes from
VectorCursor.getColumnSelectorFactory() and is used to create vector selectors.A semantic interface for use with
Column objects.Vectorized cursor used during query execution.
Class that helps vectorized query engines handle "granularity" parameters.
Contains logic to process a groupBy query on a single
CursorFactory in a vectorized
manner.Like a
Grouper, but vectorized.Implementation class for ReadableVectorMatch.
Make a 2 argument, math processor with the following type rules
long, long -> double
long, double -> double
double, long -> double
double, double -> double
Make a 2 argument, math processor with the following type rules
long, long -> long
long, double -> long
double, long -> long
double, double -> long
Make a 2 argument, math processor with the following type rules
long, long -> long
long, double -> double
double, long -> double
double, double -> double
Vectorizized addition processor factory
Make a 1 argument math processor with the following type rules
long -> double
double -> double
1 argument math processor with the following type rules
long -> long
double -> long
1 argument math processor with the following type rules
long -> long
double -> double
Vectorized object selector.
The movable version of ReadableVectorOffset.
Common interface for vectorized column selectors, matchers, etc, where callers are given the ability to inspect
current and maximum vector sizes.
An object that returns a boolean indicating if the "current" row should be selected or not.
Vectorized selector for primitive columns.
VersionedIntervalTimeline is a data structure that manages objects on a specific timeline.
Stores a
PartitionChunk for a given interval and version.Deprecated.
Virtual columns are "views" created over a
ColumnSelectorFactory or ColumnSelector.Class allowing lookup and usage of virtual columns.
JsonInclude filter for getVirtualColumns().Provides
ColumnCapabilities for both virtual and non-virtual columns by building on top of another base
ColumnInspector.ColumnSelectorFactory which can create selectors for both virtual and non-virtual columnsStreams integers out in the binary format described by
VSizeColumnarIntsStreams arrays of objects out in the binary format described by
VSizeColumnarMultiInts.Currently only support big endian
Unpack bitpacked long values from an underlying contiguous memory block
This Processor assumes that data has already been sorted for it.
This Processor assumes that data has already been sorted for it.
A query that can compute window functions on top of a completely in-memory inline datasource or query results.
An Operator that applies a
Processor, see javadoc on that interface for an explanation.This Processor assumes that data has already been sorted for it.
This Processor assumes that data has already been sorted for it.
WrappedBitSetBitmap implements MutableBitmap for java.util.BitSet
WrappedImmutableBitSetBitmap implements ImmutableBitmap for java.util.BitSet
Simple
Segment implementation for a segment that wraps a base segment such as
UnnestSegment or FilteredSegmentWrapper for
TimeBoundaryInspector used by Segment implementations that may filter out rows
from an underlying segment, but do not modify ColumnHolder.TIME_COLUMN_NAME.Interface for writing a sequence of frames.
Frame channel backed by a
FrameFileWriter.Appendable byte sequence for temporary storage.
A Yielder is an object that tries to act like the yield() command/continuations in other languages.
A YieldingAccumulator is used along with a Yielder in order to replicate continuations in Java.
A Sequence that is based entirely on the Yielder implementation.
An IndexedInts that always returns a row containing a single zero.