Package org.apache.druid.query.operator

Interface Operator

All Known Implementing Classes:
AbstractPartitioningOperator, AbstractSortOperator, GlueingPartitioningOperator, LimitTimeIntervalOperator, NaivePartitioningOperator, NaiveSortOperator, PartitionSortOperator, ScanOperator, SegmentToRowsAndColumnsOperator, SequenceOperator, SortedInnerJoinOperator, WindowProcessorOperator
public interface Operator
An Operator interface that intends to have implementations that align relatively closely with the Operators that other databases would also tend to be implemented using. While a lot of Operator interfaces tend to use a pull-based orientation, we use a push-based interface. This is to give us good stacktraces. Because of the organization of the go() method, the stack traces thrown out of an Operator will be 1. All of the go() calls from the top-level Operator down to the leaf Operator, this part of the stacktrace gives visibility into what all of the actions that we expect to happen from the operator chain are 2. All of the push() calls up until the exception happens, this part of the stack trace gives us a view of all of the things that have happened to the data up until the exception was thrown.

This "hour glass" structure of the stacktrace is by design. It is very important that implementations of this interface never resort to a fluent style, inheritance or other code structuring that removes the name of the active Operator from the stacktrace. It should always be possible to find ways to avoid code duplication and still keep the Operator's name on the stacktrace.

The other benefit of the go() method is that it fully encapsulates the lifecycle of the underlying resources. This means that it should be possible to use try/finally blocks around calls to go() in order to ensure that resources are properly closed.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Interface
    Description
    static interface 
    Operator.Receiver
     
    static enum 
    Operator.Signal
    This is the return object from a receiver.

  • Method Summary

    Modifier and Type
    Method
    Description
    static void
    go(Operator op, Operator.Receiver receiver)
    Convenience method to run an Operator until completion.
    Closeable
    goOrContinue(Closeable continuationObject, Operator.Receiver receiver)
    This is the primary workhorse method of an Operator.

  • Method Details

    • go

      static void go(Operator op, Operator.Receiver receiver)
      Convenience method to run an Operator until completion. Data will be pushed into the Receiver. This is the primary entry point that users of Operators will call to do their work.
      Parameters:
      op - the operator to run to completion
      receiver - a receiver that will receive data

    • goOrContinue

      @Nullable Closeable goOrContinue(Closeable continuationObject, Operator.Receiver receiver)
      This is the primary workhorse method of an Operator. That said, users of Operators are not expected to use this method and instead are expected to call the static method go(Operator, Receiver).

      Data will be pushed into the Receiver. The Receiver has the option of returning any of the Operator.Signal signals to indicate its degree of readiness for more data to be received.

      If a Receiver returns a Operator.Signal.PAUSE signal, then if there is processing left to do, then it is expected that a non-null "continuation" object will be returned. This allows for flow control to be returned to the caller to, e.g., process another Operator or just exert backpressure. In this case, when the controller wants to resume, it must call this method again and include the continuation object that it received.

      The continuation object is Closeable because it is possible that while processing is paused on one Operator, the processing of another Operator could obviate the need for further processing. In this case, instead of resuming the paused Operation and returning Operator.Signal.STOP on the next push into the Receiver, the code must call Closeable.close() on the continuation object to cancel all further processing and clean up all related resources. If, instead, the continuation object is passed back into a call to goOrContinue, then close() must NOT be called on the continuation object. Said again, the controller must either 1) pass the continuation object back into a call to goOrContinue, OR 2) call close() on the continuation object and NEVER do both.

      Once a reference to a continuation object has been passed back to a goOrContinue method, it should never be reused by the controller. This is to give Operator implementations the ability to decide whether it makes sense to reuse the objects on subsequent calls or create new ones.

      A null return value from this method indicates that processing is complete. The Receiver should have had its Operator.Receiver.completed() method called and any resources associated with processing have already been cleaned up. Additionally, if an exception escapes a call to this method, any resources associated with processing should have been cleaned up.

      For implementators of the interface, if an Operator does not have any resources of its own to clean up, then it is safe to just pass through the continuation object to the caller. However, if there are resources associated with the processing that must be cleaned up, the Operator implementation must wrap the received Closeable in a new Closeable that will close those resources. In this case, when the object comes back to the Operator on a call to goOrContinue, the Operator must unwrap the internal Closeable and pass that back down. In a similar fashion, if there is any state that an Operator requires to be able to resume its processing, then it is expected that the Operator will cast the object back to an instance of the type that it had originally returned.

      Parameters:
      receiver - a receiver that will receive data
      Returns:
      null if processing is complete, non-null if the Receiver returned a Operator.Signal.PAUSE signal