Package org.apache.druid.frame.processor.manager

Class ConcurrencyLimitedProcessorManager<T,R>

java.lang.Object

org.apache.druid.frame.processor.manager.ConcurrencyLimitedProcessorManager<T,R>

All Implemented Interfaces:

Closeable, AutoCloseable, ProcessorManager<T,R>

public class ConcurrencyLimitedProcessorManager<T,R>
extends Object
implements ProcessorManager<T,R>

Manager that limits the number of processors that may run concurrently.

Constructor Summary

Constructors

Constructor	Description
ConcurrencyLimitedProcessorManager(ProcessorManager<T,R> delegate, int limit)

Method Summary

Modifier and Type	Method	Description
void	close()	Called when all processors are done, or when one has failed.
com.google.common.util.concurrent.ListenableFuture<Optional<ProcessorAndCallback<T>>>	next()	Returns the next processor that should be run, along with a callback.
R	result()	Called after all procesors are done, prior to ProcessorManager.close(), to retrieve the result of this computation.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.druid.frame.processor.manager.ProcessorManager

withAccumulation

Constructor Details

ConcurrencyLimitedProcessorManager

public ConcurrencyLimitedProcessorManager(ProcessorManager<T,R> delegate,
 int limit)

Method Details

next

public com.google.common.util.concurrent.ListenableFuture<Optional<ProcessorAndCallback<T>>> next()

Description copied from interface: ProcessorManager

Returns the next processor that should be run, along with a callback. The callback is called when the processor completes successfully, along with the result of the processor. If the processor fails, the callback is not called. The callback is called in a thread-safe manner: it will never be called concurrently with another callback, or concurrently with a call to "next" or ProcessorManager.close(). To ensure this, FrameProcessorExecutor.runAllFully(org.apache.druid.frame.processor.manager.ProcessorManager<T, R>, int, org.apache.druid.frame.processor.Bouncer, java.lang.String) synchronizes executions of callbacks for the same processor manager. Therefore, it is important that the callbacks executed quickly. This method returns a future, so it allows for logic where the construction of later processors depends on the results of earlier processors. Returns an empty Optional if there are no more processors to run. Behavior of this method is undefined if called after ProcessorManager.close().

Specified by:

next in interface ProcessorManager<T,R>

result

public R result()

Description copied from interface: ProcessorManager

Called after all procesors are done, prior to ProcessorManager.close(), to retrieve the result of this computation. Behavior of this method is undefined if called after ProcessorManager.close(), or if called prior to ProcessorManager.next() returning an empty Optional, or if called prior to all callbacks from ProcessorManager.next() having been called.

Specified by:

result in interface ProcessorManager<T,R>

close

public void close()

Description copied from interface: ProcessorManager

Called when all processors are done, or when one has failed. This method releases all resources associated with this manager. After calling this method, callers must call no other methods.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in interface ProcessorManager<T,R>