org.apache.gobblin.util.executors

Class IteratorExecutor<T>

java.lang.Object

org.apache.gobblin.util.executors.IteratorExecutor<T>

Type Parameters:

T -

public class IteratorExecutor<T>
extends Object

Executes tasks in an Iterator. Tasks need not be generated until they can be executed.

Constructor Summary

Constructors

Constructor and Description
IteratorExecutor(Iterator<Callable<T>> runnableIterator, int numThreads, ThreadFactory threadFactory)

Method Summary

Modifier and Type	Method and Description
List<Future<T>>	execute() Execute the tasks in the task Iterator.
List<Either<T,ExecutionException>>	executeAndGetResults() Execute the tasks in the task Iterator.
static <T> void	logAndThrowFailures(List<Either<T,ExecutionException>> results, org.slf4j.Logger useLogger, int atMost) Log failures in the output of executeAndGetResults(), and also propagate exception to upper layer.
static <T> void	logFailures(List<Either<T,ExecutionException>> results, org.slf4j.Logger useLogger, int atMost) Log failures in the output of executeAndGetResults().
static <T> boolean	verifyAllSuccessful(List<Either<T,ExecutionException>> results) Utility method that checks whether all tasks succeeded from the output of executeAndGetResults().

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

IteratorExecutor

public IteratorExecutor(Iterator<Callable<T>> runnableIterator,
                        int numThreads,
                        ThreadFactory threadFactory)

Method Detail

execute

public List<Future<T>> execute()
                        throws InterruptedException

Execute the tasks in the task Iterator. Blocks until all tasks are completed.

Note: this method only guarantees tasks have finished, not that they have finished successfully. It is the caller's responsibility to verify the returned futures are successful. Also see executeAndGetResults() for different semantics.

Returns:

a list of completed futures.

Throws:

InterruptedException

executeAndGetResults

public List<Either<T,ExecutionException>> executeAndGetResults()
                                                        throws InterruptedException

Execute the tasks in the task Iterator. Blocks until all tasks are completed. Gets the results of each task, and for each task returns either its result or the thrown ExecutionException.

Returns:

a list containing for each task either its result or the ExecutionException thrown, in the same order as the input Iterator.

Throws:

InterruptedException

verifyAllSuccessful

public static <T> boolean verifyAllSuccessful(List<Either<T,ExecutionException>> results)

Utility method that checks whether all tasks succeeded from the output of executeAndGetResults().

Returns:

true if all tasks succeeded.

logFailures

public static <T> void logFailures(List<Either<T,ExecutionException>> results,
                                   org.slf4j.Logger useLogger,
                                   int atMost)

Log failures in the output of executeAndGetResults().

Parameters:

results - output of executeAndGetResults()

useLogger - logger to log the messages into.

atMost - will log at most this many errors.

logAndThrowFailures

public static <T> void logAndThrowFailures(List<Either<T,ExecutionException>> results,
                                           org.slf4j.Logger useLogger,
                                           int atMost)

Log failures in the output of executeAndGetResults(), and also propagate exception to upper layer.

Parameters:

results - output of executeAndGetResults()

useLogger - logger to log the messages into.

atMost - will log at most this many errors.