A B C D E F G H I J K L M N O P Q R S T U V W Z _

A

AbandonedTransitionException(Thread) - Constructor for exception org.apache.gobblin.fsm.FiniteStateMachine.AbandonedTransitionException

accept(Path) - Method in class org.apache.gobblin.util.AvroUtils.AvroPathFilter

accept(Path) - Method in class org.apache.gobblin.util.filters.AndPathFilter

accept(Path) - Method in class org.apache.gobblin.util.filters.HiddenFilter

accept(Path) - Method in class org.apache.gobblin.util.filters.RegexPathFilter

accept(Path) - Method in class org.apache.gobblin.util.filters.TarGpgPathFilter

accept(Class<?>) - Method in class org.apache.gobblin.util.WritableShimSerialization

acceptDb(String) - Method in class org.apache.gobblin.data.management.copy.hive.WhitelistBlacklist

acceptsLogFileExtensions(Set<String>) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the set of acceptable log file extensions.

acceptTable(String, String) - Method in class org.apache.gobblin.data.management.copy.hive.WhitelistBlacklist

acquirePermits(long) - Method in class org.apache.gobblin.util.limiter.CountBasedLimiter

acquirePermits(long) - Method in interface org.apache.gobblin.util.limiter.Limiter

Acquire a given number of permits.

acquirePermits(long) - Method in class org.apache.gobblin.util.limiter.MultiLimiter

acquirePermits(long) - Method in class org.apache.gobblin.util.limiter.NonRefillableLimiter

acquirePermits(long) - Method in class org.apache.gobblin.util.limiter.NoopLimiter

acquirePermits(long) - Method in class org.apache.gobblin.util.limiter.PoolBasedLimiter

acquirePermits(long) - Method in class org.apache.gobblin.util.limiter.RateBasedLimiter

acquirePermits(long) - Method in class org.apache.gobblin.util.limiter.TimeBasedLimiter

add(T) - Method in class org.apache.gobblin.util.request_allocation.ConcurrentBoundedPriorityIterable

Offer an element to the container.

add(ResourceRequirement) - Method in class org.apache.gobblin.util.request_allocation.ResourceRequirement

Vector addition of this and other ResourceRequirement.

add(ResourceRequirement, ResourceRequirement, ResourceRequirement) - Static method in class org.apache.gobblin.util.request_allocation.ResourceRequirement

addBrokerKeys(Map<String, String>, Iterable<Map.Entry<S, T>>) - Static method in class org.apache.gobblin.broker.SharedResourcesBrokerFactory

addClusterMapping(String, String) - Method in class org.apache.gobblin.util.ClustersNames

addClusterMapping(URL, String) - Method in class org.apache.gobblin.util.ClustersNames

addExtension(Path, String...) - Static method in class org.apache.gobblin.util.PathUtils

Suffix all extensions to path.

addGobblinSite() - Static method in class org.apache.gobblin.util.HadoopUtils

Add "gobblin-site.xml" as a Configuration resource.

AdditionalCloseableInputStream - Class in org.apache.gobblin.util.io

An extension of FSDataInputStream that takes additional Closeable objects, which will all be closed when InputStream is closed.

AdditionalCloseableInputStream(InputStream, Closeable...) - Constructor for class org.apache.gobblin.util.io.AdditionalCloseableInputStream

addListener(L) - Method in class org.apache.gobblin.util.callbacks.CallbacksDispatcher

addListener(PathAlterationListener) - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserver

Add a file system listener.

addMetadata(String, String) - Method in class org.apache.gobblin.util.event.ContainerHealthCheckFailureEvent

addObserver(PathAlterationObserver) - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserverScheduler

Add a file system observer to this monitor.

addPathAlterationObserver(PathAlterationListener, Optional<PathAlterationObserver>, Path) - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserverScheduler

Create and attach PathAlterationObserverSchedulers for the given root directory and any nested subdirectories under the root directory to the given PathAlterationObserverScheduler.

addSchemaCreationTime(Schema, Schema) - Static method in class org.apache.gobblin.util.AvroUtils

addToHadoopConfiguration(Configuration) - Static method in class org.apache.gobblin.util.WritableShimSerialization

Helper method to add this serializer to an existing Hadoop config.

addTransition(T, T) - Method in class org.apache.gobblin.fsm.FiniteStateMachine.Builder

Add a legal transition to the FiniteStateMachine.

addUniversalEnd(T) - Method in class org.apache.gobblin.fsm.FiniteStateMachine.Builder

Specify that a state is a valid end state for a transition starting from any state.

addWeakListener(L) - Method in class org.apache.gobblin.util.callbacks.CallbacksDispatcher

Only weak references are stored for weak listeners.

ALL - Static variable in enum org.apache.gobblin.util.Id.Parts

AllEqualPrioritizer() - Constructor for class org.apache.gobblin.util.request_allocation.RequestAllocatorConfig.AllEqualPrioritizer

AllocatedRequestsIterator<T extends Request<T>> - Interface in org.apache.gobblin.util.request_allocation

An Iterator over Request that also provides with the total resources used by all consumed entries.

AllocatedRequestsIteratorBase<T extends Request<T>> - Class in org.apache.gobblin.util.request_allocation

A basic implementation of AllocatedRequestsIterator.

AllocatedRequestsIteratorBase(Iterator<AllocatedRequestsIteratorBase.RequestWithResourceRequirement<T>>, ResourcePool) - Constructor for class org.apache.gobblin.util.request_allocation.AllocatedRequestsIteratorBase

AllocatedRequestsIteratorBase.RequestWithResourceRequirement<T> - Class in org.apache.gobblin.util.request_allocation

Stores and element and its ResourceRequirement.

AllocatedRequestsIteratorBase.TExtractor<T> - Class in org.apache.gobblin.util.request_allocation

A Function used to extract the actual Request from a AllocatedRequestsIteratorBase.RequestWithResourceRequirement.

allocateRequests(Iterator<? extends Requestor<T>>, ResourcePool) - Method in class org.apache.gobblin.util.request_allocation.HierarchicalAllocator

allocateRequests(Iterator<? extends Requestor<T>>, ResourcePool) - Method in class org.apache.gobblin.util.request_allocation.PriorityIterableBasedRequestAllocator

allocateRequests(Iterator<? extends Requestor<T>>, ResourcePool) - Method in interface org.apache.gobblin.util.request_allocation.RequestAllocator

Compute the subset of accepted Requests from the input Requestors which fit withing the ResourcePool.

ALLOCATOR_ALIAS_KEY - Static variable in class org.apache.gobblin.util.request_allocation.RequestAllocatorUtils

allowParallelization() - Method in class org.apache.gobblin.util.request_allocation.RequestAllocatorConfig.Builder

allowParallelization(int) - Method in class org.apache.gobblin.util.request_allocation.RequestAllocatorConfig.Builder

AndPathFilter - Class in org.apache.gobblin.util.filters

Combines multiple PathFilters.

AndPathFilter(PathFilter...) - Constructor for class org.apache.gobblin.util.filters.AndPathFilter

append(Path, int, Progressable) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

append(byte[]...) - Method in class org.apache.gobblin.util.guid.Guid

Creates a new Guid which is a unique, replicable representation of the pair (this, byteArrays).

append(Guid...) - Method in class org.apache.gobblin.util.guid.Guid

Creates a new Guid which is a unique, replicable representation of the pair (this, guids).

append(HasGuid...) - Method in class org.apache.gobblin.util.guid.Guid

Creates a new Guid which is a unique, replicable representation of the pair (this, objs).

applicableCharacteristics() - Method in interface org.apache.gobblin.util.filesystem.DataFileVersionStrategy

applicableCharacteristics() - Method in class org.apache.gobblin.util.filesystem.ModTimeDataFileVersionStrategy

ApplicationLauncherUtils - Class in org.apache.gobblin.util

A utility class for the application launcher.

ApplicationLauncherUtils() - Constructor for class org.apache.gobblin.util.ApplicationLauncherUtils

apply(WorkUnitState) - Method in class org.apache.gobblin.util.PublisherUtils.WorkUnitStateSuccess

apply(AllocatedRequestsIteratorBase.RequestWithResourceRequirement<T>) - Method in class org.apache.gobblin.util.request_allocation.AllocatedRequestsIteratorBase.TExtractor

applyCommandLineOptions(CommandLine, T) - Method in class org.apache.gobblin.runtime.cli.PublicMethodsCliObjectFactory

For each method for which the helper created an Option and for which the input CommandLine contains that option, this method will automatically call the method on the input object with the correct arguments.

AUTH_TOKEN_PATH - Static variable in class org.apache.gobblin.util.ProxiedFileSystemUtils

AUTH_TYPE_KEY - Static variable in class org.apache.gobblin.util.ProxiedFileSystemUtils

AutoCloseableLock - Class in org.apache.gobblin.util

An auto-closeable Lock that can be used with try-with-resources.

AutoCloseableLock(Lock) - Constructor for class org.apache.gobblin.util.AutoCloseableLock

AutoResetEvent - Class in org.apache.gobblin.util.concurrent

AutoResetEvent() - Constructor for class org.apache.gobblin.util.concurrent.AutoResetEvent

AutoResetEvent(boolean) - Constructor for class org.apache.gobblin.util.concurrent.AutoResetEvent

AutoReturnableObject<T> - Class in org.apache.gobblin.util

Borrow an object from a GenericObjectPool and returns it automatically on close.

AutoReturnableObject(GenericObjectPool<T>) - Constructor for class org.apache.gobblin.util.AutoReturnableObject

AVRO_TO_HIVE_COLUMN_MAPPING_V_12 - Static variable in class org.apache.gobblin.util.HiveAvroTypeConstants

AvroFlattener - Class in org.apache.gobblin.util

This class provides methods to flatten an Avro Schema to make it more optimal for ORC (Hive does not support predicate pushdown for ORC with nested fields) The behavior of Avro Schema un-nesting is listed below: 1.

AvroFlattener() - Constructor for class org.apache.gobblin.util.AvroFlattener

AvroPathFilter() - Constructor for class org.apache.gobblin.util.AvroUtils.AvroPathFilter

AvroUtils - Class in org.apache.gobblin.util

A Utils class for dealing with Avro objects

AvroUtils() - Constructor for class org.apache.gobblin.util.AvroUtils

AvroUtils.AvroPathFilter - Class in org.apache.gobblin.util

AvroUtils.SchemaEntry - Class in org.apache.gobblin.util

awaitNextRetry() - Method in class org.apache.gobblin.util.ExponentialBackoff

Block until next retry can be executed.

awaitNextRetryIfAvailable() - Method in class org.apache.gobblin.util.ExponentialBackoff

Block until next retry can be executed unless max retries has been reached.

AzkabanTags - Class in org.apache.gobblin.util

Utility class for collecting metadata specific to a Azkaban runtime environment.

AzkabanTags() - Constructor for class org.apache.gobblin.util.AzkabanTags

B

BaseLimiterType - Enum in org.apache.gobblin.util.limiter

An enumeration of types of Limiters supported out-of-the-box.

basicDataSource - Variable in class org.apache.gobblin.util.jdbc.DataSourceProvider

BatchedMeterDecorator - Class in org.apache.gobblin.util.io

A decorator to a Meter that batches updates for performance.

BatchedMeterDecorator(Meter, int) - Constructor for class org.apache.gobblin.util.io.BatchedMeterDecorator

bindSharedResourceAtScope(SharedResourceFactory<T, K, S>, K, S, T) - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl

BLACKLIST - Static variable in class org.apache.gobblin.data.management.copy.hive.WhitelistBlacklist

BROKER_CONF_FILE_KEY - Static variable in class org.apache.gobblin.broker.SharedResourcesBrokerFactory

BrokerConfigurationKeyGenerator - Class in org.apache.gobblin.broker

Generates key strings that the default SharedResourcesBroker can understand.

BrokerConfigurationKeyGenerator() - Constructor for class org.apache.gobblin.broker.BrokerConfigurationKeyGenerator

BruteForceAllocator<T extends Request<T>> - Class in org.apache.gobblin.util.request_allocation

A RequestAllocator that simply adds every work unit to a ConcurrentBoundedPriorityIterable, then returns the iterator.

BruteForceAllocator(RequestAllocatorConfig<T>) - Constructor for class org.apache.gobblin.util.request_allocation.BruteForceAllocator

BruteForceAllocator.Factory - Class in org.apache.gobblin.util.request_allocation

build() - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl.SubscopedBrokerBuilder

build(T) - Method in class org.apache.gobblin.fsm.FiniteStateMachine.Builder

Build a FiniteStateMachine starting at the given initial state.

build() - Method in class org.apache.gobblin.util.jdbc.DataSourceBuilder

build() - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Build a new LogCopier instance.

build() - Method in class org.apache.gobblin.util.request_allocation.RequestAllocatorConfig.Builder

build() - Method in class org.apache.gobblin.util.request_allocation.ResourceRequirement.Builder

build() - Method in class org.apache.gobblin.util.test.StdoutWriter.Builder

Builder() - Constructor for class org.apache.gobblin.fsm.FiniteStateMachine.Builder

builder() - Static method in class org.apache.gobblin.util.jdbc.DataSourceBuilder

Builder() - Constructor for class org.apache.gobblin.util.logs.LogCopier.Builder

builder(ResourceEstimator<T>) - Static method in class org.apache.gobblin.util.request_allocation.RequestAllocatorConfig

Builder(ResourceEstimator<T>) - Constructor for class org.apache.gobblin.util.request_allocation.RequestAllocatorConfig.Builder

Builder(ResourcePool) - Constructor for class org.apache.gobblin.util.request_allocation.ResourceRequirement.Builder

Builder() - Constructor for class org.apache.gobblin.util.test.StdoutWriter.Builder

buildLimiter(Config) - Method in class org.apache.gobblin.util.limiter.CountBasedLimiter.Factory

buildLimiter(Config) - Method in interface org.apache.gobblin.util.limiter.LimiterFactory

buildLimiter(Config) - Method in class org.apache.gobblin.util.limiter.PoolBasedLimiter.Factory

buildLimiter(Config) - Method in class org.apache.gobblin.util.limiter.RateBasedLimiter.Factory

buildLimiter(Config) - Method in class org.apache.gobblin.util.limiter.TimeBasedLimiter.Factory

buildObject(CommandLine) - Method in interface org.apache.gobblin.runtime.cli.CliObjectFactory

Build an instance of T from the parsed CommandLine.

buildObject(String[], int, boolean, String) - Method in interface org.apache.gobblin.runtime.cli.CliObjectFactory

Build an instance of T from the input arguments.

buildObject(CommandLine) - Method in class org.apache.gobblin.runtime.cli.PublicMethodsCliObjectFactory

buildObject(String[], int, boolean, String) - Method in class org.apache.gobblin.runtime.cli.PublicMethodsCliObjectFactory

byteBufferToOutputStream(ByteBuffer, OutputStream) - Static method in class org.apache.gobblin.util.io.StreamUtils

Reads the full contents of a ByteBuffer and writes them to an OutputStream.

bytesWritten() - Method in class org.apache.gobblin.util.test.StdoutWriter

C

call() - Method in class org.apache.gobblin.util.callbacks.CallbacksDispatcher.CallbackCallable

call() - Method in class org.apache.gobblin.util.executors.MDCPropagatingCallable

Callback<L,R> - Class in org.apache.gobblin.util.callbacks

A helper class to wrap a function and provide a name for logging purposes

Callback() - Constructor for class org.apache.gobblin.util.callbacks.Callback

CallbackCallable() - Constructor for class org.apache.gobblin.util.callbacks.CallbacksDispatcher.CallbackCallable

CallbackResult<R> - Class in org.apache.gobblin.util.callbacks

A helper class to simplify the access the result of a callback (if any)

CallbackResult() - Constructor for class org.apache.gobblin.util.callbacks.CallbackResult

CallbackResults() - Constructor for class org.apache.gobblin.util.callbacks.CallbacksDispatcher.CallbackResults

CallbacksDispatcher<L> - Class in org.apache.gobblin.util.callbacks

A helper to dispatch callbacks to a set of listeners.

CallbacksDispatcher(Optional<ExecutorService>, Optional<Logger>) - Constructor for class org.apache.gobblin.util.callbacks.CallbacksDispatcher

Constructor

CallbacksDispatcher() - Constructor for class org.apache.gobblin.util.callbacks.CallbacksDispatcher

CallbacksDispatcher(Logger) - Constructor for class org.apache.gobblin.util.callbacks.CallbacksDispatcher

CallbacksDispatcher(ExecutorService, Logger) - Constructor for class org.apache.gobblin.util.callbacks.CallbacksDispatcher

CallbacksDispatcher.CallbackCallable<R> - Class in org.apache.gobblin.util.callbacks

CallbacksDispatcher.CallbackResults<L,R> - Class in org.apache.gobblin.util.callbacks

cancel(T) - Method in class org.apache.gobblin.util.concurrent.TaskScheduler

Attempts to cancel the specified ScheduledTask.

canProxyAs(String, String, Path) - Static method in class org.apache.gobblin.util.ProxiedFileSystemUtils

Returns true if superUserName can proxy as userNameToProxyAs using the specified superUserKeytabLocation, false otherwise.

changeEndState(T) - Method in class org.apache.gobblin.fsm.FiniteStateMachine.Transition

Change the end state of the transition.

checkAndNotify() - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserver

Check whether the file and its chlidren have been created, modified or deleted.

checkReaderWriterCompatibility(Schema, Schema, boolean) - Static method in class org.apache.gobblin.util.AvroUtils

Validates that the provided reader schema can be used to decode avro data written with the provided writer schema.

cleanJobStagingData(State, Logger) - Static method in class org.apache.gobblin.util.JobLauncherUtils

Cleanup staging data of all tasks of a job.

cleanStagingData(List<? extends State>, Logger) - Static method in class org.apache.gobblin.util.JobLauncherUtils

Cleanup the staging data for a list of Gobblin tasks.

cleanTaskStagingData(State, Logger) - Static method in class org.apache.gobblin.util.JobLauncherUtils

Cleanup staging data of a Gobblin task.

cleanTaskStagingData(State, Logger, Closer, Map<String, ParallelRunner>) - Static method in class org.apache.gobblin.util.JobLauncherUtils

Cleanup staging data of a Gobblin task using a ParallelRunner.

cleanup() - Method in class org.apache.gobblin.util.test.StdoutWriter

cleanUpOldJobData(State, Logger, boolean, boolean) - Static method in class org.apache.gobblin.util.JobLauncherUtils

CliApplication - Interface in org.apache.gobblin.runtime.cli

An application that can be called by GobblinCli.

CliObjectFactory<T> - Interface in org.apache.gobblin.runtime.cli

A factory used to create an object from CLI arguments backed by commons-cli library.

CliObjectFactory.HelpArgumentFound - Exception in org.apache.gobblin.runtime.cli

Thrown if help argument (-h) was found in the app arguments.

CliObjectOption - Annotation Type in org.apache.gobblin.runtime.cli

Specify additional information to use when building a CLI option from a method.

CliObjectSupport - Annotation Type in org.apache.gobblin.runtime.cli

Specifies that the annotated constructor should be used to instantiate the object for running from CLI.

CLIPasswordEncryptor - Class in org.apache.gobblin.util

A command line tool for encrypting password.

CLIPasswordEncryptor() - Constructor for class org.apache.gobblin.util.CLIPasswordEncryptor

cloneAtCurrentState() - Method in class org.apache.gobblin.fsm.FiniteStateMachine

cloneAtInitialState() - Method in class org.apache.gobblin.fsm.FiniteStateMachine

close() - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl

close() - Method in class org.apache.gobblin.fsm.FiniteStateMachine.Transition

Close the current transition moving the FiniteStateMachine to the end state and releasing all locks.

close() - Method in class org.apache.gobblin.util.AutoCloseableLock

close() - Method in class org.apache.gobblin.util.AutoReturnableObject

Return the borrowed object to the pool.

close() - Method in class org.apache.gobblin.util.callbacks.CallbacksDispatcher

close() - Method in class org.apache.gobblin.util.concurrent.TaskScheduler

Closes this TaskScheduler, ensuring that new tasks cannot be created and cancelling existing tasks.

close() - Method in class org.apache.gobblin.util.filesystem.FileSystemInstrumentation

close() - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

close() - Method in class org.apache.gobblin.util.HiveJdbcConnector

close() - Method in class org.apache.gobblin.util.io.AdditionalCloseableInputStream

close() - Method in class org.apache.gobblin.util.io.CloseableHttpConn

close() - Method in class org.apache.gobblin.util.io.SeekableFSInputStream

close() - Method in class org.apache.gobblin.util.NoopCloseable

close() - Method in class org.apache.gobblin.util.ParallelRunner

close() - Method in class org.apache.gobblin.util.test.FastSequentialSource.FastSequentialExtractor

close() - Method in class org.apache.gobblin.util.test.HelloWorldSource.ExtractorImpl

close() - Method in class org.apache.gobblin.util.test.StdoutWriter

close() - Method in class org.apache.gobblin.util.test.StressTestingSource.ExtractorImpl

close() - Method in class org.apache.gobblin.util.test.TestingSource.Extract

CloseableHttpConn - Class in org.apache.gobblin.util.io

Wraps a HttpURLConnection into a Closeable object.

CloseableHttpConn() - Constructor for class org.apache.gobblin.util.io.CloseableHttpConn

closeChannelsOnComplete() - Method in class org.apache.gobblin.util.io.StreamCopier

Close the input and output Channels after copy, whether the copy was successful or not.

closed - Variable in class org.apache.gobblin.util.filesystem.FileSystemInstrumentation

closeWithoutCallbacks() - Method in class org.apache.gobblin.fsm.FiniteStateMachine.Transition

Close the current transition moving the FiniteStateMachine to the end state and releasing all locks without calling any callbacks.

CLUSTER_IDENTIFIER_TAG_NAME - Static variable in class org.apache.gobblin.util.ClusterNameTags

ClusterNameTags - Class in org.apache.gobblin.util

Utility class for collecting metadata specific to the current Hadoop cluster.

ClusterNameTags() - Constructor for class org.apache.gobblin.util.ClusterNameTags

ClustersNames - Class in org.apache.gobblin.util

Allows conversion of URLs identifying a Hadoop cluster (e.g.

ClustersNames() - Constructor for class org.apache.gobblin.util.ClustersNames

combine(Guid...) - Static method in class org.apache.gobblin.util.guid.Guid

Combine multiple Guids into a single Guid.

combinePaths(String...) - Static method in class org.apache.gobblin.util.PathUtils

combineProperties(Properties...) - Static method in class org.apache.gobblin.util.PropertiesUtils

Combine a variable number of Properties into a single Properties.

combineSysAndJobProperties(Properties, Properties) - Static method in class org.apache.gobblin.util.JobConfigurationUtils

Get a new Properties instance by combining a given system configuration Properties object (first) and a job configuration Properties object (second).

commit() - Method in class org.apache.gobblin.util.test.StdoutWriter

CompactionRecordCountProvider - Class in org.apache.gobblin.util.recordcount

Implementation of RecordCountProvider, which provides record count from file path.

CompactionRecordCountProvider() - Constructor for class org.apache.gobblin.util.recordcount.CompactionRecordCountProvider

compare(T, T) - Method in class org.apache.gobblin.util.request_allocation.RequestAllocatorConfig.AllEqualPrioritizer

compare(T, T) - Method in class org.apache.gobblin.util.request_allocation.SimpleHierarchicalPrioritizer

compareRequestors(Requestor<T>, Requestor<T>) - Method in interface org.apache.gobblin.util.request_allocation.HierarchicalPrioritizer

compareRequestors(Requestor<T>, Requestor<T>) - Method in class org.apache.gobblin.util.request_allocation.SimpleHierarchicalPrioritizer

compareWithoutSchemeAndAuthority(Path, Path) - Static method in class org.apache.gobblin.util.PathUtils

Compare two path without shedme and authority (the prefix)

COMPUTE_TIME_MICRO_KEY - Static variable in class org.apache.gobblin.util.test.StressTestingSource

ConcurrentBoundedPriorityIterable<T> - Class in org.apache.gobblin.util.request_allocation

A concurrent bounded priority Iterable.

ConcurrentBoundedPriorityIterable(Comparator<? super T>, ResourceEstimator<T>, String, ResourcePool) - Constructor for class org.apache.gobblin.util.request_allocation.ConcurrentBoundedPriorityIterable

CONFIG_NAMESPACE - Static variable in class org.apache.gobblin.util.test.HelloWorldSource

CONFIG_NAMESPACE - Static variable in class org.apache.gobblin.util.test.StressTestingSource

configToProperties(Config) - Static method in class org.apache.gobblin.util.ConfigUtils

Convert a given Config instance to a Properties instance.

configToProperties(Config, Optional<String>) - Static method in class org.apache.gobblin.util.ConfigUtils

Convert a given Config instance to a Properties instance.

configToProperties(Config, String) - Static method in class org.apache.gobblin.util.ConfigUtils

Convert a given Config instance to a Properties instance.

configToState(Config) - Static method in class org.apache.gobblin.util.ConfigUtils

Convert a given Config to a State instance.

configure() - Method in class org.apache.gobblin.util.jdbc.DataSourceModule

ConfigUtils - Class in org.apache.gobblin.util

Utility class for dealing with Config objects.

ConfigUtils(FileUtils) - Constructor for class org.apache.gobblin.util.ConfigUtils

CONN_DRIVER - Static variable in class org.apache.gobblin.util.jdbc.DataSourceProvider

CONN_URL - Static variable in class org.apache.gobblin.util.jdbc.DataSourceProvider

constructFileName(String, long) - Static method in class org.apache.gobblin.util.recordcount.CompactionRecordCountProvider

Deprecated.

discouraged since default behavior is not obvious from API itself.

constructFileName(String, String, long) - Static method in class org.apache.gobblin.util.recordcount.CompactionRecordCountProvider

Construct the file name as {filenamePrefix}{recordCount}.{SystemCurrentTimeInMills}.{RandomInteger}{extension}.

constructFilePath(String, long) - Static method in class org.apache.gobblin.util.recordcount.IngestionRecordCountProvider

Construct a new file path by appending record count to the filename of the given file path, separated by SEPARATOR.

constructLateFilePath(String, FileSystem, Path) - Method in class org.apache.gobblin.util.recordcount.LateFileRecordCountProvider

Construct filename for a late file.

constructObject(CommandLine) - Method in class org.apache.gobblin.runtime.cli.ConstructorAndPublicMethodsCliObjectFactory

constructObject(CommandLine) - Method in class org.apache.gobblin.runtime.cli.PublicMethodsCliObjectFactory

ConstructorAndPublicMethodsCliObjectFactory<T> - Class in org.apache.gobblin.runtime.cli

A helper class for automatically inferring Options from the constructor and public methods in a class.

ConstructorAndPublicMethodsCliObjectFactory(Class<? extends T>) - Constructor for class org.apache.gobblin.runtime.cli.ConstructorAndPublicMethodsCliObjectFactory

CONTAINER_HEALTH_CHECK_EVENT_BUS_NAME - Static variable in class org.apache.gobblin.util.event.ContainerHealthCheckFailureEvent

ContainerHealthCheckFailureEvent - Class in org.apache.gobblin.util.event

ContainerHealthCheckFailureEvent(Config, String) - Constructor for class org.apache.gobblin.util.event.ContainerHealthCheckFailureEvent

contractPool(ResourceRequirement) - Method in class org.apache.gobblin.util.request_allocation.ResourcePool

convertFieldToSchemaWithProps(Map<String, JsonNode>, Schema) - Static method in class org.apache.gobblin.util.AvroUtils

Generate a Schema object from Schema.Field with Field's properties carried over to the new object.

convertPath(Path, String, RecordCountProvider) - Method in class org.apache.gobblin.util.recordcount.CompactionRecordCountProvider

This method currently supports converting the given Path from IngestionRecordCountProvider.

convertRecordSchema(GenericRecord, Schema) - Static method in class org.apache.gobblin.util.AvroUtils

Change the schema of an Avro record.

convertStream(InputStream) - Static method in class org.apache.gobblin.util.io.StreamUtils

Convert an instance of InputStream to a FSDataInputStream that is Seekable and PositionedReadable.

copy() - Method in class org.apache.gobblin.util.io.StreamCopier

Execute the copy of bytes from the input to the output stream.

copy(InputStream, OutputStream) - Static method in class org.apache.gobblin.util.io.StreamUtils

Copies an InputStream to and OutputStream using Channels.

copy(ReadableByteChannel, WritableByteChannel) - Static method in class org.apache.gobblin.util.io.StreamUtils

Copies a ReadableByteChannel to a WritableByteChannel.

copyFile(FileSystem, Path, FileSystem, Path, Path, boolean, Configuration) - Static method in class org.apache.gobblin.util.HadoopUtils

Copies a src Path from a srcFs FileSystem to a dst Path on a dstFs FileSystem.

copyFile(FileSystem, Path, FileSystem, Path, boolean, Configuration) - Static method in class org.apache.gobblin.util.HadoopUtils

Copy a file from a srcFs FileSystem to a dstFs FileSystem.

copyPath(FileSystem, Path, FileSystem, Path, Configuration) - Static method in class org.apache.gobblin.util.HadoopUtils

Copies data from a src Path to a dst Path.

copyPath(FileSystem, Path, FileSystem, Path, boolean, Configuration) - Static method in class org.apache.gobblin.util.HadoopUtils

Copies data from a src Path to a dst Path.

COUNT_KEY - Static variable in class org.apache.gobblin.util.limiter.CountBasedLimiter.Factory

CountBasedLimiter - Class in org.apache.gobblin.util.limiter

An implementation of Limiter that limits the number of permits allowed to be issued.

CountBasedLimiter(long) - Constructor for class org.apache.gobblin.util.limiter.CountBasedLimiter

CountBasedLimiter.Factory - Class in org.apache.gobblin.util.limiter

create(Path, FsPermission, boolean, int, short, long, Progressable) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

create(String) - Static method in class org.apache.gobblin.util.Id.Job

create(String, long) - Static method in class org.apache.gobblin.util.Id.Job

create(String) - Static method in class org.apache.gobblin.util.Id.MultiTask

create(String, int) - Static method in class org.apache.gobblin.util.Id.MultiTask

create(String) - Static method in class org.apache.gobblin.util.Id.Task

create(String, int) - Static method in class org.apache.gobblin.util.Id.Task

create(Config) - Method in interface org.apache.gobblin.util.request_allocation.ResourceEstimator.Factory

createCancelled() - Static method in class org.apache.gobblin.util.callbacks.CallbackResult

createDataFileVersionStrategy(FileSystem, Config) - Method in interface org.apache.gobblin.util.filesystem.DataFileVersionStrategy.DataFileVersionFactory

Build a DataFileVersionStrategy with the input configuration.

createDataFileVersionStrategy(FileSystem, Config) - Method in class org.apache.gobblin.util.filesystem.ModTimeDataFileVersionStrategy.Factory

createDefaultTopLevelBroker(Config, ScopeInstance<S>) - Static method in class org.apache.gobblin.broker.SharedResourcesBrokerFactory

Create a root SharedResourcesBroker.

createExtractToWorkUnitStateMap(Collection<? extends WorkUnitState>) - Static method in class org.apache.gobblin.util.PublisherUtils

Creates a Multimap that maps Extract to their corresponds WorkUnitStates.

createFailed(Throwable) - Static method in class org.apache.gobblin.util.callbacks.CallbackResult

createFromFuture(Future<R>) - Static method in class org.apache.gobblin.util.callbacks.CallbackResult

createRequestAllocator(RequestAllocatorConfig<T>) - Method in class org.apache.gobblin.util.request_allocation.BruteForceAllocator.Factory

createRequestAllocator(RequestAllocatorConfig<T>) - Method in class org.apache.gobblin.util.request_allocation.GreedyAllocator.Factory

createRequestAllocator(RequestAllocatorConfig<T>) - Method in class org.apache.gobblin.util.request_allocation.HierarchicalAllocator.Factory

createRequestAllocator(RequestAllocatorConfig<T>) - Method in class org.apache.gobblin.util.request_allocation.PreOrderAllocator.Factory

createRequestAllocator(RequestAllocatorConfig<T>) - Method in interface org.apache.gobblin.util.request_allocation.RequestAllocator.Factory

Create a RequestAllocator with the input prioritizer and ResourceEstimator.

createResource(SharedResourcesBroker<S>, ScopedConfigView<S, EventBusKey>) - Method in class org.apache.gobblin.util.eventbus.EventBusFactory

createResource(SharedResourcesBroker<S>, ScopedConfigView<S, FileSystemKey>) - Method in class org.apache.gobblin.util.filesystem.FileSystemFactory

createResource(SharedResourcesBroker<S>, ScopedConfigView<S, EmptyKey>) - Method in class org.apache.gobblin.util.io.StreamThrottler.Factory

createResource(SharedResourcesBroker<S>, ScopedConfigView<S, SharedLimiterKey>) - Method in class org.apache.gobblin.util.limiter.broker.SharedLimiterFactory

createSocket(Socket, String, int, boolean) - Method in class org.apache.gobblin.util.TrustManagerSecureSocketFactory

createSocket(String, int) - Method in class org.apache.gobblin.util.TrustManagerSecureSocketFactory

createSocket(String, int, InetAddress, int) - Method in class org.apache.gobblin.util.TrustManagerSecureSocketFactory

createSocket(InetAddress, int) - Method in class org.apache.gobblin.util.TrustManagerSecureSocketFactory

createSocket(InetAddress, int, InetAddress, int) - Method in class org.apache.gobblin.util.TrustManagerSecureSocketFactory

createSocket(Socket, String, int, boolean) - Method in class org.apache.gobblin.util.TrustManagerSocketFactory

createSocket(String, int) - Method in class org.apache.gobblin.util.TrustManagerSocketFactory

createSocket(InetAddress, int) - Method in class org.apache.gobblin.util.TrustManagerSocketFactory

createSocket(String, int, InetAddress, int) - Method in class org.apache.gobblin.util.TrustManagerSocketFactory

createSocket(InetAddress, int, InetAddress, int) - Method in class org.apache.gobblin.util.TrustManagerSocketFactory

createSuccessful(R) - Static method in class org.apache.gobblin.util.callbacks.CallbackResult

cryptoKeyLocation(String) - Method in class org.apache.gobblin.util.jdbc.DataSourceBuilder

D

DATA_FILE_VERSION_STRATEGY_KEY - Static variable in interface org.apache.gobblin.util.filesystem.DataFileVersionStrategy

DataFileVersionStrategy<T extends Comparable<T> & Serializable> - Interface in org.apache.gobblin.util.filesystem

An interface to set and get "versions" to data files.

DataFileVersionStrategy.Characteristic - Enum in org.apache.gobblin.util.filesystem

Characteristics a DataFileVersionStrategy may have.

DataFileVersionStrategy.DataFileVersionFactory<T extends Comparable<T> & Serializable> - Interface in org.apache.gobblin.util.filesystem

A Factory for DataFileVersionStrategys.

DATASET - Static variable in class org.apache.gobblin.util.dataset.DatasetUtils

DATASET_SPECIFIC_PROPS - Static variable in class org.apache.gobblin.util.dataset.DatasetUtils

A configuration key that allows a user to specify config parameters on a dataset specific level.

DatasetFilterUtils - Class in org.apache.gobblin.util

A utility class for filtering datasets through blacklist and whitelist.

DatasetFilterUtils() - Constructor for class org.apache.gobblin.util.DatasetFilterUtils

DatasetUtils - Class in org.apache.gobblin.util.dataset

DataSourceBuilder - Class in org.apache.gobblin.util.jdbc

DataSourceBuilder() - Constructor for class org.apache.gobblin.util.jdbc.DataSourceBuilder

DataSourceModule - Class in org.apache.gobblin.util.jdbc

A Guice module defining the dependencies used by the jdbc data source.

DataSourceModule(Properties) - Constructor for class org.apache.gobblin.util.jdbc.DataSourceModule

DataSourceProvider - Class in org.apache.gobblin.util.jdbc

A provider class for DataSources.

DataSourceProvider(Properties) - Constructor for class org.apache.gobblin.util.jdbc.DataSourceProvider

DataSourceProvider() - Constructor for class org.apache.gobblin.util.jdbc.DataSourceProvider

DATE - Static variable in class org.apache.gobblin.util.HiveAvroTypeConstants

DatePartitionType - Enum in org.apache.gobblin.util

Temporal granularity types for writing (partitioner.TimeBasedWriterPartitioner) and reading (DatePartitionedAvroFileSource) date partitioned data.

DECIMAL - Static variable in class org.apache.gobblin.util.HiveAvroTypeConstants

Following are supported Avro logical types where they would be mapped to corresponding Hive types: Decimal -> "decimal" Date -> "date" TIME_MILLIS = "timestamp"

decorateRecord(GenericRecord, Map<String, Object>, Schema) - Static method in class org.apache.gobblin.util.AvroUtils

Decorate a GenericRecord with additional fields and make it conform to an extended Schema It is the caller's responsibility to ensure that the outputSchema is the merge of the inputRecord's schema and the additional fields.

decorateRecordSchema(Schema, List<Schema.Field>) - Static method in class org.apache.gobblin.util.AvroUtils

Decorate the Schema for a record with additional Schema.Fields.

deepCopySchemaFields(Schema) - Static method in class org.apache.gobblin.util.AvroUtils

deepestNonGlobPath(Path) - Static method in class org.apache.gobblin.util.PathUtils

Finds the deepest ancestor of input that is not a glob.

DEFAULT_BROKER_CONF_FILE - Static variable in class org.apache.gobblin.broker.SharedResourcesBrokerFactory

DEFAULT_BUFFER_SIZE - Static variable in class org.apache.gobblin.util.io.StreamCopier

DEFAULT_CFG - Static variable in class org.apache.gobblin.util.service.StandardServiceConfig

DEFAULT_COMPUTE_TIME_MICRO - Static variable in class org.apache.gobblin.util.test.StressTestingSource

DEFAULT_CONN_DRIVER - Static variable in class org.apache.gobblin.util.jdbc.DataSourceProvider

DEFAULT_DATA_FILE_VERSION_STRATEGY - Static variable in interface org.apache.gobblin.util.filesystem.DataFileVersionStrategy

DEFAULT_DIMENSION_TOLERANCE - Static variable in class org.apache.gobblin.util.request_allocation.ResourcePool

DEFAULT_HIVESERVER_VERSION - Static variable in class org.apache.gobblin.util.HiveJdbcConnector

DEFAULT_HOCON_PULL_FILE_EXTENSIONS - Static variable in class org.apache.gobblin.util.PullFileLoader

DEFAULT_JAVA_PROPS_PULL_FILE_EXTENSIONS - Static variable in class org.apache.gobblin.util.PullFileLoader

DEFAULT_MEM_ALLOC_BYTES - Static variable in class org.apache.gobblin.util.test.StressTestingSource

DEFAULT_NUM_HELLOS - Static variable in class org.apache.gobblin.util.test.HelloWorldSource

DEFAULT_NUM_RECORDS - Static variable in class org.apache.gobblin.util.test.StressTestingSource

DEFAULT_NUM_WORK_UNITS - Static variable in class org.apache.gobblin.util.test.StressTestingSource

DEFAULT_PARALLEL_RUNNER_THREADS - Static variable in class org.apache.gobblin.util.ParallelRunner

DEFAULT_PROPERTY_DELIMITER_PARSING_ENABLED_KEY - Static variable in class org.apache.gobblin.util.PullFileLoader

DEFAULT_RUN_DURATION - Static variable in class org.apache.gobblin.util.test.StressTestingSource

DEFAULT_SLEEP_TIME - Static variable in class org.apache.gobblin.util.test.StressTestingSource

DefaultLimiterFactory - Class in org.apache.gobblin.util.limiter

A default factory class for Limiters.

DefaultLimiterFactory() - Constructor for class org.apache.gobblin.util.limiter.DefaultLimiterFactory

defaultThreadFactory() - Static method in class org.apache.gobblin.util.ExecutorsUtils

Get a default ThreadFactory.

delegate() - Method in class org.apache.gobblin.util.executors.MDCPropagatingExecutorService

delegate() - Method in class org.apache.gobblin.util.executors.MDCPropagatingScheduledExecutorService

delete(Path) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

delete(Path, boolean) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

deleteDirectories(FileSystem, List<String>, boolean, boolean) - Static method in class org.apache.gobblin.util.HadoopUtils

Calls deletePath() on each directory in the given list of directories to delete.

deleteEmptyParentDirectories(FileSystem, Path, Path) - Static method in class org.apache.gobblin.util.PathUtils

Deletes empty directories starting with startPath and all ancestors up to but not including limitPath.

deleteIfExists(FileSystem, Path, boolean) - Static method in class org.apache.gobblin.util.HadoopUtils

A wrapper around FileSystem.delete(Path, boolean) that only deletes a given Path if it is present on the given FileSystem.

deletePath(FileSystem, Path, boolean) - Static method in class org.apache.gobblin.util.HadoopUtils

A wrapper around FileSystem.delete(Path, boolean) which throws IOException if the given Path exists, and FileSystem.delete(Path, boolean) returns False.

deletePath(Path, boolean) - Method in class org.apache.gobblin.util.ParallelRunner

Delete a Path.

deletePathAndEmptyAncestors(FileSystem, Path, boolean) - Static method in class org.apache.gobblin.util.HadoopUtils

deletePathByRegex(FileSystem, Path, String) - Static method in class org.apache.gobblin.util.HadoopUtils

Delete files according to the regular expression provided

DeprecationUtils - Class in org.apache.gobblin.util.deprecation

Utilities to handle deprecations in Gobblin.

DeprecationUtils() - Constructor for class org.apache.gobblin.util.deprecation.DeprecationUtils

desanitizeKey(String) - Static method in class org.apache.gobblin.util.ConfigUtils

deserialize(String) - Static method in class org.apache.gobblin.util.guid.Guid

Reverse of Guid.toString().

deserialize(String) - Static method in class org.apache.gobblin.util.PropertiesUtils

deserialize(String, Class<T>) - Static method in class org.apache.gobblin.util.SerializationUtils

Deserialize a String obtained via SerializationUtils.serialize(Serializable) into an object, using BaseEncoding.base64().

deserialize(String, Class<T>, BaseEncoding) - Static method in class org.apache.gobblin.util.SerializationUtils

Deserialize a String obtained via SerializationUtils.serialize(Serializable) into an object, using the given BaseEncoding, which must be the same BaseEncoding used to serialize the object.

deserializeFromBytes(byte[], Class<T>) - Static method in class org.apache.gobblin.util.SerializationUtils

Deserialize a String obtained via SerializationUtils.serializeIntoBytes(Serializable) into an object.

deserializeFromFile(T, Path) - Method in class org.apache.gobblin.util.ParallelRunner

Deserialize a State object from a file.

deserializeFromSequenceFile(Class<? extends Writable>, Class<T>, Path, Collection<T>, boolean) - Method in class org.apache.gobblin.util.ParallelRunner

Deserialize a list of State objects from a Hadoop SequenceFile.

deserializeFromString(Class<? extends Writable>, String) - Static method in class org.apache.gobblin.util.HadoopUtils

Deserialize a Writable object from a string.

deserializeFromString(Class<? extends Writable>, String, Configuration) - Static method in class org.apache.gobblin.util.HadoopUtils

Deserialize a Writable object from a string.

deserializeFsPermission(State, String, FsPermission) - Static method in class org.apache.gobblin.util.HadoopUtils

Get FsPermission from a State object.

deserializeState(FileSystem, Path, T) - Static method in class org.apache.gobblin.util.SerializationUtils

Deserialize/read a State instance from a file.

deserializeStateFromInputStream(InputStream, T) - Static method in class org.apache.gobblin.util.SerializationUtils

Deserialize/read a State instance from a file.

deserializeWriterDirPermissions(State, int, int) - Static method in class org.apache.gobblin.util.HadoopUtils

Deserializes a FsPermissions object that should be used when a DataWriter is creating directories.

deserializeWriterFilePermissions(State, int, int) - Static method in class org.apache.gobblin.util.HadoopUtils

Deserializes a FsPermissions object that should be used when a DataWriter is writing a file.

destroy() - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserver

Final processing.

downloadJar(String, String, String, boolean) - Static method in class org.apache.gobblin.util.DownloadUtils

Download jar through Grape given an org, module and version It is assumed that an ivy settings file exists on the classpath

downloadJar(Map<String, Object>) - Static method in class org.apache.gobblin.util.DownloadUtils

DownloadUtils - Class in org.apache.gobblin.util

Utility class for downloads using grape

driver(String) - Method in class org.apache.gobblin.util.jdbc.DataSourceBuilder

dropRecursiveFields(Schema) - Static method in class org.apache.gobblin.util.AvroUtils

Drop recursive fields from a Schema.

E

Either<S,T> - Class in org.apache.gobblin.util

Represents a value of one of two possible types.

Either() - Constructor for class org.apache.gobblin.util.Either

Either.Left<S,T> - Class in org.apache.gobblin.util

An instance of Either with value of type S.

Either.Right<S,T> - Class in org.apache.gobblin.util

An instance of Either with value of type T.

EmailUtils - Class in org.apache.gobblin.util

A utility class for sending emails.

EmailUtils() - Constructor for class org.apache.gobblin.util.EmailUtils

emptyIfNotPresent(Config, String) - Static method in class org.apache.gobblin.util.ConfigUtils

Return string value at path if config has path.

EmptyInputStream - Class in org.apache.gobblin.util.io

An InputStream with empty content

EmptyIterable<T> - Class in org.apache.gobblin.util

A type of Iterables corresponding to empty Collections.

EmptyIterable() - Constructor for class org.apache.gobblin.util.EmptyIterable

EmptyKey - Class in org.apache.gobblin.broker

A dummy SharedResourceKey.

EmptyKey() - Constructor for class org.apache.gobblin.broker.EmptyKey

equals(Object) - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl

equals(Object) - Method in class org.apache.gobblin.util.filesystem.FileStatusEntry

Compare if this object is equal to another object

equals(Object) - Method in class org.apache.gobblin.util.filesystem.FileSystemKey

equals(Object) - Method in class org.apache.gobblin.util.filesystem.FileSystemLimiterKey

equals(Object) - Method in class org.apache.gobblin.util.http.HttpLimiterKey

equals(Object) - Method in class org.apache.gobblin.util.ImmutableProperties

equals(Object) - Method in class org.apache.gobblin.util.io.StreamCopierSharedLimiterKey

errorState(T) - Method in class org.apache.gobblin.fsm.FiniteStateMachine.Builder

Specify the error state to which this machine can transition if nothing else is possible.

estimateRequirement(T, ResourcePool) - Method in interface org.apache.gobblin.util.request_allocation.ResourceEstimator

EventBusFactory<S extends ScopeType<S>> - Class in org.apache.gobblin.util.eventbus

A SharedResourceFactory for creating EventBus instances.

EventBusFactory() - Constructor for class org.apache.gobblin.util.eventbus.EventBusFactory

EventBusKey - Class in org.apache.gobblin.util.eventbus

EventBusKey(String) - Constructor for class org.apache.gobblin.util.eventbus.EventBusKey

exceedsHardBound(ResourceRequirement, boolean) - Method in class org.apache.gobblin.util.request_allocation.ResourcePool

exceedsSoftBound(ResourceRequirement, boolean) - Method in class org.apache.gobblin.util.request_allocation.ResourcePool

ExceptionCatchingPathAlterationListenerDecorator - Class in org.apache.gobblin.util.filesystem

A decorator for PathAlterationListener that catches and logs any exception thrown by the underlying listener, preventing it from failing the application.

ExceptionCatchingPathAlterationListenerDecorator() - Constructor for class org.apache.gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator

execCallbacks(Function<? super L, R>, L) - Method in class org.apache.gobblin.util.callbacks.CallbacksDispatcher

execCallbacks(Function<? super L, R>) - Method in class org.apache.gobblin.util.callbacks.CallbacksDispatcher

execute() - Method in class org.apache.gobblin.util.executors.IteratorExecutor

Execute the tasks in the task Iterator.

execute(Runnable) - Method in class org.apache.gobblin.util.executors.MDCPropagatingExecutorService

execute(Runnable) - Method in class org.apache.gobblin.util.executors.MDCPropagatingScheduledExecutorService

executeAndGetResults() - Method in class org.apache.gobblin.util.executors.IteratorExecutor

Execute the tasks in the task Iterator.

executeStatements(String...) - Method in class org.apache.gobblin.util.HiveJdbcConnector

Executes the given SQL statements.

EXECUTOR_SERVICE_SHUTDOWN_TIMEOUT - Static variable in class org.apache.gobblin.util.ExecutorsUtils

EXECUTOR_SERVICE_SHUTDOWN_TIMEOUT_TIMEUNIT - Static variable in class org.apache.gobblin.util.ExecutorsUtils

ExecutorsUtils - Class in org.apache.gobblin.util

A utility class to use with Executors in cases such as when creating new thread pools.

ExecutorsUtils() - Constructor for class org.apache.gobblin.util.ExecutorsUtils

exists(Path) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

ExponentialBackoff - Class in org.apache.gobblin.util

Utility class for exponential backoff.

ExponentialBackoff.NoMoreRetriesException - Exception in org.apache.gobblin.util

Thrown if no more retries are available for ExponentialBackoff.

Extract() - Constructor for class org.apache.gobblin.util.test.TestingSource.Extract

EXTRACT_LIMIT_COUNT_LIMIT_KEY - Static variable in class org.apache.gobblin.util.limiter.DefaultLimiterFactory

EXTRACT_LIMIT_POOL_SIZE_KEY - Static variable in class org.apache.gobblin.util.limiter.DefaultLimiterFactory

EXTRACT_LIMIT_RATE_LIMIT_KEY - Static variable in class org.apache.gobblin.util.limiter.DefaultLimiterFactory

EXTRACT_LIMIT_RATE_LIMIT_KEY_DEP - Static variable in class org.apache.gobblin.util.limiter.DefaultLimiterFactory

Deprecated.

EXTRACT_LIMIT_RATE_LIMIT_TIMEUNIT_KEY - Static variable in class org.apache.gobblin.util.limiter.DefaultLimiterFactory

EXTRACT_LIMIT_RATE_LIMIT_TIMEUNIT_KEY_DEP - Static variable in class org.apache.gobblin.util.limiter.DefaultLimiterFactory

Deprecated.

EXTRACT_LIMIT_TIME_LIMIT_KEY - Static variable in class org.apache.gobblin.util.limiter.DefaultLimiterFactory

EXTRACT_LIMIT_TIME_LIMIT_KEY_DEP - Static variable in class org.apache.gobblin.util.limiter.DefaultLimiterFactory

Deprecated.

EXTRACT_LIMIT_TIME_LIMIT_TIMEUNIT_KEY - Static variable in class org.apache.gobblin.util.limiter.DefaultLimiterFactory

EXTRACT_LIMIT_TIME_LIMIT_TIMEUNIT_KEY_DEP - Static variable in class org.apache.gobblin.util.limiter.DefaultLimiterFactory

Deprecated.

EXTRACT_LIMIT_TYPE_KEY - Static variable in class org.apache.gobblin.util.limiter.DefaultLimiterFactory

ExtractorImpl(int) - Constructor for class org.apache.gobblin.util.test.HelloWorldSource.ExtractorImpl

ExtractorImpl(WorkUnitState) - Constructor for class org.apache.gobblin.util.test.StressTestingSource.ExtractorImpl

extractPropertiesWithPrefix(Properties, Optional<String>) - Static method in class org.apache.gobblin.util.PropertiesUtils

Extract all the keys that start with a prefix in Properties to a new Properties instance.

extractPropertiesWithPrefixAfterRemovingPrefix(Properties, String) - Static method in class org.apache.gobblin.util.PropertiesUtils

Extract all the keys that start with a prefix in Properties to a new Properties instance.

F

Factory() - Constructor for class org.apache.gobblin.util.filesystem.ModTimeDataFileVersionStrategy.Factory

Factory() - Constructor for class org.apache.gobblin.util.filesystem.ThrottledFileSystem.Factory

Factory() - Constructor for class org.apache.gobblin.util.io.StreamThrottler.Factory

Factory() - Constructor for class org.apache.gobblin.util.limiter.CountBasedLimiter.Factory

Factory() - Constructor for class org.apache.gobblin.util.limiter.PoolBasedLimiter.Factory

Factory() - Constructor for class org.apache.gobblin.util.limiter.RateBasedLimiter.Factory

Factory() - Constructor for class org.apache.gobblin.util.limiter.TimeBasedLimiter.Factory

Factory() - Constructor for class org.apache.gobblin.util.request_allocation.BruteForceAllocator.Factory

Factory() - Constructor for class org.apache.gobblin.util.request_allocation.GreedyAllocator.Factory

Factory() - Constructor for class org.apache.gobblin.util.request_allocation.HierarchicalAllocator.Factory

Factory() - Constructor for class org.apache.gobblin.util.request_allocation.PreOrderAllocator.Factory

FACTORY_ALIAS - Static variable in class org.apache.gobblin.util.limiter.CountBasedLimiter

FACTORY_NAME - Static variable in class org.apache.gobblin.util.eventbus.EventBusFactory

FACTORY_NAME - Static variable in class org.apache.gobblin.util.filesystem.FileSystemFactory

FAIL_IF_NO_GLOBAL_LIMITER_KEY - Static variable in class org.apache.gobblin.util.limiter.broker.SharedLimiterFactory

FAIL_ON_UNKNOWN_RESOURCE_ID - Static variable in class org.apache.gobblin.util.limiter.broker.SharedLimiterFactory

FailedTransitionCallbackException(FiniteStateMachine<?>.Transition, FiniteStateMachine.FailedCallback, Throwable) - Constructor for exception org.apache.gobblin.fsm.FiniteStateMachine.FailedTransitionCallbackException

FastSequentialExtractor() - Constructor for class org.apache.gobblin.util.test.FastSequentialSource.FastSequentialExtractor

FastSequentialSource - Class in org.apache.gobblin.util.test

A very simple Source that just emits long values counting up from 1 until either it reaches a user specified value or a user specified time elapses.

FastSequentialSource() - Constructor for class org.apache.gobblin.util.test.FastSequentialSource

FastSequentialSource.FastSequentialExtractor - Class in org.apache.gobblin.util.test

fetchJobFilesRecursively(Path) - Method in class org.apache.gobblin.util.PullFileLoader

FIELD_LOCATION_DELIMITER - Static variable in class org.apache.gobblin.util.AvroUtils

FieldWeighter - Class in org.apache.gobblin.util.binpacking

A WorkUnitWeighter implementation that parses the weight from a field in the work unit.

FieldWeighter() - Constructor for class org.apache.gobblin.util.binpacking.FieldWeighter

FileListUtils - Class in org.apache.gobblin.util

Utility class for listing files on a FileSystem.

FileListUtils() - Constructor for class org.apache.gobblin.util.FileListUtils

FileStatusEntry - Class in org.apache.gobblin.util.filesystem

FileStatusEntry(Path) - Constructor for class org.apache.gobblin.util.filesystem.FileStatusEntry

FileSystemFactory<S extends ScopeType<S>> - Class in org.apache.gobblin.util.filesystem

A SharedResourceFactory for creating FileSystems.

FileSystemFactory() - Constructor for class org.apache.gobblin.util.filesystem.FileSystemFactory

FileSystemInstrumentation - Class in org.apache.gobblin.util.filesystem

A base class for FileSystem decorators used for FileSystemInstrumentationFactorys.

FileSystemInstrumentation(FileSystem) - Constructor for class org.apache.gobblin.util.filesystem.FileSystemInstrumentation

FileSystemInstrumentationFactory<S extends ScopeType<S>> - Class in org.apache.gobblin.util.filesystem

A factory that instruments FileSystem.

FileSystemInstrumentationFactory() - Constructor for class org.apache.gobblin.util.filesystem.FileSystemInstrumentationFactory

FileSystemKey - Class in org.apache.gobblin.util.filesystem

FileSystemKey(URI, Configuration) - Constructor for class org.apache.gobblin.util.filesystem.FileSystemKey

FileSystemLimiterKey - Class in org.apache.gobblin.util.filesystem

SharedLimiterKey used for NameNode throttling.

FileSystemLimiterKey(URI) - Constructor for class org.apache.gobblin.util.filesystem.FileSystemLimiterKey

FileSystemLimiterKey(URI, String) - Constructor for class org.apache.gobblin.util.filesystem.FileSystemLimiterKey

FileSystemSupplier - Interface in org.apache.gobblin.util.filesystem

An interface of supplier to get FileSystem

fileToProperties(String, Configuration) - Static method in class org.apache.gobblin.util.JobConfigurationUtils

Load the properties from the specified file into a Properties object.

fileToProperties(String) - Static method in class org.apache.gobblin.util.JobConfigurationUtils

Load the properties from the specified file into a Properties object.

FileUtils - Class in org.apache.gobblin.util

FileUtils() - Constructor for class org.apache.gobblin.util.FileUtils

filter(List<String>, List<Pattern>, List<Pattern>) - Static method in class org.apache.gobblin.util.DatasetFilterUtils

filter(Set<String>, List<Pattern>, List<Pattern>) - Static method in class org.apache.gobblin.util.DatasetFilterUtils

FilterStreamUnpacker - Class in org.apache.gobblin.util.io

Contains utilities for getting uderlying streams to a filter stream.

FilterStreamUnpacker() - Constructor for class org.apache.gobblin.util.io.FilterStreamUnpacker

findFullPrefixKeys(Properties, Optional<String>) - Static method in class org.apache.gobblin.util.ConfigUtils

Finds a list of properties whose keys are complete prefix of other keys.

findWrappedMeteredInputStream(InputStream) - Static method in class org.apache.gobblin.util.io.MeteredInputStream

Find the lowest MeteredInputStream in a chain of FilterInputStreams.

findWrappedMeteredOutputStream(OutputStream) - Static method in class org.apache.gobblin.util.io.MeteredOutputStream

Find the lowest MeteredOutputStream in a chain of FilterOutputStreams.

FiniteStateMachine<T> - Class in org.apache.gobblin.fsm

An implementation of a basic FiniteStateMachine that allows keeping track of the state its state and gating certain logic on whether a transition is valid or not.

FiniteStateMachine(SetMultimap<T, T>, Set<T>, T, T) - Constructor for class org.apache.gobblin.fsm.FiniteStateMachine

FiniteStateMachine.AbandonedTransitionException - Exception in org.apache.gobblin.fsm

Thrown when a transition was initiated by a thread that no longer exists, likely implying that the transition can never be closed.

FiniteStateMachine.Builder<T> - Class in org.apache.gobblin.fsm

Used to build a FiniteStateMachine instance.

FiniteStateMachine.FailedCallback - Enum in org.apache.gobblin.fsm

FiniteStateMachine.FailedTransitionCallbackException - Exception in org.apache.gobblin.fsm

Thrown when the callbacks when closing a transition fail.

FiniteStateMachine.ReentrantStableStateWait - Exception in org.apache.gobblin.fsm

Thrown when a thread that has started a transition is waiting for a non-transitioning state, which is a deadlock situation.

FiniteStateMachine.Transition - Class in org.apache.gobblin.fsm

A handle used for controlling the transition of the FiniteStateMachine.

FiniteStateMachine.UnallowedTransitionException - Exception in org.apache.gobblin.fsm

If a transition is not allowed to happen.

flatten(Schema, boolean) - Method in class org.apache.gobblin.util.AvroFlattener

Flatten the Schema to un-nest recursive Records (to make it optimal for ORC)

FLATTENED_SOURCE_KEY - Static variable in class org.apache.gobblin.util.AvroFlattener

flattenWorkUnits(Collection<WorkUnit>) - Static method in class org.apache.gobblin.util.JobLauncherUtils

Utility method that takes in a List of WorkUnits, and flattens them.

flush() - Method in class org.apache.gobblin.util.test.StdoutWriter

ForceQueuePolicy - Class in org.apache.gobblin.util.executors

RejectedExecutionHandler that just re-attempts to put into the queue.

ForceQueuePolicy() - Constructor for class org.apache.gobblin.util.executors.ForceQueuePolicy

ForkOperatorUtils - Class in org.apache.gobblin.util

Utility class for use with the ForkOperator class.

ForkOperatorUtils() - Constructor for class org.apache.gobblin.util.ForkOperatorUtils

FormatException(String) - Constructor for exception org.apache.gobblin.util.StringParsingUtils.FormatException

FormatException(String, Throwable) - Constructor for exception org.apache.gobblin.util.StringParsingUtils.FormatException

FormatException(Throwable) - Constructor for exception org.apache.gobblin.util.StringParsingUtils.FormatException

formatJvmArguments(Optional<String>) - Static method in class org.apache.gobblin.util.JvmUtils

Formats the specified jvm arguments such that any tokens are replaced with concrete values;

forName(String) - Static method in enum org.apache.gobblin.util.limiter.BaseLimiterType

Get a BaseLimiterType for the given name.

fromByteArrays(byte[]...) - Static method in class org.apache.gobblin.util.guid.Guid

Generate a Guid for an array of byte arrays.

fromHasGuid(HasGuid...) - Static method in class org.apache.gobblin.util.guid.Guid

Generate a Guid for an array of HasGuid.

fromStrings(String...) - Static method in class org.apache.gobblin.util.guid.Guid

Generate a Guid for an array of Strings.

FS_SCHEMES_NON_ATOMIC - Static variable in class org.apache.gobblin.util.HadoopUtils

A Collection of all known FileSystem schemes that do not support atomic renames or copies.

G

generateDumpScript(Path, FileSystem, String, String) - Static method in class org.apache.gobblin.util.HeapDumpForTaskUtils

Generate the dumpScript, which is used when OOM error is thrown during task execution.

generateKey(SharedResourceFactory, SharedResourceKey, ScopeType, String) - Static method in class org.apache.gobblin.broker.BrokerConfigurationKeyGenerator

Generate a SharedResourcesBroker configuration key for a particular SharedResourceFactory, SharedResourceKey and ScopeType.

get() - Method in class org.apache.gobblin.util.AutoReturnableObject

get(String, Optional<String>) - Static method in class org.apache.gobblin.util.concurrent.TaskSchedulerFactory

Gets an instance of the TaskScheduler with the specified type and ensures that it is started.

get(TaskSchedulerType, Optional<String>) - Static method in class org.apache.gobblin.util.concurrent.TaskSchedulerFactory

Gets an instance of the TaskScheduler with the specified type and ensures that it is started.

get() - Method in class org.apache.gobblin.util.Either

Get the value of this Either instance, returned as class Object.

get(String, SharedResourcesBroker<S>) - Static method in class org.apache.gobblin.util.eventbus.EventBusFactory

get(Configuration, SharedResourcesBroker<S>) - Static method in class org.apache.gobblin.util.filesystem.FileSystemFactory

Equivalent to FileSystem.get(Configuration), but uses the input SharedResourcesBroker to configure add-ons to the FileSystem (e.g.

get(URI, Configuration, SharedResourcesBroker<S>) - Static method in class org.apache.gobblin.util.filesystem.FileSystemFactory

Equivalent to FileSystem.get(URI, Configuration), but uses the input SharedResourcesBroker to configure add-ons to the FileSystem (e.g.

get(EnumSet<Id.Parts>) - Method in class org.apache.gobblin.util.Id

get() - Method in class org.apache.gobblin.util.jdbc.DataSourceProvider

getAllFSTokens(Configuration, Credentials, String, Optional<String>, List<String>) - Static method in class org.apache.gobblin.util.hadoop.TokenUtils

getAllFSTokensImpl(Configuration, Credentials, String, List<String>) - Static method in class org.apache.gobblin.util.hadoop.TokenUtils

getAnyNonHiddenFile(FileSystem, Path) - Static method in class org.apache.gobblin.util.FileListUtils

Get any data file, which is not hidden or a directory, from the given path

getAutoScope(SharedResourcesBroker<S>, ConfigView<S, EventBusKey>) - Method in class org.apache.gobblin.util.eventbus.EventBusFactory

getAutoScope(SharedResourcesBroker<S>, ConfigView<S, FileSystemKey>) - Method in class org.apache.gobblin.util.filesystem.FileSystemFactory

getAutoScope(SharedResourcesBroker<S>, ConfigView<S, EmptyKey>) - Method in class org.apache.gobblin.util.io.StreamThrottler.Factory

getAutoScope(SharedResourcesBroker<S>, ConfigView<S, SharedLimiterKey>) - Method in class org.apache.gobblin.util.limiter.broker.SharedLimiterFactory

getAzkabanTags() - Static method in class org.apache.gobblin.util.AzkabanTags

Uses AzkabanTags.getAzkabanTags(Configuration) with default Hadoop Configuration

getAzkabanTags(Configuration) - Static method in class org.apache.gobblin.util.AzkabanTags

Gets all useful Azkaban runtime properties required by metrics as a Map.

getBoolean(Config, String, boolean) - Static method in class org.apache.gobblin.util.ConfigUtils

Return boolean value at path if config has path.

getBytesProcessedMeter() - Method in class org.apache.gobblin.util.io.MeteredInputStream

getBytesProcessedMeter() - Method in class org.apache.gobblin.util.io.MeteredOutputStream

getBytesProcessedMeter() - Method in interface org.apache.gobblin.util.io.MeteredStream

getChildren() - Method in class org.apache.gobblin.util.filesystem.FileStatusEntry

Return the directory's files.

getClusterName(String) - Method in class org.apache.gobblin.util.ClustersNames

Returns human-readable name of the cluster.

getClusterName() - Method in class org.apache.gobblin.util.ClustersNames

Returns the cluster name on which the application is running.

getClusterName(Configuration) - Method in class org.apache.gobblin.util.ClustersNames

Returns the cluster name on which the application is running.

getClusterNameTags() - Static method in class org.apache.gobblin.util.ClusterNameTags

Uses ClusterNameTags.getClusterNameTags(Configuration) with default Hadoop Configuration.

getClusterNameTags(Configuration) - Static method in class org.apache.gobblin.util.ClusterNameTags

Gets all useful Hadoop cluster metrics.

getCodecFactory(Optional<String>, Optional<String>) - Static method in class org.apache.gobblin.util.WriterUtils

Creates a CodecFactory based on the specified codec name and deflate level.

getConfFromProperties(Properties) - Static method in class org.apache.gobblin.util.HadoopUtils

getConfFromState(State) - Static method in class org.apache.gobblin.util.HadoopUtils

getConfFromState(State, Optional<String>) - Static method in class org.apache.gobblin.util.HadoopUtils

Provides Hadoop configuration given state.

getConfig() - Method in class org.apache.gobblin.broker.KeyedScopedConfigViewImpl

getConfig(Config, String, Config) - Static method in class org.apache.gobblin.util.ConfigUtils

Return Config value at path if config has path.

getConfigOrEmpty(Config, String) - Static method in class org.apache.gobblin.util.ConfigUtils

getConfigView(S, K, String) - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl

Get a ConfigView for the input scope, key, and factory.

getConnection() - Method in class org.apache.gobblin.util.HiveJdbcConnector

getCurrentState() - Method in class org.apache.gobblin.fsm.FiniteStateMachine

Get the current state.

getDataPublisherFinalDir(State, int, int) - Static method in class org.apache.gobblin.util.WriterUtils

Get the Path corresponding the to the directory a given BaseDataPublisher should commits its output data.

getDatasetSpecificProps(Iterable<String>, State) - Static method in class org.apache.gobblin.util.dataset.DatasetUtils

Given a Iterable of dataset identifiers (e.g., name, URN, etc.), return a Map that links each dataset with the extra configuration information specified in the state via DatasetUtils.DATASET_SPECIFIC_PROPS.

getDateTimeFieldType() - Method in enum org.apache.gobblin.util.DatePartitionType

getDateTimePattern() - Method in enum org.apache.gobblin.util.DatePartitionType

getDecoratedObject() - Method in class org.apache.gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator

getDecoratedObject() - Method in class org.apache.gobblin.util.io.BatchedMeterDecorator

getDefault() - Static method in class org.apache.gobblin.util.TrustManagerSocketFactory

getDefaultCipherSuites() - Method in class org.apache.gobblin.util.TrustManagerSecureSocketFactory

getDefaultCipherSuites() - Method in class org.apache.gobblin.util.TrustManagerSocketFactory

getDefaultExecutor(Logger) - Static method in class org.apache.gobblin.util.callbacks.CallbacksDispatcher

getDefaultJacksonFactory() - Static method in class org.apache.gobblin.util.json.JsonUtils

Get a Jackson JsonFactory configured with standard settings.

getDefaultObjectMapper() - Static method in class org.apache.gobblin.util.json.JsonUtils

Get a Jackson ObjectMapper configured with standard settings.

getDefaultWriterFilePath(State, int, int) - Static method in class org.apache.gobblin.util.WriterUtils

Creates the default Path for the ConfigurationKeys.WRITER_FILE_PATH key.

getDeserializer(Class<WritableShim>) - Method in class org.apache.gobblin.util.WritableShimSerialization

getDirectorySchema(Path, FileSystem, boolean) - Static method in class org.apache.gobblin.util.AvroUtils

Get the latest avro schema for a directory

getDirectorySchema(Path, Configuration, boolean) - Static method in class org.apache.gobblin.util.AvroUtils

Get the latest avro schema for a directory

getDouble(Config, String, double) - Static method in class org.apache.gobblin.util.ConfigUtils

Return double value at path if config has path.

getExpectedRecordCount() - Method in class org.apache.gobblin.util.test.FastSequentialSource.FastSequentialExtractor

getExpectedRecordCount() - Method in class org.apache.gobblin.util.test.HelloWorldSource.ExtractorImpl

getExpectedRecordCount() - Method in class org.apache.gobblin.util.test.StressTestingSource.ExtractorImpl

getExpectedRecordCount() - Method in class org.apache.gobblin.util.test.TestingSource.Extract

getExtractor(WorkUnitState) - Method in class org.apache.gobblin.util.test.FastSequentialSource

getExtractor(WorkUnitState) - Method in class org.apache.gobblin.util.test.HelloWorldSource

getExtractor(WorkUnitState) - Method in class org.apache.gobblin.util.test.StressTestingSource

getExtractor(WorkUnitState) - Method in class org.apache.gobblin.util.test.TestingSource

getExtractsForPredicate(Multimap<Extract, WorkUnitState>, Predicate<WorkUnitState>) - Static method in class org.apache.gobblin.util.PublisherUtils

Given a Multimap of Extracts to WorkUnitStates, filter out any Extracts where all of the corresponding WorkUnitStates do not meet the given Predicate.

getFactorySpecificConfig() - Method in class org.apache.gobblin.broker.KeyedScopedConfigViewImpl

getField(Schema, String) - Static method in class org.apache.gobblin.util.AvroUtils

Given a GenericRecord, this method will return the field specified by the path parameter.

getField(DateTime) - Method in enum org.apache.gobblin.util.DatePartitionType

getFieldSchema(Schema, String) - Static method in class org.apache.gobblin.util.AvroUtils

Given a GenericRecord, this method will return the schema of the field specified by the path parameter.

getFieldValue(GenericRecord, String) - Static method in class org.apache.gobblin.util.AvroUtils

Given a GenericRecord, this method will return the field specified by the path parameter.

getFileStatus(Path) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

getFileSystem() - Method in interface org.apache.gobblin.util.filesystem.FileSystemSupplier

Function to get fileSystem

getForkId(String, int) - Static method in class org.apache.gobblin.util.ForkOperatorUtils

Get the fork branch ID of a branch of a given task.

getFsConfiguration(State) - Static method in class org.apache.gobblin.util.WriterUtils

getFsWithProxy(State, String, Configuration) - Static method in class org.apache.gobblin.util.JobLauncherUtils

getGroupMembers(String) - Method in class org.apache.gobblin.util.LdapUtils

getHadoopAndHiveTokensForProxyUser(State, Optional<File>, UserGroupInformation, IMetaStoreClient, String) - Static method in class org.apache.gobblin.util.hadoop.TokenUtils

Get Hadoop tokens (tokens for job history server, job tracker, hive and HDFS) using Kerberos keytab, on behalf on a proxy user, embed tokens into a UserGroupInformation as returned result, persist in-memory credentials if tokenFile specified Note that when a super-user is fetching tokens for other users, TokenUtils.fetchHcatToken(String, HiveConf, String, IMetaStoreClient) getDelegationToken} explicitly contains a string parameter indicating proxy user, while other hadoop services require impersonation first.

getHadoopFSTokens(State, Optional<File>, Credentials, String) - Static method in class org.apache.gobblin.util.hadoop.TokenUtils

getHadoopTokens(State, Optional<File>, Credentials) - Static method in class org.apache.gobblin.util.hadoop.TokenUtils

Get Hadoop tokens (tokens for job history server, job tracker and HDFS) using Kerberos keytab.

getHighWatermark() - Method in class org.apache.gobblin.util.test.FastSequentialSource.FastSequentialExtractor

getHighWatermark() - Method in class org.apache.gobblin.util.test.HelloWorldSource.ExtractorImpl

getHighWatermark() - Method in class org.apache.gobblin.util.test.StressTestingSource.ExtractorImpl

getHighWatermark() - Method in class org.apache.gobblin.util.test.TestingSource.Extract

getHiveToken(State, IMetaStoreClient, Credentials, String, UserGroupInformation) - Static method in class org.apache.gobblin.util.hadoop.TokenUtils

getHostName() - Static method in class org.apache.gobblin.util.HostUtils

getImplicitBroker() - Static method in class org.apache.gobblin.broker.SharedResourcesBrokerFactory

Get the implicit SharedResourcesBroker in the callers thread.

getInstance() - Static method in class org.apache.gobblin.util.ClustersNames

getInt(Config, String, Integer) - Static method in class org.apache.gobblin.util.ConfigUtils

Return Integer value at path if config has path.

getIvySettingsFile() - Static method in class org.apache.gobblin.util.DownloadUtils

Get ivy settings file from classpath

getJavaClassPath() - Method in class org.apache.gobblin.util.SystemPropertiesWrapper

getJavaHome() - Method in class org.apache.gobblin.util.SystemPropertiesWrapper

The path to the JRE that is used to run the program which does the lookup.

getJoinIterator(Iterator<? extends Requestor<T>>, ConcurrentBoundedPriorityIterable<T>) - Method in class org.apache.gobblin.util.request_allocation.BruteForceAllocator

getJoinIterator(Iterator<? extends Requestor<T>>, ConcurrentBoundedPriorityIterable<T>) - Method in class org.apache.gobblin.util.request_allocation.GreedyAllocator

getJoinIterator(Iterator<? extends Requestor<T>>, ConcurrentBoundedPriorityIterable<T>) - Method in class org.apache.gobblin.util.request_allocation.PreOrderAllocator

getJoinIterator(Iterator<? extends Requestor<T>>, ConcurrentBoundedPriorityIterable<T>) - Method in class org.apache.gobblin.util.request_allocation.PriorityIterableBasedRequestAllocator

getJvmInputArguments() - Static method in class org.apache.gobblin.util.JvmUtils

Gets the input arguments passed to the JVM.

getKey() - Method in interface org.apache.gobblin.util.concurrent.ScheduledTask

The unique key used to identify this task.

getKeyedConfig() - Method in class org.apache.gobblin.broker.KeyedScopedConfigViewImpl

getKeyedScopedConfig() - Method in class org.apache.gobblin.broker.KeyedScopedConfigViewImpl

getLeft() - Method in class org.apache.gobblin.util.Either.Left

getLevel() - Method in class org.apache.gobblin.util.filesystem.FileStatusEntry

Return the level

getListeners() - Method in class org.apache.gobblin.util.callbacks.CallbacksDispatcher

getListeners() - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserver

Returns the set of registered file system listeners.

getLocalFSToken(Configuration, Credentials, String) - Static method in class org.apache.gobblin.util.hadoop.TokenUtils

getLog() - Method in class org.apache.gobblin.util.callbacks.CallbacksDispatcher

getLong(Config, String, Long) - Static method in class org.apache.gobblin.util.ConfigUtils

Return Long value at path if config has path.

getLowestIntervalUnit(String) - Static method in enum org.apache.gobblin.util.DatePartitionType

getMRTokenRenewerInternal(JobConf) - Static method in class org.apache.gobblin.util.hadoop.TokenUtils

getMultiFieldValue(GenericRecord, String) - Static method in class org.apache.gobblin.util.AvroUtils

getName() - Method in class org.apache.gobblin.util.eventbus.EventBusFactory

getName() - Method in class org.apache.gobblin.util.filesystem.FileSystemFactory

getName() - Method in class org.apache.gobblin.util.io.StreamThrottler.Factory

getName() - Method in class org.apache.gobblin.util.limiter.broker.SharedLimiterFactory

getNamespaceTableWriterFilePath(State) - Static method in class org.apache.gobblin.util.WriterUtils

Creates Path for case WriterUtils.WriterFilePathType.NAMESPACE_TABLE with configurations ConfigurationKeys.EXTRACT_NAMESPACE_NAME_KEY and ConfigurationKeys.EXTRACT_TABLE_NAME_KEY

getObservers() - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserverScheduler

Returns the set of PathAlterationObserver registered with this monitor.

getOptionallyThrottledFileSystem(FileSystem, State) - Static method in class org.apache.gobblin.util.HadoopUtils

Calls HadoopUtils.getOptionallyThrottledFileSystem(FileSystem, int) parsing the qps from the input State at key HadoopUtils.MAX_FILESYSTEM_QPS.

getOptionallyThrottledFileSystem(FileSystem, int) - Static method in class org.apache.gobblin.util.HadoopUtils

Get a throttled FileSystem that limits the number of queries per second to a FileSystem.

getOptions() - Method in interface org.apache.gobblin.runtime.cli.CliObjectFactory

getOptions() - Method in class org.apache.gobblin.runtime.cli.ConstructorAndPublicMethodsCliObjectFactory

getParent() - Method in class org.apache.gobblin.util.filesystem.FileStatusEntry

Return the parent entry.

getPath() - Method in class org.apache.gobblin.util.filesystem.FileStatusEntry

Return the path from the instance FileStatus variable

getPathForBranch(State, String, int, int) - Static method in class org.apache.gobblin.util.ForkOperatorUtils

Get a new path with the given branch name as a sub directory.

getPathWithoutSchemeAndAuthority(Path) - Static method in class org.apache.gobblin.util.PathUtils

Removes the Scheme and Authority from a Path.

getPatternList(State, String) - Static method in class org.apache.gobblin.util.DatasetFilterUtils

getPatternList(State, String, String) - Static method in class org.apache.gobblin.util.DatasetFilterUtils

getPatternsFromStrings(List<String>) - Static method in class org.apache.gobblin.util.DatasetFilterUtils

Convert a list of Strings to a list of Patterns.

getPos() - Method in class org.apache.gobblin.util.io.SeekableFSInputStream

getPrefix() - Method in class org.apache.gobblin.util.Id

getPrefix() - Method in class org.apache.gobblin.util.Id.Job

getPrefix() - Method in class org.apache.gobblin.util.Id.MultiTask

getPrefix() - Method in class org.apache.gobblin.util.Id.Task

getPrincipalUsingHostname(String, String) - Static method in class org.apache.gobblin.util.HostUtils

getPropAsBoolean(Properties, String, String) - Static method in class org.apache.gobblin.util.PropertiesUtils

getPropAsInt(Properties, String, int) - Static method in class org.apache.gobblin.util.PropertiesUtils

getPropAsList(Properties, String) - Static method in class org.apache.gobblin.util.PropertiesUtils

Get the value of a comma separated property as a List of strings.

getPropAsList(Properties, String, String) - Static method in class org.apache.gobblin.util.PropertiesUtils

Get the value of a property as a list of strings, using the given default value if the property is not set.

getPropAsLong(Properties, String, long) - Static method in class org.apache.gobblin.util.PropertiesUtils

getProperty(String) - Method in class org.apache.gobblin.util.ImmutableProperties

Need to override this method otherwise it will call super.get(key).

getPropertyNameForBranch(String, int, int) - Static method in class org.apache.gobblin.util.ForkOperatorUtils

Get a new property key from an original one with branch index (if applicable).

getPropertyNameForBranch(String, int) - Static method in class org.apache.gobblin.util.ForkOperatorUtils

Get a new property key from an original one with branch index (if applicable).

getPropertyNameForBranch(WorkUnitState, String) - Static method in class org.apache.gobblin.util.ForkOperatorUtils

Get a new property key from an original one based on the branch id.

getProxiedFileSystem(String, Properties, URI) - Static method in class org.apache.gobblin.util.ProxiedFileSystemCache

Deprecated.

use #fromProperties

getProxiedFileSystem(String, Properties, Configuration) - Static method in class org.apache.gobblin.util.ProxiedFileSystemCache

Deprecated.

use #fromProperties

getProxiedFileSystem(String, Properties, URI, Configuration) - Static method in class org.apache.gobblin.util.ProxiedFileSystemCache

Deprecated.

Use #fromProperties

getProxiedFileSystem(State, ProxiedFileSystemWrapper.AuthType, String, String) - Method in class org.apache.gobblin.util.ProxiedFileSystemWrapper

Deprecated.

Same as @see #getProxiedFileSystem(State, AuthType, String, String, Configuration) where state properties will be copied into Configuration.

getProxiedFileSystem(State, ProxiedFileSystemWrapper.AuthType, String, String, Configuration) - Method in class org.apache.gobblin.util.ProxiedFileSystemWrapper

Deprecated.

Getter for proxiedFs, using the passed parameters to create an instance of a proxiedFs.

getProxiedFileSystemUsingKeytab(State, URI, Configuration) - Static method in class org.apache.gobblin.util.ProxiedFileSystemCache

Deprecated.

use #fromKeytab.

getProxiedFileSystemUsingKeytab(String, String, Path, URI, Configuration) - Static method in class org.apache.gobblin.util.ProxiedFileSystemCache

Deprecated.

use #fromKeytab.

getProxiedFileSystemUsingToken(String, Token<?>, URI, Configuration) - Static method in class org.apache.gobblin.util.ProxiedFileSystemCache

Deprecated.

use #fromToken.

getRateIfRateControlled(FileSystem) - Static method in class org.apache.gobblin.util.RateControlledFileSystem

Deprecated.

Determines whether the file system is rate controlled, and if so, returns the allowed rate in operations per second.

getRateLimiter() - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

getRateLimiter() - Method in class org.apache.gobblin.util.RateControlledFileSystem

Deprecated.

getRecordCount(Path) - Method in class org.apache.gobblin.util.recordcount.CompactionRecordCountProvider

Get the record count through filename.

getRecordCount(Path) - Method in class org.apache.gobblin.util.recordcount.IngestionRecordCountProvider

The record count should be the last component before the filename extension.

getRecordCount(Path) - Method in class org.apache.gobblin.util.recordcount.LateFileRecordCountProvider

Get record count from a given filename (possibly having LATE_COMPONENT{RandomInteger}), using the original FileNameWithRecordCountFormat.

getRejectedAndDroppedRequests(ConcurrentBoundedPriorityIterable<T>) - Method in class org.apache.gobblin.util.request_allocation.PriorityIterableBasedRequestAllocator

getRemoteFSTokenFromURI(Configuration, Credentials, String, List<String>) - Static method in class org.apache.gobblin.util.hadoop.TokenUtils

getRequestor() - Method in interface org.apache.gobblin.util.request_allocation.Request

getRequests(Comparator<T>) - Method in interface org.apache.gobblin.util.request_allocation.PushDownRequestor

Return an Iterator of Requests already sorted by the input prioritizer.

getResource() - Method in class org.apache.gobblin.broker.ImmediatelyInvalidResourceEntry

getResource() - Method in class org.apache.gobblin.broker.ResourceInstance

This method returns the resource, but may have logic before the return.

getResourceIfValid() - Method in class org.apache.gobblin.broker.ImmediatelyInvalidResourceEntry

This method is synchronized so that the validity check and validity change is atomic for callers of this method.

getResourceRequirementBuilder() - Method in class org.apache.gobblin.util.request_allocation.ResourcePool

Use to create a ResourceRequirement compatible with this ResourcePool.

getRestrictedFieldByReflection(Object, String, Class) - Static method in class org.apache.gobblin.util.reflection.RestrictedFieldAccessingUtils

Getting field defined in containingObj which was not publicly-accessible, using java-reflection.

getRestrictedFieldByReflectionRecursively(Object, String, Class) - Static method in class org.apache.gobblin.util.reflection.RestrictedFieldAccessingUtils

Getting field defined in superclass(es) which was not publicly-accessible, using java-reflection.

getRight() - Method in class org.apache.gobblin.util.Either.Right

getRootPath(Path) - Static method in class org.apache.gobblin.util.PathUtils

Returns the root path for the specified path.

getScheduledTask(K) - Method in class org.apache.gobblin.util.concurrent.TaskScheduler

Gets the ScheduledTask with the specified key.

getScheduledTasks() - Method in class org.apache.gobblin.util.concurrent.TaskScheduler

Gets all ScheduledTasks.

getSchema() - Method in class org.apache.gobblin.util.test.FastSequentialSource.FastSequentialExtractor

getSchema() - Method in class org.apache.gobblin.util.test.HelloWorldSource.ExtractorImpl

getSchema() - Method in class org.apache.gobblin.util.test.StressTestingSource.ExtractorImpl

getSchema() - Method in class org.apache.gobblin.util.test.TestingSource.Extract

getSchemaCreationTime(Schema) - Static method in class org.apache.gobblin.util.AvroUtils

getSchemaFromDataFile(Path, FileSystem) - Static method in class org.apache.gobblin.util.AvroUtils

Get Avro schema from an Avro data file.

getScope(S) - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl

getScopedConfig() - Method in class org.apache.gobblin.broker.KeyedScopedConfigViewImpl

getScopedView(S) - Method in class org.apache.gobblin.broker.KeyedScopedConfigViewImpl

getSerializer(Class<WritableShim>) - Method in class org.apache.gobblin.util.WritableShimSerialization

getServiceName() - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

getSharedResource(SharedResourceFactory<T, K, S>, K) - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl

getSharedResourceAtScope(SharedResourceFactory<T, K, S>, K, S) - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl

getSourceFileSystem(State) - Static method in class org.apache.gobblin.util.HadoopUtils

Get a FileSystem object for the uri specified at ConfigurationKeys.SOURCE_FILEBASED_FS_URI.

getStartState() - Method in class org.apache.gobblin.fsm.FiniteStateMachine.Transition

Get the state at the beginning of this transition.

getStateFromConf(Configuration) - Static method in class org.apache.gobblin.util.HadoopUtils

getString(Config, String, String) - Static method in class org.apache.gobblin.util.ConfigUtils

Return string value at path if config has path.

getStringList(Config, String) - Static method in class org.apache.gobblin.util.ConfigUtils

An extension to Config.getStringList(String).

getSupportedCipherSuites() - Method in class org.apache.gobblin.util.TrustManagerSecureSocketFactory

getSupportedCipherSuites() - Method in class org.apache.gobblin.util.TrustManagerSocketFactory

getTableNameWriterFilePath(State) - Static method in class org.apache.gobblin.util.WriterUtils

Creates Path for the ConfigurationKeys.WRITER_FILE_PATH key according to ConfigurationKeys.EXTRACT_TABLE_NAME_KEY.

getTimeUnit(Config, String, TimeUnit) - Static method in class org.apache.gobblin.util.ConfigUtils

Return TimeUnit value at path if config has path.

getTokenFromSeqFile(String, Path) - Static method in class org.apache.gobblin.util.ProxiedFileSystemUtils

Retrives a Token from a given sequence file for a specified user.

getUnderlyingMeter() - Method in class org.apache.gobblin.util.io.BatchedMeterDecorator

getUnitMilliseconds() - Method in enum org.apache.gobblin.util.DatePartitionType

Get the number of milliseconds associated with a partition type.

getUsageString() - Method in interface org.apache.gobblin.runtime.cli.CliObjectFactory

Get a usage string for display on the command line.

getUsageString() - Method in class org.apache.gobblin.runtime.cli.PublicMethodsCliObjectFactory

getValueClassName() - Method in class org.apache.gobblin.util.hadoop.GobblinSequenceFileReader

Returns the name of the value class.

getValuesAsList(Properties, Optional<String>) - Static method in class org.apache.gobblin.util.PropertiesUtils

Extract all the values whose keys start with a prefix

getVersion(Path) - Method in interface org.apache.gobblin.util.filesystem.DataFileVersionStrategy

Get the version of a path.

getVersion(Path) - Method in class org.apache.gobblin.util.filesystem.ModTimeDataFileVersionStrategy

getWorkunits(SourceState) - Method in class org.apache.gobblin.util.test.FastSequentialSource

getWorkunits(SourceState) - Method in class org.apache.gobblin.util.test.HelloWorldSource

getWorkunits(SourceState) - Method in class org.apache.gobblin.util.test.StressTestingSource

getWorkunits(SourceState) - Method in class org.apache.gobblin.util.test.TestingSource

getWriterFileName(State, int, int, String, String) - Static method in class org.apache.gobblin.util.WriterUtils

Get the value of ConfigurationKeys.WRITER_FILE_NAME for the a given DataWriter.

getWriterFilePath(State, int, int) - Static method in class org.apache.gobblin.util.WriterUtils

Get the Path corresponding the the relative file path for a given DataWriter.

getWriterFileSystem(State, int, int) - Static method in class org.apache.gobblin.util.HadoopUtils

Get a FileSystem object for the uri specified at ConfigurationKeys.WRITER_FILE_SYSTEM_URI.

getWriterFS(State, int, int) - Static method in class org.apache.gobblin.util.WriterUtils

getWriterFs(State) - Static method in class org.apache.gobblin.util.WriterUtils

getWriterFsUri(State, int, int) - Static method in class org.apache.gobblin.util.WriterUtils

getWriterOutputDir(State, int, int) - Static method in class org.apache.gobblin.util.WriterUtils

Get the Path corresponding the to the directory a given DataWriter should be writing its output data.

getWriterStagingDir(State, int, int) - Static method in class org.apache.gobblin.util.WriterUtils

Get the Path corresponding the to the directory a given DataWriter should be writing its staging data.

getWriterStagingDir(State, int, int, String) - Static method in class org.apache.gobblin.util.WriterUtils

Get the staging Path for DataWriter that has attemptId in the path.

GLOB_TOKENS - Static variable in class org.apache.gobblin.util.PathUtils

GLOBAL_HOCON_EXTENSION - Static variable in class org.apache.gobblin.util.PullFileLoader

GLOBAL_HOCON_PATH_FILTER - Static variable in class org.apache.gobblin.util.PullFileLoader

GLOBAL_PATH_FILTER - Static variable in class org.apache.gobblin.util.PullFileLoader

GLOBAL_PROPS_EXTENSION - Static variable in class org.apache.gobblin.util.PullFileLoader

GLOBAL_PROPS_PATH_FILTER - Static variable in class org.apache.gobblin.util.PullFileLoader

globStatus(Path) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

GOBBLIN_UTIL_JDBC_PREFIX - Static variable in class org.apache.gobblin.util.jdbc.DataSourceProvider

GobblinCli - Class in org.apache.gobblin.runtime.cli

Instantiates a CliApplication and runs it.

GobblinCli() - Constructor for class org.apache.gobblin.runtime.cli.GobblinCli

GobblinConstructorUtils - Class in org.apache.gobblin.util.reflection

Helper methods to instantiate classes

GobblinConstructorUtils() - Constructor for class org.apache.gobblin.util.reflection.GobblinConstructorUtils

GobblinProcessBuilder - Class in org.apache.gobblin.util

GobblinProcessBuilder() - Constructor for class org.apache.gobblin.util.GobblinProcessBuilder

GobblinSequenceFileReader - Class in org.apache.gobblin.util.hadoop

Override the SequenceFile.Reader mainly to override the {@link #getValueClassName()} so that we can handle the package name issue properly.

GobblinSequenceFileReader(FileSystem, Path, Configuration) - Constructor for class org.apache.gobblin.util.hadoop.GobblinSequenceFileReader

GreedyAllocator<T extends Request<T>> - Class in org.apache.gobblin.util.request_allocation

A RequestAllocator that selects Requests without any order guarantees until the ResourcePool is full, then stops.

GreedyAllocator(RequestAllocatorConfig<T>) - Constructor for class org.apache.gobblin.util.request_allocation.GreedyAllocator

GreedyAllocator.Factory - Class in org.apache.gobblin.util.request_allocation

Guid - Class in org.apache.gobblin.util.guid

Class wrapping a byte array representing a guid.

Guid(byte[]) - Constructor for class org.apache.gobblin.util.guid.Guid

Creates a Guid by computing the sha of the input bytes.

guid() - Method in interface org.apache.gobblin.util.guid.HasGuid

GUID_LENGTH - Static variable in class org.apache.gobblin.util.guid.Guid

H

HadoopUtils - Class in org.apache.gobblin.util

A utility class for working with Hadoop.

HadoopUtils() - Constructor for class org.apache.gobblin.util.HadoopUtils

hasCharacteristic(DataFileVersionStrategy.Characteristic) - Method in interface org.apache.gobblin.util.filesystem.DataFileVersionStrategy

hasContent(FileSystem, Path) - Static method in class org.apache.gobblin.util.HadoopUtils

hasFailed() - Method in class org.apache.gobblin.util.callbacks.CallbackResult

HasGuid - Interface in org.apache.gobblin.util.guid

Represents an object for which we can compute a unique, replicable Guid.

hashCode() - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl

hashCode() - Method in class org.apache.gobblin.util.filesystem.FileStatusEntry

Returns a hash code value for the object, which is defined as the hash code of the path name.

hashCode() - Method in class org.apache.gobblin.util.filesystem.FileSystemKey

hashCode() - Method in class org.apache.gobblin.util.filesystem.FileSystemLimiterKey

hashCode() - Method in class org.apache.gobblin.util.http.HttpLimiterKey

hashCode() - Method in class org.apache.gobblin.util.ImmutableProperties

hashCode() - Method in class org.apache.gobblin.util.io.StreamCopierSharedLimiterKey

hasNext() - Method in class org.apache.gobblin.util.iterators.InterruptibleIterator

hasNext() - Method in class org.apache.gobblin.util.request_allocation.AllocatedRequestsIteratorBase

hasNonEmptyPath(Config, String) - Static method in class org.apache.gobblin.util.ConfigUtils

Check if the given key exists in config and it is not null or empty Uses StringUtils.isNotBlank(CharSequence)

hasRejectedElement() - Method in class org.apache.gobblin.util.request_allocation.ConcurrentBoundedPriorityIterable

HDFS_ILLEGAL_TOKEN_REGEX - Static variable in class org.apache.gobblin.util.HadoopUtils

HeapDumpForTaskUtils - Class in org.apache.gobblin.util

A utility class for generating script to move the heap dump .prof files to HDFS for hadoop tasks, when Java heap out of memory error is thrown.

HeapDumpForTaskUtils() - Constructor for class org.apache.gobblin.util.HeapDumpForTaskUtils

HELLO_ID_FULL_KEY - Static variable in class org.apache.gobblin.util.test.HelloWorldSource

HELLO_ID_KEY - Static variable in class org.apache.gobblin.util.test.HelloWorldSource

helloMessage(int) - Static method in class org.apache.gobblin.util.test.HelloWorldSource.ExtractorImpl

HelloWorldSource - Class in org.apache.gobblin.util.test

Hello world!

HelloWorldSource() - Constructor for class org.apache.gobblin.util.test.HelloWorldSource

HelloWorldSource.ExtractorImpl - Class in org.apache.gobblin.util.test

HelpArgumentFound() - Constructor for exception org.apache.gobblin.runtime.cli.CliObjectFactory.HelpArgumentFound

HiddenFilter - Class in org.apache.gobblin.util.filters

A PathFilter that filters out hidden files (those starting with '_' or '.').

HiddenFilter() - Constructor for class org.apache.gobblin.util.filters.HiddenFilter

HierarchicalAllocator<T extends Request<T>> - Class in org.apache.gobblin.util.request_allocation

A RequestAllocator optimized for HierarchicalPrioritizers.

HierarchicalAllocator() - Constructor for class org.apache.gobblin.util.request_allocation.HierarchicalAllocator

HierarchicalAllocator.Factory - Class in org.apache.gobblin.util.request_allocation

HierarchicalPrioritizer<T extends Request> - Interface in org.apache.gobblin.util.request_allocation

A Comparator for Requests that can also compare Requestors, and which guarantees that given Requests r1, r2, then r1.getRequestor > r2.getRequestor implies r1 > r2.

HIVE_COMPATIBLE_TYPES - Static variable in class org.apache.gobblin.util.HiveAvroTypeConstants

HIVE_EXECUTION_SIMULATE - Static variable in class org.apache.gobblin.util.HiveJdbcConnector

HiveAvroTypeConstants - Class in org.apache.gobblin.util

HiveAvroTypeConstants() - Constructor for class org.apache.gobblin.util.HiveAvroTypeConstants

HiveJdbcConnector - Class in org.apache.gobblin.util

A class for managing a Hive JDBC connection.

HIVESERVER_CONNECTION_STRING - Static variable in class org.apache.gobblin.util.HiveJdbcConnector

HIVESERVER_PASSWORD - Static variable in class org.apache.gobblin.util.HiveJdbcConnector

HIVESERVER_URL - Static variable in class org.apache.gobblin.util.HiveJdbcConnector

HIVESERVER_USER - Static variable in class org.apache.gobblin.util.HiveJdbcConnector

HIVESERVER_VERSION - Static variable in class org.apache.gobblin.util.HiveJdbcConnector

HIVESITE_DIR - Static variable in class org.apache.gobblin.util.HiveJdbcConnector

HostUtils - Class in org.apache.gobblin.util

HostUtils() - Constructor for class org.apache.gobblin.util.HostUtils

HttpLimiterKey - Class in org.apache.gobblin.util.http

HttpLimiterKey used for http client throttling.

HttpLimiterKey(String) - Constructor for class org.apache.gobblin.util.http.HttpLimiterKey

HUMAN_READABLE_SIZE_PATTERN - Static variable in class org.apache.gobblin.util.StringParsingUtils

humanReadableToByteCount(String) - Static method in class org.apache.gobblin.util.StringParsingUtils

Convert a human readable string (e.g.

I

Id - Class in org.apache.gobblin.util

Id(String) - Constructor for class org.apache.gobblin.util.Id

Id(String, long) - Constructor for class org.apache.gobblin.util.Id

Id.Job - Class in org.apache.gobblin.util

Id.MultiTask - Class in org.apache.gobblin.util

Id.Parts - Enum in org.apache.gobblin.util

Id.Task - Class in org.apache.gobblin.util

IGNORE_CASE - Static variable in class org.apache.gobblin.data.management.copy.hive.WhitelistBlacklist

ImmediatelyInvalidResourceEntry<T> - Class in org.apache.gobblin.broker

A ResourceEntry that expires immediately.

ImmediatelyInvalidResourceEntry(T) - Constructor for class org.apache.gobblin.broker.ImmediatelyInvalidResourceEntry

ImmutableProperties - Class in org.apache.gobblin.util

Immutable wrapper for Properties.

ImmutableProperties(Properties) - Constructor for class org.apache.gobblin.util.ImmutableProperties

inferFromConfig(RequestAllocatorConfig<T>) - Static method in class org.apache.gobblin.util.request_allocation.RequestAllocatorUtils

Infer and construct a RequestAllocator from an input Config.

IngestionRecordCountProvider - Class in org.apache.gobblin.util.recordcount

Implementation of RecordCountProvider, which provides record count from file path.

IngestionRecordCountProvider() - Constructor for class org.apache.gobblin.util.recordcount.IngestionRecordCountProvider

initialize(URI, Configuration) - Method in class org.apache.gobblin.util.filesystem.InstrumentedFileSystem

initialize() - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserver

Initialize the observer.

INSTANCE - Static variable in class org.apache.gobblin.broker.EmptyKey

A singleton instance

instance - Static variable in class org.apache.gobblin.util.io.EmptyInputStream

INSTANCE - Static variable in class org.apache.gobblin.util.NoopCloseable

INSTANCE_NAME - Static variable in enum org.apache.gobblin.util.Id.Parts

instantiateDataFileVersionStrategy(FileSystem, Config) - Static method in interface org.apache.gobblin.util.filesystem.DataFileVersionStrategy

Instantiate a DataFileVersionStrategy according to input configuration.

InstrumentedFileSystem - Class in org.apache.gobblin.util.filesystem

A base FileSystem that uses FileSystemFactory to create an underlying FileSystem with available instrumentations.

InstrumentedFileSystem(String, FileSystem) - Constructor for class org.apache.gobblin.util.filesystem.InstrumentedFileSystem

InstrumentedFileSystemUtils - Class in org.apache.gobblin.util.filesystem

Common methods for Instrumented FileSystems.

InstrumentedFileSystemUtils() - Constructor for class org.apache.gobblin.util.filesystem.InstrumentedFileSystemUtils

InstrumentedHDFSFileSystem - Class in org.apache.gobblin.util.filesystem

A InstrumentedFileSystem for the hdfs scheme.

InstrumentedHDFSFileSystem() - Constructor for class org.apache.gobblin.util.filesystem.InstrumentedHDFSFileSystem

InstrumentedLocalFileSystem - Class in org.apache.gobblin.util.filesystem

A InstrumentedFileSystem for the file scheme.

InstrumentedLocalFileSystem() - Constructor for class org.apache.gobblin.util.filesystem.InstrumentedLocalFileSystem

InstrumentedWebHDFSFileSystem - Class in org.apache.gobblin.util.filesystem

A InstrumentedFileSystem for the webhdfs scheme.

InstrumentedWebHDFSFileSystem() - Constructor for class org.apache.gobblin.util.filesystem.InstrumentedWebHDFSFileSystem

instrumentFileSystem(FileSystem, SharedResourcesBroker<S>, ConfigView<S, FileSystemKey>) - Method in class org.apache.gobblin.util.filesystem.FileSystemInstrumentationFactory

Return an instrumented version of the input FileSystem.

instrumentFileSystem(FileSystem, SharedResourcesBroker<S>, ConfigView<S, FileSystemKey>) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem.Factory

InterruptibleIterator<T> - Class in org.apache.gobblin.util.iterators

An iterator that allows ending prematurely (i.e.

InterruptibleIterator() - Constructor for class org.apache.gobblin.util.iterators.InterruptibleIterator

invokeConstructor(Class<T>, String, Object...) - Static method in class org.apache.gobblin.util.reflection.GobblinConstructorUtils

Utility method to create an instance of clsName using the constructor matching the arguments, args

invokeFirstConstructor(Class<T>, List<Object>...) - Static method in class org.apache.gobblin.util.reflection.GobblinConstructorUtils

Convenience method on top of ConstructorUtils.invokeConstructor(Class, Object[]) that returns a new instance of the cls based on a constructor priority order.

invokeLongestConstructor(Class<T>, Object...) - Static method in class org.apache.gobblin.util.reflection.GobblinConstructorUtils

Returns a new instance of the cls based on a set of arguments.

isAbsoluteAndSchemeAuthorityNull(Path) - Static method in class org.apache.gobblin.util.PathUtils

Is an absolute path (ie a slash relative path part) AND a scheme is null AND authority is null.

isAllowedTransition(T, T) - Method in class org.apache.gobblin.fsm.FiniteStateMachine

isAncestor(Path, Path) - Static method in class org.apache.gobblin.util.PathUtils

Checks whether possibleAncestor is an ancestor of fullPath.

isCompressed(byte[]) - Static method in class org.apache.gobblin.util.io.StreamUtils

isDirectory() - Method in class org.apache.gobblin.util.filesystem.FileStatusEntry

Return whether the path is a directory from the instance FileStatus variable.

isExists() - Method in class org.apache.gobblin.util.filesystem.FileStatusEntry

Indicate whether the file existed the last time it was checked.

isFull() - Method in class org.apache.gobblin.util.request_allocation.ConcurrentBoundedPriorityIterable

isGlob(Path) - Static method in class org.apache.gobblin.util.PathUtils

isSchemaRecursive(Schema, Optional<Logger>) - Static method in class org.apache.gobblin.util.AvroUtils

Check if a schema has recursive fields inside it

isScopeTypeAncestor(S, S) - Static method in class org.apache.gobblin.broker.SharedResourcesBrokerUtils

Determine if a ScopeType is an ancestor of another ScopeType.

isSubPath(File, File) - Static method in class org.apache.gobblin.util.FileUtils

Check if child path is child of parent path.

isSubPath(Path, Path) - Static method in class org.apache.gobblin.util.FileUtils

Check if child path is child of parent path.

isSuccessful() - Method in class org.apache.gobblin.util.callbacks.CallbackResult

isTimeInRange(List<String>, String, String, DateTime) - Static method in class org.apache.gobblin.util.TimeRangeChecker

Checks if a specified time is on a day that is specified the a given List of acceptable days, and that the hours + minutes of the specified time fall into a range defined by startTimeStr and endTimeStr.

isValid() - Method in class org.apache.gobblin.broker.ImmediatelyInvalidResourceEntry

isValid() - Method in class org.apache.gobblin.broker.ResourceInstance

isValid() - Method in class org.apache.gobblin.broker.TTLResourceEntry

iterator() - Method in class org.apache.gobblin.util.EmptyIterable

iterator() - Method in class org.apache.gobblin.util.request_allocation.ConcurrentBoundedPriorityIterable

IteratorExecutor<T> - Class in org.apache.gobblin.util.executors

Executes tasks in an Iterator.

IteratorExecutor(Iterator<Callable<T>>, int, ThreadFactory) - Constructor for class org.apache.gobblin.util.executors.IteratorExecutor

IVY_SETTINGS_FILE_NAME - Static variable in class org.apache.gobblin.util.DownloadUtils

J

JobConfigurationUtils - Class in org.apache.gobblin.util

A utility class for working with job configurations.

JobConfigurationUtils() - Constructor for class org.apache.gobblin.util.JobConfigurationUtils

JobLauncherUtils - Class in org.apache.gobblin.util

Utility class for the job scheduler and job launchers.

JobLauncherUtils() - Constructor for class org.apache.gobblin.util.JobLauncherUtils

JsonUtils - Class in org.apache.gobblin.util.json

JSON related utilities

JsonUtils() - Constructor for class org.apache.gobblin.util.json.JsonUtils

JvmUtils - Class in org.apache.gobblin.util

K

KeyedScopedConfigViewImpl<S extends ScopeType<S>,K extends SharedResourceKey> - Class in org.apache.gobblin.broker

An implementation of ScopedConfigView that knows how to extract relevant subconfiguration from an input Config.

KeyedScopedConfigViewImpl() - Constructor for class org.apache.gobblin.broker.KeyedScopedConfigViewImpl

klazz - Variable in class org.apache.gobblin.runtime.cli.PublicMethodsCliObjectFactory

L

LateFileRecordCountProvider - Class in org.apache.gobblin.util.recordcount

LateFileRecordCountProvider() - Constructor for class org.apache.gobblin.util.recordcount.LateFileRecordCountProvider

LATEST_MOD_TIME_ORDER - Static variable in class org.apache.gobblin.util.FileListUtils

LDAP_BASE_DN_KEY - Static variable in class org.apache.gobblin.util.LdapUtils

LDAP_HOST_KEY - Static variable in class org.apache.gobblin.util.LdapUtils

LDAP_PASSWORD_KEY - Static variable in class org.apache.gobblin.util.LdapUtils

LDAP_PORT_KEY - Static variable in class org.apache.gobblin.util.LdapUtils

LDAP_PREFIX - Static variable in class org.apache.gobblin.util.LdapUtils

LDAP_USE_SECURE_TRUSTMANAGER - Static variable in class org.apache.gobblin.util.LdapUtils

LDAP_USER_KEY - Static variable in class org.apache.gobblin.util.LdapUtils

LdapUtils - Class in org.apache.gobblin.util

This is a utility class for accessing Active Directory.

LdapUtils(Config) - Constructor for class org.apache.gobblin.util.LdapUtils

left(S) - Static method in class org.apache.gobblin.util.Either

Create an instance of Either with value of type S.

Left(S) - Constructor for class org.apache.gobblin.util.Either.Left

Limiter - Interface in org.apache.gobblin.util.limiter

An interface for classes that implement some logic limiting on the occurrences of some events, e.g., data record extraction using an Extractor.

LIMITER_CLASS_KEY - Static variable in class org.apache.gobblin.util.limiter.broker.SharedLimiterFactory

LimiterFactory - Interface in org.apache.gobblin.util.limiter

Interface for factories that build Limiters.

listFilesRecursively(FileSystem, Path) - Static method in class org.apache.gobblin.util.FileListUtils

listFilesRecursively(FileSystem, Iterable<Path>) - Static method in class org.apache.gobblin.util.FileListUtils

listFilesRecursively(FileSystem, Path, PathFilter) - Static method in class org.apache.gobblin.util.FileListUtils

Helper method to list out all files under a specified path.

listFilesRecursively(FileSystem, Path, PathFilter, boolean) - Static method in class org.apache.gobblin.util.FileListUtils

Helper method to list out all files under a specified path.

listFilesToCopyAtPath(FileSystem, Path, PathFilter, boolean) - Static method in class org.apache.gobblin.util.FileListUtils

Given a path to copy, list all files rooted at the given path to copy

listFilesToCopyAtPath(FileSystem, Path, PathFilter, boolean, boolean) - Static method in class org.apache.gobblin.util.FileListUtils

Given a path to copy, list all files rooted at the given path to copy

LISTING_FILES_PER_PERMIT - Static variable in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

Listing operations will use 1 permit per this many listed elements.

listMostNestedPathRecursively(FileSystem, Path) - Static method in class org.apache.gobblin.util.FileListUtils

Method to list out all files, or directory if no file exists, under a specified path.

listMostNestedPathRecursively(FileSystem, Iterable<Path>) - Static method in class org.apache.gobblin.util.FileListUtils

listMostNestedPathRecursively(FileSystem, Path, PathFilter) - Static method in class org.apache.gobblin.util.FileListUtils

Method to list out all files, or directory if no file exists, under a specified path.

listPathsRecursively(FileSystem, Path, PathFilter) - Static method in class org.apache.gobblin.util.FileListUtils

Helper method to list out all paths under a specified path.

listStatus(Path) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

listStatus(Path, PathFilter) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

listStatusRecursive(FileSystem, Path) - Static method in class org.apache.gobblin.util.HadoopUtils

Deprecated.

Use FileListUtils.listFilesRecursively(FileSystem, Path).

LOAD_HADOOP_CONFIGURATION - Static variable in class org.apache.gobblin.broker.SharedResourcesBrokerFactory

loadPullFile(Path, Config, boolean, boolean) - Method in class org.apache.gobblin.util.PullFileLoader

Load a single pull file.

loadPullFile(Path, Config, boolean) - Method in class org.apache.gobblin.util.PullFileLoader

loadPullFilesRecursively(Path, Config, boolean) - Method in class org.apache.gobblin.util.PullFileLoader

Find and load all pull files under a base Path recursively in an order sorted by last modified date.

Log4jConfigurationHelper - Class in org.apache.gobblin.util.logs

A helper class for programmatically configuring log4j.

Log4jConfigurationHelper() - Constructor for class org.apache.gobblin.util.logs.Log4jConfigurationHelper

logAndThrowFailures(List<Either<T, ExecutionException>>, Logger, int) - Static method in class org.apache.gobblin.util.executors.IteratorExecutor

Log failures in the output of IteratorExecutor.executeAndGetResults(), and also propagate exception to upper layer.

LogCopier - Class in org.apache.gobblin.util.logs

A utility service that periodically reads log files in a source log file directory for changes since the last reads and appends the changes to destination log files with the same names as the source log files in a destination log directory.

LogCopier.Builder - Class in org.apache.gobblin.util.logs

A builder class for LogCopier.

logFailures(List<Either<T, ExecutionException>>, Logger, int) - Static method in class org.apache.gobblin.util.executors.IteratorExecutor

Log failures in the output of IteratorExecutor.executeAndGetResults().

loggingDecorator(ExecutorService) - Static method in class org.apache.gobblin.util.ExecutorsUtils

Creates an ListeningExecutorService whose submit and execute methods propagate the MDC information across thread boundaries.

loggingDecorator(ScheduledExecutorService) - Static method in class org.apache.gobblin.util.ExecutorsUtils

Creates an ListeningScheduledExecutorService whose submit, execute, schedule, scheduleAtFixedRate, and scheduleWithFixedDelay methods propagate the MDC information across thread boundaries.

loggingDecorator(Runnable) - Static method in class org.apache.gobblin.util.ExecutorsUtils

Creates an Runnable which propagates the MDC information across thread boundaries.

loggingDecorator(Callable<T>) - Static method in class org.apache.gobblin.util.ExecutorsUtils

Creates an Callable which propagates the MDC information across thread boundaries.

LoggingUncaughtExceptionHandler - Class in org.apache.gobblin.util

A type of Thread.UncaughtExceptionHandler that logs uncaught exceptions.

LoggingUncaughtExceptionHandler(Optional<Logger>) - Constructor for class org.apache.gobblin.util.LoggingUncaughtExceptionHandler

logStatistics(Optional<Logger>) - Method in class org.apache.gobblin.util.request_allocation.ConcurrentBoundedPriorityIterable

Log statistics about this ConcurrentBoundedPriorityIterable.

M

M_OUTPUT_FILE_PREFIX - Static variable in class org.apache.gobblin.util.recordcount.CompactionRecordCountProvider

main(String[]) - Static method in class org.apache.gobblin.runtime.cli.GobblinCli

main(String[]) - Static method in class org.apache.gobblin.util.CLIPasswordEncryptor

mark() - Method in class org.apache.gobblin.util.io.BatchedMeterDecorator

mark(long) - Method in class org.apache.gobblin.util.io.BatchedMeterDecorator

MAX_ACTIVE_CONNS - Static variable in class org.apache.gobblin.util.jdbc.DataSourceProvider

MAX_FILESYSTEM_QPS - Static variable in class org.apache.gobblin.util.HadoopUtils

MAX_IDLE_CONNS - Static variable in class org.apache.gobblin.util.jdbc.DataSourceProvider

MAX_RECORDS_PER_WORK_UNIT - Static variable in class org.apache.gobblin.util.test.FastSequentialSource

MAX_SECONDS_KEY - Static variable in class org.apache.gobblin.util.limiter.TimeBasedLimiter.Factory

MAX_SECONDS_PER_WORK_UNIT - Static variable in class org.apache.gobblin.util.test.FastSequentialSource

maxActiveConnections(int) - Method in class org.apache.gobblin.util.jdbc.DataSourceBuilder

maxIdleConnections(int) - Method in class org.apache.gobblin.util.jdbc.DataSourceBuilder

MAXIMUM_PORT - Static variable in class org.apache.gobblin.util.PortUtils

MDCPropagatingCallable<T> - Class in org.apache.gobblin.util.executors

MDCPropagatingCallable(Callable<T>) - Constructor for class org.apache.gobblin.util.executors.MDCPropagatingCallable

MDCPropagatingExecutorService - Class in org.apache.gobblin.util.executors

MDCPropagatingExecutorService(ExecutorService) - Constructor for class org.apache.gobblin.util.executors.MDCPropagatingExecutorService

MDCPropagatingRunnable - Class in org.apache.gobblin.util.executors

MDCPropagatingRunnable(Runnable) - Constructor for class org.apache.gobblin.util.executors.MDCPropagatingRunnable

MDCPropagatingScheduledExecutorService - Class in org.apache.gobblin.util.executors

MDCPropagatingScheduledExecutorService(ScheduledExecutorService) - Constructor for class org.apache.gobblin.util.executors.MDCPropagatingScheduledExecutorService

MEM_ALLOC_BYTES_KEY - Static variable in class org.apache.gobblin.util.test.StressTestingSource

mergePaths(Path, Path) - Static method in class org.apache.gobblin.util.PathUtils

mergeTemplateWithUserCustomizedFile(Properties, Properties) - Static method in class org.apache.gobblin.util.TemplateUtils

create a complete property file based on the given template

MeteredInputStream - Class in org.apache.gobblin.util.io

A FilterInputStream that counts the bytes read from the underlying InputStream.

MeteredOutputStream - Class in org.apache.gobblin.util.io

A FilterOutputStream that counts the bytes read from the underlying OutputStream.

MeteredOutputStream(OutputStream, Meter, int) - Constructor for class org.apache.gobblin.util.io.MeteredOutputStream

Builds a MeteredOutputStream.

MeteredStream - Interface in org.apache.gobblin.util.io

A InputStream or OutputStream that measures bytes processed.

MINIMUM_PORT - Static variable in class org.apache.gobblin.util.PortUtils

mkdirs(Path, FsPermission) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

mkdirsWithRecursivePermission(FileSystem, Path, FsPermission) - Static method in class org.apache.gobblin.util.WriterUtils

Create the given dir as well as all missing ancestor dirs.

mkdirsWithRecursivePermissionWithRetry(FileSystem, Path, FsPermission, Config) - Static method in class org.apache.gobblin.util.WriterUtils

MockSleeper() - Constructor for class org.apache.gobblin.util.Sleeper.MockSleeper

ModTimeDataFileVersionStrategy - Class in org.apache.gobblin.util.filesystem

An implementation of DataFileVersionStrategy that uses modtime as the file version.

ModTimeDataFileVersionStrategy() - Constructor for class org.apache.gobblin.util.filesystem.ModTimeDataFileVersionStrategy

ModTimeDataFileVersionStrategy.Factory - Class in org.apache.gobblin.util.filesystem

movePath(FileSystem, Path, FileSystem, Path, Configuration) - Static method in class org.apache.gobblin.util.HadoopUtils

Moves a src Path from a srcFs FileSystem to a dst Path on a dstFs FileSystem.

movePath(FileSystem, Path, FileSystem, Path, boolean, Configuration) - Static method in class org.apache.gobblin.util.HadoopUtils

Moves a src Path from a srcFs FileSystem to a dst Path on a dstFs FileSystem.

movePath(Path, FileSystem, Path, Optional<String>) - Method in class org.apache.gobblin.util.ParallelRunner

Move a Path.

movePath(Path, FileSystem, Path, boolean, Optional<String>) - Method in class org.apache.gobblin.util.ParallelRunner

Move a Path.

moveToTrash(FileSystem, Path) - Static method in class org.apache.gobblin.util.HadoopUtils

Moves the object to the filesystem trash according to the file system policy.

MR_OUTPUT_FILE_PREFIX - Static variable in class org.apache.gobblin.util.recordcount.CompactionRecordCountProvider

MultiLimiter - Class in org.apache.gobblin.util.limiter

A Limiter that contains multiple underlying Limiters and requests permits from all those limiters.

MultiLimiter(Limiter...) - Constructor for class org.apache.gobblin.util.limiter.MultiLimiter

N

NAME - Static variable in class org.apache.gobblin.util.io.StreamThrottler.Factory

NAME - Static variable in class org.apache.gobblin.util.limiter.broker.SharedLimiterFactory

NamedFuture() - Constructor for class org.apache.gobblin.util.ParallelRunner.NamedFuture

newAppId(String) - Static method in class org.apache.gobblin.util.ApplicationLauncherUtils

Create a new app ID.

newBuilder() - Static method in class org.apache.gobblin.util.logs.LogCopier

Get a new LogCopier.Builder instance for building a LogCopier.

newChildInstance(Path) - Method in class org.apache.gobblin.util.filesystem.FileStatusEntry

Create a new child instance.

newConfiguration() - Static method in class org.apache.gobblin.util.HadoopUtils

newConnectorWithProps(Properties) - Static method in class org.apache.gobblin.util.HiveJdbcConnector

Create a new HiveJdbcConnector based on the configs in a Properties object

newDaemonThreadFactory(Optional<Logger>, Optional<String>) - Static method in class org.apache.gobblin.util.ExecutorsUtils

Get a new ThreadFactory that uses a LoggingUncaughtExceptionHandler to handle uncaught exceptions, uses the given thread name format, and produces daemon threads.

newEmbeddedConnector(int) - Static method in class org.apache.gobblin.util.HiveJdbcConnector

Create a new HiveJdbcConnector using the specified Hive server version.

newInstance(Config) - Static method in class org.apache.gobblin.util.retry.RetryerFactory

Creates new instance of retryer based on the config.

newJobId(String) - Static method in class org.apache.gobblin.util.JobLauncherUtils

Create a new job ID.

newLimiter(State) - Static method in class org.apache.gobblin.util.limiter.DefaultLimiterFactory

Create a new Limiter instance of one of the types in BaseLimiterType.

newMultiTaskId(String, int) - Static method in class org.apache.gobblin.util.JobLauncherUtils

Create an ID for a new multi-task (corresponding to a MultiWorkUnit) for the job with the given job ID.

newScalingThreadPool(int, int, long) - Static method in class org.apache.gobblin.util.executors.ScalingThreadPoolExecutor

Creates a ScalingThreadPoolExecutor.

newScalingThreadPool(int, int, long, ThreadFactory) - Static method in class org.apache.gobblin.util.executors.ScalingThreadPoolExecutor

Creates a ScalingThreadPoolExecutor.

newSubscopedBuilder(ScopeInstance<S>) - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl

Get a builder to create a descendant SharedResourcesBrokerImpl (i.e.

newTaskId(String, int) - Static method in class org.apache.gobblin.util.JobLauncherUtils

Create a new task ID for the job with the given job ID.

newThreadFactory(Optional<Logger>) - Static method in class org.apache.gobblin.util.ExecutorsUtils

Get a new ThreadFactory that uses a LoggingUncaughtExceptionHandler to handle uncaught exceptions.

newThreadFactory(Optional<Logger>, Optional<String>) - Static method in class org.apache.gobblin.util.ExecutorsUtils

Get a new ThreadFactory that uses a LoggingUncaughtExceptionHandler to handle uncaught exceptions and the given thread name format.

next() - Method in class org.apache.gobblin.util.iterators.InterruptibleIterator

next() - Method in class org.apache.gobblin.util.request_allocation.AllocatedRequestsIteratorBase

NO_OP_CLOSEABLE - Static variable in class org.apache.gobblin.util.limiter.NonRefillableLimiter

NO_OP_PATH_FILTER - Static variable in class org.apache.gobblin.util.FileListUtils

NO_RETRY_CONFIG - Static variable in class org.apache.gobblin.util.WriterUtils

NoMoreRetriesException(String) - Constructor for exception org.apache.gobblin.util.ExponentialBackoff.NoMoreRetriesException

NonRefillableLimiter - Class in org.apache.gobblin.util.limiter

A type of Limiters that do not support permit refills by returning a no-op Closeable in NonRefillableLimiter.acquirePermits(long).

NonRefillableLimiter() - Constructor for class org.apache.gobblin.util.limiter.NonRefillableLimiter

NonTransientException - Exception in org.apache.gobblin.exception

NonTransientException that shows this is a permanent failure where retry cannot solve.

NonTransientException() - Constructor for exception org.apache.gobblin.exception.NonTransientException

NonTransientException(String) - Constructor for exception org.apache.gobblin.exception.NonTransientException

NonTransientException(String, Throwable) - Constructor for exception org.apache.gobblin.exception.NonTransientException

NonTransientException(Throwable) - Constructor for exception org.apache.gobblin.exception.NonTransientException

NoopCloseable - Class in org.apache.gobblin.util

A Closeable that does nothing.

NoopCloseable() - Constructor for class org.apache.gobblin.util.NoopCloseable

NoopLimiter - Class in org.apache.gobblin.util.limiter

A Limiter that satisfies all requests.

NoopLimiter() - Constructor for class org.apache.gobblin.util.limiter.NoopLimiter

NotEnoughPermitsException - Exception in org.apache.gobblin.util.limiter

Indicates there were not enough permits in the Limiter to finish the copy.

NotEnoughPermitsException(String) - Constructor for exception org.apache.gobblin.util.limiter.NotEnoughPermitsException

NotEnoughPermitsException(String, Throwable) - Constructor for exception org.apache.gobblin.util.limiter.NotEnoughPermitsException

NotOnCli - Annotation Type in org.apache.gobblin.runtime.cli

Specifies that the annotated method should not be offered as an option in the CLI.

nullifyFieldsForSchemaMerge(Schema, Schema) - Static method in class org.apache.gobblin.util.AvroUtils

Merge oldSchema and newSchame.

NUM_HELLOS_FULL_KEY - Static variable in class org.apache.gobblin.util.test.HelloWorldSource

NUM_HELLOS_KEY - Static variable in class org.apache.gobblin.util.test.HelloWorldSource

NUM_RECORDS_KEY - Static variable in class org.apache.gobblin.util.test.StressTestingSource

NUM_WORK_UNITS - Static variable in class org.apache.gobblin.util.test.FastSequentialSource

NUM_WORK_UNITS_KEY - Static variable in class org.apache.gobblin.util.test.StressTestingSource

O

obtainKerberosPrincipal(State) - Static method in class org.apache.gobblin.util.hadoop.TokenUtils

Obtain kerberos principal in a dynamic way, where the instance's value is determined by the hostname of the machine that the job is currently running on.

onClose() - Method in class org.apache.gobblin.util.filesystem.FileSystemInstrumentation

A method called when the FileSystem is being closed or when the JVM is shutting down.

onDirectoryChange(Path) - Method in class org.apache.gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator

onDirectoryChange(Path) - Method in interface org.apache.gobblin.util.filesystem.PathAlterationListener

onDirectoryChange(Path) - Method in class org.apache.gobblin.util.filesystem.PathAlterationListenerAdaptor

onDirectoryCreate(Path) - Method in class org.apache.gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator

onDirectoryCreate(Path) - Method in interface org.apache.gobblin.util.filesystem.PathAlterationListener

onDirectoryCreate(Path) - Method in class org.apache.gobblin.util.filesystem.PathAlterationListenerAdaptor

onDirectoryDelete(Path) - Method in class org.apache.gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator

onDirectoryDelete(Path) - Method in interface org.apache.gobblin.util.filesystem.PathAlterationListener

onDirectoryDelete(Path) - Method in class org.apache.gobblin.util.filesystem.PathAlterationListenerAdaptor

onEnterState(T) - Method in interface org.apache.gobblin.fsm.StateWithCallbacks

Called when an FSM reaches this state.

onFileChange(Path) - Method in class org.apache.gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator

onFileChange(Path) - Method in interface org.apache.gobblin.util.filesystem.PathAlterationListener

onFileChange(Path) - Method in class org.apache.gobblin.util.filesystem.PathAlterationListenerAdaptor

onFileCreate(Path) - Method in class org.apache.gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator

onFileCreate(Path) - Method in interface org.apache.gobblin.util.filesystem.PathAlterationListener

onFileCreate(Path) - Method in class org.apache.gobblin.util.filesystem.PathAlterationListenerAdaptor

onFileDelete(Path) - Method in class org.apache.gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator

onFileDelete(Path) - Method in interface org.apache.gobblin.util.filesystem.PathAlterationListener

onFileDelete(Path) - Method in class org.apache.gobblin.util.filesystem.PathAlterationListenerAdaptor

onInvalidate() - Method in class org.apache.gobblin.broker.ImmediatelyInvalidResourceEntry

onInvalidate() - Method in class org.apache.gobblin.broker.ResourceInstance

onInvalidate() - Method in class org.apache.gobblin.broker.TTLResourceEntry

onLeaveState(T) - Method in interface org.apache.gobblin.fsm.StateWithCallbacks

Called when an FSM leaves this state.

onStart(PathAlterationObserver) - Method in class org.apache.gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator

onStart(PathAlterationObserver) - Method in interface org.apache.gobblin.util.filesystem.PathAlterationListener

onStart(PathAlterationObserver) - Method in class org.apache.gobblin.util.filesystem.PathAlterationListenerAdaptor

onStop(PathAlterationObserver) - Method in class org.apache.gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator

onStop(PathAlterationObserver) - Method in interface org.apache.gobblin.util.filesystem.PathAlterationListener

onStop(PathAlterationObserver) - Method in class org.apache.gobblin.util.filesystem.PathAlterationListenerAdaptor

open(Path, int) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

org.apache.gobblin.broker - package org.apache.gobblin.broker

A SharedResourcesBroker is an object that provides and manages objects that are accessed from multiple points in the application, allowing disconnected components to use shared objects, as well as easy user configuration for those objects.

org.apache.gobblin.data.management.copy.hive - package org.apache.gobblin.data.management.copy.hive

org.apache.gobblin.exception - package org.apache.gobblin.exception

org.apache.gobblin.fsm - package org.apache.gobblin.fsm

org.apache.gobblin.runtime.cli - package org.apache.gobblin.runtime.cli

org.apache.gobblin.util - package org.apache.gobblin.util

org.apache.gobblin.util.binpacking - package org.apache.gobblin.util.binpacking

org.apache.gobblin.util.callbacks - package org.apache.gobblin.util.callbacks

This package provides some simple instrumentation to handling callback execution.

org.apache.gobblin.util.concurrent - package org.apache.gobblin.util.concurrent

org.apache.gobblin.util.dataset - package org.apache.gobblin.util.dataset

org.apache.gobblin.util.deprecation - package org.apache.gobblin.util.deprecation

org.apache.gobblin.util.event - package org.apache.gobblin.util.event

org.apache.gobblin.util.eventbus - package org.apache.gobblin.util.eventbus

org.apache.gobblin.util.executors - package org.apache.gobblin.util.executors

org.apache.gobblin.util.filesystem - package org.apache.gobblin.util.filesystem

org.apache.gobblin.util.filters - package org.apache.gobblin.util.filters

org.apache.gobblin.util.guid - package org.apache.gobblin.util.guid

org.apache.gobblin.util.hadoop - package org.apache.gobblin.util.hadoop

org.apache.gobblin.util.http - package org.apache.gobblin.util.http

org.apache.gobblin.util.io - package org.apache.gobblin.util.io

org.apache.gobblin.util.iterators - package org.apache.gobblin.util.iterators

org.apache.gobblin.util.jdbc - package org.apache.gobblin.util.jdbc

org.apache.gobblin.util.json - package org.apache.gobblin.util.json

org.apache.gobblin.util.limiter - package org.apache.gobblin.util.limiter

org.apache.gobblin.util.limiter.broker - package org.apache.gobblin.util.limiter.broker

org.apache.gobblin.util.logs - package org.apache.gobblin.util.logs

org.apache.gobblin.util.recordcount - package org.apache.gobblin.util.recordcount

org.apache.gobblin.util.reflection - package org.apache.gobblin.util.reflection

org.apache.gobblin.util.request_allocation - package org.apache.gobblin.util.request_allocation

org.apache.gobblin.util.retry - package org.apache.gobblin.util.retry

org.apache.gobblin.util.service - package org.apache.gobblin.util.service

org.apache.gobblin.util.test - package org.apache.gobblin.util.test

OTHER_NAMENODES - Static variable in class org.apache.gobblin.util.hadoop.TokenUtils

overrideNameAndNamespace(GenericRecord, String, Optional<Map<String, String>>) - Static method in class org.apache.gobblin.util.AvroUtils

Given a generic record, Override the name and namespace of the schema and return a new generic record

overrideNameAndNamespace(Schema, String, Optional<Map<String, String>>) - Static method in class org.apache.gobblin.util.AvroUtils

Given a input schema, Override the name and namespace of the schema and return a new schema

P

pack(List<WorkUnit>, WorkUnitWeighter) - Method in class org.apache.gobblin.util.binpacking.WorstFitDecreasingBinPacking

PARALLEL_RUNNER_THREADS_KEY - Static variable in class org.apache.gobblin.util.ParallelRunner

parallelize(List<F>, Function<F, T>, int, int, Optional<Logger>) - Static method in class org.apache.gobblin.util.ExecutorsUtils

A utility method to parallelize loops.

ParallelRunner - Class in org.apache.gobblin.util

A class that is responsible for running certain methods in parallel.

ParallelRunner(int, FileSystem) - Constructor for class org.apache.gobblin.util.ParallelRunner

ParallelRunner(int, FileSystem, ParallelRunner.FailPolicy) - Constructor for class org.apache.gobblin.util.ParallelRunner

ParallelRunner.FailPolicy - Enum in org.apache.gobblin.util

Policies indicating how ParallelRunner should handle failure of tasks.

ParallelRunner.NamedFuture - Class in org.apache.gobblin.util

A future with a name / message for reporting.

parse(String) - Static method in enum org.apache.gobblin.util.concurrent.TaskSchedulerType

Return the TaskSchedulerType with the specified name.

parse(String) - Static method in class org.apache.gobblin.util.Id

parseSchemaFromFile(Path, FileSystem) - Static method in class org.apache.gobblin.util.AvroUtils

Parse Avro schema from a schema file.

passWord(String) - Method in class org.apache.gobblin.util.jdbc.DataSourceBuilder

PASSWORD - Static variable in class org.apache.gobblin.util.jdbc.DataSourceProvider

PathAlterationListener - Interface in org.apache.gobblin.util.filesystem

A listener that receives events of general file system modifications.

PathAlterationListenerAdaptor - Class in org.apache.gobblin.util.filesystem

PathAlterationListenerAdaptor() - Constructor for class org.apache.gobblin.util.filesystem.PathAlterationListenerAdaptor

PathAlterationObserver - Class in org.apache.gobblin.util.filesystem

PathAlterationObserver(String) - Constructor for class org.apache.gobblin.util.filesystem.PathAlterationObserver

Construct an observer for the specified directory.

PathAlterationObserver(String, PathFilter) - Constructor for class org.apache.gobblin.util.filesystem.PathAlterationObserver

Construct an observer for the specified directory and file filter.

PathAlterationObserver(Path) - Constructor for class org.apache.gobblin.util.filesystem.PathAlterationObserver

Construct an observer for the specified directory.

PathAlterationObserver(Path, PathFilter) - Constructor for class org.apache.gobblin.util.filesystem.PathAlterationObserver

Construct an observer for the specified directory and file filter.

PathAlterationObserver(FileStatusEntry, PathFilter) - Constructor for class org.apache.gobblin.util.filesystem.PathAlterationObserver

The comparison between path is always case-sensitive in this general file system context.

PathAlterationObserverScheduler - Class in org.apache.gobblin.util.filesystem

A runnable that spawns a monitoring thread triggering any registered PathAlterationObserver at a specified interval.

PathAlterationObserverScheduler() - Constructor for class org.apache.gobblin.util.filesystem.PathAlterationObserverScheduler

PathAlterationObserverScheduler(long) - Constructor for class org.apache.gobblin.util.filesystem.PathAlterationObserverScheduler

PathUtils - Class in org.apache.gobblin.util

PathUtils() - Constructor for class org.apache.gobblin.util.PathUtils

POOL_SIZE_KEY - Static variable in class org.apache.gobblin.util.limiter.PoolBasedLimiter.Factory

PoolBasedLimiter - Class in org.apache.gobblin.util.limiter

An implementation of Limiter that ony allows permits to be acquired from a pool.

PoolBasedLimiter(int) - Constructor for class org.apache.gobblin.util.limiter.PoolBasedLimiter

PoolBasedLimiter.Factory - Class in org.apache.gobblin.util.limiter

PortUtils - Class in org.apache.gobblin.util

PortUtils() - Constructor for class org.apache.gobblin.util.PortUtils

PortUtils.ServerSocketPortLocator - Class in org.apache.gobblin.util

PREFIX - Static variable in class org.apache.gobblin.util.Id.Job

PREFIX - Static variable in class org.apache.gobblin.util.Id.MultiTask

PREFIX - Static variable in class org.apache.gobblin.util.Id.Task

PreOrderAllocator<T extends Request<T>> - Class in org.apache.gobblin.util.request_allocation

A RequestAllocator that operates over PushDownRequestors, getting a pre-ordered iterator of Requests from each Requestor and merging them, stopping as soon as the ResourcePool is exhausted.

PreOrderAllocator(RequestAllocatorConfig<T>) - Constructor for class org.apache.gobblin.util.request_allocation.PreOrderAllocator

PreOrderAllocator.Factory - Class in org.apache.gobblin.util.request_allocation

prettyPrintProperties(Properties) - Static method in class org.apache.gobblin.util.PropertiesUtils

PriorityIterableBasedRequestAllocator<T extends Request<T>> - Class in org.apache.gobblin.util.request_allocation

PriorityIterableBasedRequestAllocator(Logger, RequestAllocatorConfig<T>) - Constructor for class org.apache.gobblin.util.request_allocation.PriorityIterableBasedRequestAllocator

PROPERTIES_TO_TAGS_MAP - Static variable in class org.apache.gobblin.util.AzkabanTags

propertiesToConfig(Properties) - Static method in class org.apache.gobblin.util.ConfigUtils

Convert a given Properties to a Config instance.

propertiesToConfig(Properties, Optional<String>) - Static method in class org.apache.gobblin.util.ConfigUtils

Convert all the keys that start with a prefix in Properties to a Config instance.

propertiesToTypedConfig(Properties, Optional<String>) - Static method in class org.apache.gobblin.util.ConfigUtils

Convert all the keys that start with a prefix in Properties to a Config instance.

PropertiesUtils - Class in org.apache.gobblin.util

A utility class for Properties objects.

PropertiesUtils() - Constructor for class org.apache.gobblin.util.PropertiesUtils

PROPERTY_DELIMITER_PARSING_ENABLED_KEY - Static variable in class org.apache.gobblin.util.PullFileLoader

propsToStringKeyMap(Properties) - Static method in class org.apache.gobblin.util.PropertiesUtils

Converts a Properties object to a Map where each key is a String.

ProxiedFileSystemCache - Class in org.apache.gobblin.util

A cache for storing a mapping between Hadoop users and user FileSystem objects.

ProxiedFileSystemCache() - Constructor for class org.apache.gobblin.util.ProxiedFileSystemCache

ProxiedFileSystemUtils - Class in org.apache.gobblin.util

Utility class for creating FileSystem objects while proxied as another user.

ProxiedFileSystemUtils() - Constructor for class org.apache.gobblin.util.ProxiedFileSystemUtils

ProxiedFileSystemUtils.AuthType - Enum in org.apache.gobblin.util

ProxiedFileSystemWrapper - Class in org.apache.gobblin.util

Deprecated.

ProxiedFileSystemWrapper() - Constructor for class org.apache.gobblin.util.ProxiedFileSystemWrapper

Deprecated.

ProxiedFileSystemWrapper.AuthType - Enum in org.apache.gobblin.util

Deprecated.

Use ProxiedFileSystemUtils.AuthType.

PublicMethodsCliObjectFactory<T> - Class in org.apache.gobblin.runtime.cli

A helper class for automatically inferring Options from the public methods in a class.

PublicMethodsCliObjectFactory(Class<? extends T>) - Constructor for class org.apache.gobblin.runtime.cli.PublicMethodsCliObjectFactory

PublisherUtils - Class in org.apache.gobblin.util

Utility class for DataPublisher.

PublisherUtils() - Constructor for class org.apache.gobblin.util.PublisherUtils

PublisherUtils.SplitExtractsResult - Class in org.apache.gobblin.util

PublisherUtils.WorkUnitStateSuccess - Class in org.apache.gobblin.util

Implementation of Predicate that checks if a given WorkUnitState has a WorkUnitState.WorkingState equal to WorkUnitState.WorkingState.SUCCESSFUL.

PullFileLoader - Class in org.apache.gobblin.util

Used to load pull files from the file system.

PullFileLoader(Path, FileSystem, Collection<String>, Collection<String>) - Constructor for class org.apache.gobblin.util.PullFileLoader

PushDownRequestor<T extends Request> - Interface in org.apache.gobblin.util.request_allocation

A Requestor that can provide an Iterator of Requests already sorted by the input prioritizer.

putConfigurationIntoProperties(Configuration, Properties) - Static method in class org.apache.gobblin.util.JobConfigurationUtils

Put all configuration properties in a given Configuration object into a given Properties object.

putPropertiesIntoConfiguration(Properties, Configuration) - Static method in class org.apache.gobblin.util.JobConfigurationUtils

Put all configuration properties in a given Properties object into a given Configuration object.

putStateIntoConfiguration(State, Configuration) - Static method in class org.apache.gobblin.util.JobConfigurationUtils

Put all configuration properties in a given State object into a given Configuration object.

Q

QPS_KEY - Static variable in class org.apache.gobblin.util.limiter.RateBasedLimiter.Factory

R

random() - Method in class org.apache.gobblin.util.PortUtils.ServerSocketPortLocator

RANDOM - Static variable in class org.apache.gobblin.util.request_allocation.AllocatedRequestsIteratorBase.RequestWithResourceRequirement

RateBasedLimiter - Class in org.apache.gobblin.util.limiter

An implementation of Limiter that limits the rate of some events.

RateBasedLimiter(double) - Constructor for class org.apache.gobblin.util.limiter.RateBasedLimiter

RateBasedLimiter(double, TimeUnit) - Constructor for class org.apache.gobblin.util.limiter.RateBasedLimiter

RateBasedLimiter.Factory - Class in org.apache.gobblin.util.limiter

RateControlledFileSystem - Class in org.apache.gobblin.util

Deprecated.

use ThrottledFileSystem

RateControlledFileSystem(FileSystem, long) - Constructor for class org.apache.gobblin.util.RateControlledFileSystem

Deprecated.

read() - Method in class org.apache.gobblin.util.io.EmptyInputStream

read() - Method in class org.apache.gobblin.util.io.MeteredInputStream

read(byte[], int, int) - Method in class org.apache.gobblin.util.io.MeteredInputStream

read(byte[], int, int) - Method in class org.apache.gobblin.util.io.SeekableFSInputStream

read() - Method in class org.apache.gobblin.util.io.SeekableFSInputStream

read() - Method in class org.apache.gobblin.util.io.ThrottledInputStream

read(byte[]) - Method in class org.apache.gobblin.util.io.ThrottledInputStream

read(byte[], int, int) - Method in class org.apache.gobblin.util.io.ThrottledInputStream

readAllRecords(String, String) - Static method in class org.apache.gobblin.util.test.TestIOUtils

Reads all records from a json file as GenericRecords

readFrom(Path) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the path of the source log file directory to read from.

readFrom(List<Path>) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the paths of the source log file directories to read from.

readRecord(Long) - Method in class org.apache.gobblin.util.test.FastSequentialSource.FastSequentialExtractor

readRecord(String) - Method in class org.apache.gobblin.util.test.HelloWorldSource.ExtractorImpl

readRecord(byte[]) - Method in class org.apache.gobblin.util.test.StressTestingSource.ExtractorImpl

Read a record with configurable idle and compute time.

readRecord(String) - Method in class org.apache.gobblin.util.test.TestingSource.Extract

recordsWritten() - Method in class org.apache.gobblin.util.test.StdoutWriter

recordToByteArray(GenericRecord) - Static method in class org.apache.gobblin.util.AvroUtils

Convert a GenericRecord to a byte array.

ReentrantStableStateWait(String) - Constructor for exception org.apache.gobblin.fsm.FiniteStateMachine.ReentrantStableStateWait

refresh(Path) - Method in class org.apache.gobblin.util.filesystem.FileStatusEntry

REGEX - Static variable in class org.apache.gobblin.util.filters.RegexPathFilter

RegexPathFilter - Class in org.apache.gobblin.util.filters

Use a regex to filter Path.

RegexPathFilter(Properties) - Constructor for class org.apache.gobblin.util.filters.RegexPathFilter

RegexPathFilter(String) - Constructor for class org.apache.gobblin.util.filters.RegexPathFilter

RegexPathFilter(String, boolean) - Constructor for class org.apache.gobblin.util.filters.RegexPathFilter

registerImplicitBroker(SharedResourcesBroker<?>) - Static method in class org.apache.gobblin.broker.SharedResourcesBrokerFactory

Register a SharedResourcesBroker to be used as the implicit broker for this and all new children threads.

rejectedExecution(Runnable, ThreadPoolExecutor) - Method in class org.apache.gobblin.util.executors.ForceQueuePolicy

relativizePath(Path, Path) - Static method in class org.apache.gobblin.util.PathUtils

remove() - Method in class org.apache.gobblin.util.iterators.InterruptibleIterator

remove() - Method in class org.apache.gobblin.util.request_allocation.AllocatedRequestsIteratorBase

removeExtension(Path, String...) - Static method in class org.apache.gobblin.util.PathUtils

Removes all extensions from path if they exist.

removeListener(L) - Method in class org.apache.gobblin.util.callbacks.CallbacksDispatcher

removeListener(PathAlterationListener) - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserver

Remove a file system listener.

removeObserver(PathAlterationObserver) - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserverScheduler

Remove a file system observer from this monitor.

removeUncomparableFields(Schema) - Static method in class org.apache.gobblin.util.AvroUtils

Remove map, array, enum fields, as well as union fields that contain map, array or enum, from an Avro schema.

rename(Path, Path) - Method in class org.apache.gobblin.util.filesystem.ThrottledFileSystem

renameDeprecatedKeys(State, String, List<String>) - Static method in class org.apache.gobblin.util.deprecation.DeprecationUtils

Sets an option in a State to the first available value of a list of deprecatedKeys.

renamePath(FileContext, Path, Path) - Static method in class org.apache.gobblin.util.HadoopUtils

A wrapper around FileContext.rename(Path, Path, Options.Rename...).

renamePath(FileContext, Path, Path, boolean) - Static method in class org.apache.gobblin.util.HadoopUtils

A wrapper around FileContext.rename(Path, Path, Options.Rename...)}.

renamePath(FileSystem, Path, Path) - Static method in class org.apache.gobblin.util.HadoopUtils

A wrapper around FileSystem.rename(Path, Path) which throws IOException if FileSystem.rename(Path, Path) returns False.

renamePath(FileSystem, Path, Path, boolean) - Static method in class org.apache.gobblin.util.HadoopUtils

A wrapper around FileSystem.rename(Path, Path) which throws IOException if FileSystem.rename(Path, Path) returns False.

renamePath(Path, Path, Optional<String>) - Method in class org.apache.gobblin.util.ParallelRunner

Rename a Path.

renamePathHandleLocalFSRace(FileSystem, Path, Path) - Static method in class org.apache.gobblin.util.HadoopUtils

Renames a src Path on fs FileSystem to a dst Path.

renameRecursively(FileSystem, Path, Path) - Static method in class org.apache.gobblin.util.HadoopUtils

This method is an additive implementation of the FileSystem.rename(Path, Path) method.

replacePortTokens(String) - Method in class org.apache.gobblin.util.PortUtils

Replaces any port tokens in the specified string.

replaceScheme(URI, String, String) - Static method in class org.apache.gobblin.util.filesystem.InstrumentedFileSystemUtils

Replace the scheme of the input URI if it matches the string to replace.

replaceScheme(Path, String, String) - Static method in class org.apache.gobblin.util.filesystem.InstrumentedFileSystemUtils

Replace the scheme of the input Path if it matches the string to replace.

replaceScheme(Path[], String, String) - Static method in class org.apache.gobblin.util.filesystem.InstrumentedFileSystemUtils

Replace the scheme of each Path if it matches the string to replace.

replaceScheme(FileStatus[], String, String) - Static method in class org.apache.gobblin.util.filesystem.InstrumentedFileSystemUtils

Replace the scheme of each FileStatus if it matches the string to replace.

replaceScheme(FileStatus, String, String) - Static method in class org.apache.gobblin.util.filesystem.InstrumentedFileSystemUtils

Replace the scheme of the input FileStatus if it matches the string to replace.

Request<T extends Request> - Interface in org.apache.gobblin.util.request_allocation

Represents an expensive request in the request allocation problem.

RequestAllocator<T extends Request<T>> - Interface in org.apache.gobblin.util.request_allocation

This interface is intended to solve the problem of selecting a subset of expensive requests based on priority when there are limited resources to perform such requests.

RequestAllocator.Factory - Interface in org.apache.gobblin.util.request_allocation

RequestAllocatorConfig<T extends Request<T>> - Class in org.apache.gobblin.util.request_allocation

RequestAllocatorConfig() - Constructor for class org.apache.gobblin.util.request_allocation.RequestAllocatorConfig

RequestAllocatorConfig.AllEqualPrioritizer<T> - Class in org.apache.gobblin.util.request_allocation

RequestAllocatorConfig.Builder<T extends Request<T>> - Class in org.apache.gobblin.util.request_allocation

RequestAllocatorConfig.StoreRejectedRequestsConfig - Enum in org.apache.gobblin.util.request_allocation

RequestAllocatorUtils - Class in org.apache.gobblin.util.request_allocation

RequestAllocatorUtils() - Constructor for class org.apache.gobblin.util.request_allocation.RequestAllocatorUtils

Requestor<T extends Request> - Interface in org.apache.gobblin.util.request_allocation

A wrapper around a Iterator of Requests used for request allocation problem.

reset() - Method in class org.apache.gobblin.util.io.ThrottledInputStream

reset() - Method in class org.apache.gobblin.util.Sleeper.MockSleeper

resolveEncrypted(Config, Optional<String>) - Static method in class org.apache.gobblin.util.ConfigUtils

Resolves encrypted config value(s) by considering on the path with "encConfigPath" as encrypted.

RESOURCE_LIMITED_PREFIX - Static variable in class org.apache.gobblin.util.filesystem.FileSystemLimiterKey

ResourceEstimator<T> - Interface in org.apache.gobblin.util.request_allocation

Computes the ResourceRequirement for Requests.

ResourceEstimator.Factory<T> - Interface in org.apache.gobblin.util.request_allocation

ResourceInstance<T> - Class in org.apache.gobblin.broker

A SharedResourceFactoryResponse that returns a newly created resource instance.

ResourceInstance() - Constructor for class org.apache.gobblin.broker.ResourceInstance

ResourcePool - Class in org.apache.gobblin.util.request_allocation

Represents a pool of available resources for a RequestAllocator.

ResourcePool(Map<String, Double>, Map<String, Double>, Map<String, Double>) - Constructor for class org.apache.gobblin.util.request_allocation.ResourcePool

ResourcePool(ResourcePool) - Constructor for class org.apache.gobblin.util.request_allocation.ResourcePool

ResourceRequirement - Class in org.apache.gobblin.util.request_allocation

Represents a requirement of resources in a ResourcePool.

ResourceRequirement(ResourceRequirement) - Constructor for class org.apache.gobblin.util.request_allocation.ResourceRequirement

ResourceRequirement.Builder - Class in org.apache.gobblin.util.request_allocation

restoreFilePath(Path) - Static method in class org.apache.gobblin.util.recordcount.LateFileRecordCountProvider

Remove the late components in the path added by LateFileRecordCountProvider.

RestrictedFieldAccessingUtils - Class in org.apache.gobblin.util.reflection

These are hacky methods that should only be used when there are access-modifiers prevents accessing fields that are essential for application code to access.

RETRY_INTERVAL_MS - Static variable in class org.apache.gobblin.util.retry.RetryerFactory

RETRY_MULTIPLIER - Static variable in class org.apache.gobblin.util.retry.RetryerFactory

RETRY_TIME_OUT_MS - Static variable in class org.apache.gobblin.util.retry.RetryerFactory

RETRY_TIMES - Static variable in class org.apache.gobblin.util.retry.RetryerFactory

RETRY_TYPE - Static variable in class org.apache.gobblin.util.retry.RetryerFactory

RetryerFactory<T> - Class in org.apache.gobblin.util.retry

Factory class that builds Retryer.

RetryerFactory() - Constructor for class org.apache.gobblin.util.retry.RetryerFactory

RetryerFactory.RetryType - Enum in org.apache.gobblin.util.retry

right(T) - Static method in class org.apache.gobblin.util.Either

Create an instance of Either with value of type T.

Right(T) - Constructor for class org.apache.gobblin.util.Either.Right

run(String[]) - Method in interface org.apache.gobblin.runtime.cli.CliApplication

Run this application.

run() - Method in class org.apache.gobblin.util.executors.MDCPropagatingRunnable

run() - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserverScheduler

RUN_DURATION_KEY - Static variable in class org.apache.gobblin.util.test.StressTestingSource

runOneIteration() - Method in interface org.apache.gobblin.util.concurrent.ScheduledTask

Run one iteration of the scheduled task.

runOneIteration() - Method in class org.apache.gobblin.util.logs.LogCopier

S

s - Variable in class org.apache.gobblin.util.Either

safeRenameIfNotExists(FileSystem, Path, Path) - Static method in class org.apache.gobblin.util.HadoopUtils

Renames from to to if to doesn't exist in a thread-safe way.

safeRenameRecursively(FileSystem, Path, Path) - Static method in class org.apache.gobblin.util.HadoopUtils

A thread safe variation of HadoopUtils.renamePath(FileSystem, Path, Path) which can be used in multi-threaded/multi-mapper environment.

sanitizedKey(String) - Static method in class org.apache.gobblin.util.ConfigUtils

returns true if is it a sanitized key

sanitizeFullPrefixKey(String) - Static method in class org.apache.gobblin.util.ConfigUtils

sanitizePath(String, String) - Static method in class org.apache.gobblin.util.HadoopUtils

Remove illegal HDFS path characters from the given path.

sanitizePath(Path, String) - Static method in class org.apache.gobblin.util.HadoopUtils

Remove illegal HDFS path characters from the given path.

sanitizeSchemaString(String) - Static method in class org.apache.gobblin.util.AvroUtils

Escaping "\", """, ";" and "'" character in the schema string when it is being used in DDL.

saveConfigToFile(Config, Path) - Method in class org.apache.gobblin.util.ConfigUtils

saveToFile(String, Path) - Method in class org.apache.gobblin.util.FileUtils

ScalingThreadPoolExecutor - Class in org.apache.gobblin.util.executors

ThreadPoolExecutor that scales from a min to a max number of threads as tasks get added.

schedule(T, long, TimeUnit) - Method in class org.apache.gobblin.util.concurrent.TaskScheduler

Schedules a subclass of ScheduledTask to run periodically.

schedule(Runnable, long, TimeUnit) - Method in class org.apache.gobblin.util.executors.MDCPropagatingScheduledExecutorService

schedule(Callable<V>, long, TimeUnit) - Method in class org.apache.gobblin.util.executors.MDCPropagatingScheduledExecutorService

scheduleAtFixedRate(Runnable, long, long, TimeUnit) - Method in class org.apache.gobblin.util.executors.MDCPropagatingScheduledExecutorService

ScheduledTask<K> - Interface in org.apache.gobblin.util.concurrent

A task which can be scheduled to run recurrently by an instance of TaskScheduler.

scheduler() - Method in class org.apache.gobblin.util.logs.LogCopier

scheduleWithFixedDelay(Runnable, long, long, TimeUnit) - Method in class org.apache.gobblin.util.executors.MDCPropagatingScheduledExecutorService

SchemaEntry() - Constructor for class org.apache.gobblin.util.AvroUtils.SchemaEntry

SCHEME - Static variable in class org.apache.gobblin.util.filesystem.InstrumentedHDFSFileSystem

SCHEME - Static variable in class org.apache.gobblin.util.filesystem.InstrumentedLocalFileSystem

SCHEME - Static variable in class org.apache.gobblin.util.filesystem.InstrumentedWebHDFSFileSystem

seek(long) - Method in class org.apache.gobblin.util.io.SeekableFSInputStream

SeekableFSInputStream - Class in org.apache.gobblin.util.io

Class that wraps an InputStream to support Seekable and PositionedReadable

SeekableFSInputStream(InputStream) - Constructor for class org.apache.gobblin.util.io.SeekableFSInputStream

seekToNewSource(long) - Method in class org.apache.gobblin.util.io.SeekableFSInputStream

selfScope() - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl

sendEmail(State, String, String) - Static method in class org.apache.gobblin.util.EmailUtils

A general method for sending emails.

sendJobCancellationEmail(String, String, State) - Static method in class org.apache.gobblin.util.EmailUtils

Send a job cancellation notification email.

sendJobCompletionEmail(String, String, String, State) - Static method in class org.apache.gobblin.util.EmailUtils

Send a job completion notification email.

sendJobFailureAlertEmail(String, String, int, State) - Static method in class org.apache.gobblin.util.EmailUtils

Send a job failure alert email.

SEPARATOR - Static variable in class org.apache.gobblin.util.Id

SerializationUtils - Class in org.apache.gobblin.util

A utility class for serializing and deserializing Objects to/from Strings.

SerializationUtils() - Constructor for class org.apache.gobblin.util.SerializationUtils

serialize(Properties) - Static method in class org.apache.gobblin.util.PropertiesUtils

serialize(T) - Static method in class org.apache.gobblin.util.SerializationUtils

Serialize an object into a String.

serialize(T, BaseEncoding) - Static method in class org.apache.gobblin.util.SerializationUtils

Serialize an object into a String.

serializeAsPath(GenericRecord, boolean, boolean) - Static method in class org.apache.gobblin.util.AvroUtils

Serialize a generic record as a relative Path.

serializeIntoBytes(T) - Static method in class org.apache.gobblin.util.SerializationUtils

Serialize an object into a byte array.

serializeState(FileSystem, Path, T) - Static method in class org.apache.gobblin.util.SerializationUtils

Serialize a State instance to a file.

serializeState(FileSystem, Path, T, short) - Static method in class org.apache.gobblin.util.SerializationUtils

Serialize a State instance to a file.

serializeToFile(T, Path) - Method in class org.apache.gobblin.util.ParallelRunner

Serialize a State object into a file.

serializeToString(Writable) - Static method in class org.apache.gobblin.util.HadoopUtils

Serialize a Writable object into a string.

serializeWriterDirPermissions(State, int, int, FsPermission) - Static method in class org.apache.gobblin.util.HadoopUtils

Given a FsPermission objects, set a key, value pair in the given State for the writer to use when creating files.

serializeWriterFilePermissions(State, int, int, FsPermission) - Static method in class org.apache.gobblin.util.HadoopUtils

Given a FsPermission objects, set a key, value pair in the given State for the writer to use when creating files.

ServerSocketPortLocator() - Constructor for class org.apache.gobblin.util.PortUtils.ServerSocketPortLocator

serviceName - Variable in class org.apache.gobblin.util.filesystem.FileSystemLimiterKey

set() - Method in class org.apache.gobblin.util.concurrent.AutoResetEvent

setChildren(FileStatusEntry[]) - Method in class org.apache.gobblin.util.filesystem.FileStatusEntry

Set the directory's files.

setDefaultVersion(Path) - Method in interface org.apache.gobblin.util.filesystem.DataFileVersionStrategy

Set the version of a path to a value automatically set by the versioning system.

setDefaultVersion(Path) - Method in class org.apache.gobblin.util.filesystem.ModTimeDataFileVersionStrategy

setGroup(FileSystem, Path, String) - Static method in class org.apache.gobblin.util.HadoopUtils

Set the group associated with a given path.

setPermissions(Path, Optional<String>, Optional<String>, FileSystem, FsPermission) - Static method in class org.apache.gobblin.util.HadoopUtils

Try to set owner and permissions for the path.

setProxiedFileSystem(FileSystem) - Method in class org.apache.gobblin.util.ProxiedFileSystemWrapper

Deprecated.

Setter for proxiedFs.

setRequirement(String, double) - Method in class org.apache.gobblin.util.request_allocation.ResourceRequirement.Builder

Specify the resource requirement along a dimension.

setSchemaCreationTime(Schema, String) - Static method in class org.apache.gobblin.util.AvroUtils

setVersion(Path, T) - Method in interface org.apache.gobblin.util.filesystem.DataFileVersionStrategy

Set the version of a path to a specific version (generally replicated from another path).

setVersion(Path, Long) - Method in class org.apache.gobblin.util.filesystem.ModTimeDataFileVersionStrategy

setWriterDirOctalPermissions(State, int, int, String) - Static method in class org.apache.gobblin.util.HadoopUtils

Given a String in octal notation, set a key, value pair in the given State for the writer to use when creating directories.

setWriterFileOctalPermissions(State, int, int, String) - Static method in class org.apache.gobblin.util.HadoopUtils

Given a String in octal notation, set a key, value pair in the given State for the writer to use when creating files.

SharedLimiterFactory<S extends ScopeType<S>> - Class in org.apache.gobblin.util.limiter.broker

A SharedResourceFactory to create shared Limiters using a SharedResourcesBroker.

SharedLimiterFactory() - Constructor for class org.apache.gobblin.util.limiter.broker.SharedLimiterFactory

SharedLimiterKey - Class in org.apache.gobblin.util.limiter.broker

A SharedResourceKey for use with SharedLimiterFactory.

SharedLimiterKey(String) - Constructor for class org.apache.gobblin.util.limiter.broker.SharedLimiterKey

SharedLimiterKey(String, SharedLimiterKey.GlobalLimiterPolicy) - Constructor for class org.apache.gobblin.util.limiter.broker.SharedLimiterKey

SharedLimiterKey.GlobalLimiterPolicy - Enum in org.apache.gobblin.util.limiter.broker

SharedResourcesBrokerFactory - Class in org.apache.gobblin.broker

Used to create a default implementation of SharedResourcesBroker.

SharedResourcesBrokerFactory() - Constructor for class org.apache.gobblin.broker.SharedResourcesBrokerFactory

SharedResourcesBrokerImpl<S extends ScopeType<S>> - Class in org.apache.gobblin.broker

An implementation of SharedResourcesBroker using a DefaultBrokerCache for storing shared objects.

SharedResourcesBrokerImpl.SubscopedBrokerBuilder - Class in org.apache.gobblin.broker

A builder used to create a descendant SharedResourcesBrokerImpl with the same backing DefaultBrokerCache.

SharedResourcesBrokerUtils - Class in org.apache.gobblin.broker

General utilities for SharedResourcesBroker functionality.

SharedResourcesBrokerUtils() - Constructor for class org.apache.gobblin.broker.SharedResourcesBrokerUtils

shutDown() - Method in class org.apache.gobblin.util.logs.LogCopier

shutdown(SourceState) - Method in class org.apache.gobblin.util.test.FastSequentialSource

shutdown(SourceState) - Method in class org.apache.gobblin.util.test.HelloWorldSource

shutdown(SourceState) - Method in class org.apache.gobblin.util.test.StressTestingSource

shutdown(SourceState) - Method in class org.apache.gobblin.util.test.TestingSource

SHUTDOWN_PREFIX - Static variable in class org.apache.gobblin.util.service.StandardServiceConfig

SHUTDOWN_TIMEOUT_MS_PROP - Static variable in class org.apache.gobblin.util.service.StandardServiceConfig

shutdownExecutorService(ExecutorService, Optional<Logger>, long, TimeUnit) - Static method in class org.apache.gobblin.util.ExecutorsUtils

Shutdown an ExecutorService gradually, first disabling new task submissions and later cancelling existing tasks.

shutdownExecutorService(ExecutorService, Optional<Logger>) - Static method in class org.apache.gobblin.util.ExecutorsUtils

Shutdown an ExecutorService gradually, first disabling new task submissions and later cancelling existing tasks.

shutdownObject(Object, Logger) - Static method in class org.apache.gobblin.broker.SharedResourcesBrokerUtils

Close Closeables and shutdown Services.

SimpleHierarchicalPrioritizer<T extends Request<T>> - Class in org.apache.gobblin.util.request_allocation

A HierarchicalPrioritizer built from two input Comparators: one for Requestor and one for Request.

SimpleHierarchicalPrioritizer() - Constructor for class org.apache.gobblin.util.request_allocation.SimpleHierarchicalPrioritizer

SKIP_GLOBAL_LIMITER_KEY - Static variable in class org.apache.gobblin.util.limiter.broker.SharedLimiterFactory

Skip use of global limiter.

sleep(long) - Method in class org.apache.gobblin.util.Sleeper.MockSleeper

sleep(long) - Method in class org.apache.gobblin.util.Sleeper

Equivalent to Thread.sleep(long).

SLEEP_TIME_MICRO_KEY - Static variable in class org.apache.gobblin.util.test.StressTestingSource

Sleeper - Class in org.apache.gobblin.util

A class surrounding Thread.sleep(long) that allows mocking sleeps during testing.

Sleeper() - Constructor for class org.apache.gobblin.util.Sleeper

Sleeper.MockSleeper - Class in org.apache.gobblin.util

A mock version of Sleeper that just register calls to sleep but returns immediately.

slowDeserializeGenericRecord(byte[], Schema) - Static method in class org.apache.gobblin.util.AvroUtils

Deserialize a GenericRecord from a byte array.

specific(int) - Method in class org.apache.gobblin.util.PortUtils.ServerSocketPortLocator

splitExtractsByPredicate(Multimap<Extract, WorkUnitState>, Predicate<WorkUnitState>) - Static method in class org.apache.gobblin.util.PublisherUtils

Given a Multimap of Extracts to WorkUnitStates, filter out any Extracts where all of the corresponding WorkUnitStates do not meet the given Predicate.

SplitExtractsResult() - Constructor for class org.apache.gobblin.util.PublisherUtils.SplitExtractsResult

StandardServiceConfig - Class in org.apache.gobblin.util.service

A wrapper around a typesafe Config object to provide standard configuration for services.

StandardServiceConfig(Config) - Constructor for class org.apache.gobblin.util.service.StandardServiceConfig

Constructor from a typesafe config object

start() - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserverScheduler

Start monitoring.

start(List<String>) - Method in class org.apache.gobblin.util.GobblinProcessBuilder

start() - Method in class org.apache.gobblin.util.limiter.CountBasedLimiter

start() - Method in interface org.apache.gobblin.util.limiter.Limiter

Start the Limiter.

start() - Method in class org.apache.gobblin.util.limiter.MultiLimiter

start() - Method in class org.apache.gobblin.util.limiter.NoopLimiter

start() - Method in class org.apache.gobblin.util.limiter.PoolBasedLimiter

start() - Method in class org.apache.gobblin.util.limiter.RateBasedLimiter

start() - Method in class org.apache.gobblin.util.limiter.TimeBasedLimiter

startRateControl() - Method in class org.apache.gobblin.util.RateControlledFileSystem

Deprecated.

startTransition(T) - Method in class org.apache.gobblin.fsm.FiniteStateMachine

Start a transition to the end state specified.

STARTUP_PREFIX - Static variable in class org.apache.gobblin.util.service.StandardServiceConfig

STARTUP_TIMEOUT_MS_PROP - Static variable in class org.apache.gobblin.util.service.StandardServiceConfig

state(State) - Method in class org.apache.gobblin.util.jdbc.DataSourceBuilder

StateWithCallbacks<T> - Interface in org.apache.gobblin.fsm

A state for a FiniteStateMachine which supports callbacks when entering and leaving the state.

StdoutWriter<D> - Class in org.apache.gobblin.util.test

A simple writer implementation that writes the output to Stdout

StdoutWriter() - Constructor for class org.apache.gobblin.util.test.StdoutWriter

StdoutWriter.Builder<D> - Class in org.apache.gobblin.util.test

stop() - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserverScheduler

Stop monitoring

stop(long) - Method in class org.apache.gobblin.util.filesystem.PathAlterationObserverScheduler

Stop monitoring

stop() - Method in class org.apache.gobblin.util.limiter.CountBasedLimiter

stop() - Method in interface org.apache.gobblin.util.limiter.Limiter

Stop the Limiter.

stop() - Method in class org.apache.gobblin.util.limiter.MultiLimiter

stop() - Method in class org.apache.gobblin.util.limiter.NoopLimiter

stop() - Method in class org.apache.gobblin.util.limiter.PoolBasedLimiter

stop() - Method in class org.apache.gobblin.util.limiter.RateBasedLimiter

stop() - Method in class org.apache.gobblin.util.limiter.TimeBasedLimiter

storeRejectedRequests(String) - Method in class org.apache.gobblin.util.request_allocation.RequestAllocatorConfig.Builder

STREAM_COPIER_KEY_PREFIX - Static variable in class org.apache.gobblin.util.io.StreamCopierSharedLimiterKey

StreamCopier - Class in org.apache.gobblin.util.io

A class that copies an InputStream to an OutputStream in a configurable way.

StreamCopier(InputStream, OutputStream) - Constructor for class org.apache.gobblin.util.io.StreamCopier

StreamCopier(InputStream, OutputStream, Long) - Constructor for class org.apache.gobblin.util.io.StreamCopier

StreamCopier(ReadableByteChannel, WritableByteChannel) - Constructor for class org.apache.gobblin.util.io.StreamCopier

StreamCopier(ReadableByteChannel, WritableByteChannel, Long) - Constructor for class org.apache.gobblin.util.io.StreamCopier

StreamCopier.NotEnoughPermitsException - Exception in org.apache.gobblin.util.io

Indicates there were not enough permits in the Limiter to finish the copy.

StreamCopierSharedLimiterKey - Class in org.apache.gobblin.util.io

A subclass of SharedLimiterKey with more detailed information on the source and target of a throttled StreamCopier.

StreamCopierSharedLimiterKey(URI, URI) - Constructor for class org.apache.gobblin.util.io.StreamCopierSharedLimiterKey

StreamThrottler<S extends ScopeType<S>> - Class in org.apache.gobblin.util.io

A class used to throttle InputStreams.

StreamThrottler() - Constructor for class org.apache.gobblin.util.io.StreamThrottler

StreamThrottler.Factory<S extends ScopeType<S>> - Class in org.apache.gobblin.util.io

A SharedResourceFactory that creates StreamThrottler.

StreamUtils - Class in org.apache.gobblin.util.io

Utility class of input/output stream helpers.

StreamUtils() - Constructor for class org.apache.gobblin.util.io.StreamUtils

StressTestingSource - Class in org.apache.gobblin.util.test

A Source to be used for stress testing This source uses an extractor that can be configured to have sleep and computation time before returning a record.

StressTestingSource() - Constructor for class org.apache.gobblin.util.test.StressTestingSource

StressTestingSource.ExtractorImpl - Class in org.apache.gobblin.util.test

stringifyRequirement(ResourceRequirement) - Method in class org.apache.gobblin.util.request_allocation.ResourcePool

Stringify a ResourceRequirement with the appropriate dimension labels.

stringInPatterns(String, List<Pattern>) - Static method in class org.apache.gobblin.util.DatasetFilterUtils

Determines whether a string matches one of the regex patterns.

StringParsingUtils - Class in org.apache.gobblin.util

StringParsingUtils() - Constructor for class org.apache.gobblin.util.StringParsingUtils

StringParsingUtils.FormatException - Exception in org.apache.gobblin.util

STRIP_SUFFIX - Static variable in class org.apache.gobblin.util.ConfigUtils

A suffix that is automatically appended to property keys that are prefixes of other property keys.

submit(Callable<T>) - Method in class org.apache.gobblin.util.executors.MDCPropagatingExecutorService

submit(Runnable) - Method in class org.apache.gobblin.util.executors.MDCPropagatingExecutorService

submit(Runnable, T) - Method in class org.apache.gobblin.util.executors.MDCPropagatingExecutorService

submit(Callable<T>) - Method in class org.apache.gobblin.util.executors.MDCPropagatingScheduledExecutorService

submit(Runnable) - Method in class org.apache.gobblin.util.executors.MDCPropagatingScheduledExecutorService

submit(Runnable, T) - Method in class org.apache.gobblin.util.executors.MDCPropagatingScheduledExecutorService

submitCallable(Callable<Void>, String) - Method in class org.apache.gobblin.util.ParallelRunner

Submit a callable to the thread pool

subtract(ResourceRequirement) - Method in class org.apache.gobblin.util.request_allocation.ResourceRequirement

Vector addition of this and other ResourceRequirement.

survived(String, List<Pattern>, List<Pattern>) - Static method in class org.apache.gobblin.util.DatasetFilterUtils

A topic survives if (1) it doesn't match the blacklist, and (2) either whitelist is empty, or it matches the whitelist.

switchEndStateToErrorState() - Method in class org.apache.gobblin.fsm.FiniteStateMachine.Transition

Change the end state of the transition to the FSM error state.

switchName(Schema, String) - Static method in class org.apache.gobblin.util.AvroUtils

Copies the input Schema but changes the schema name.

switchNamespace(Schema, Map<String, String>) - Static method in class org.apache.gobblin.util.AvroUtils

Copies the input Schema but changes the schema namespace.

SystemPropertiesWrapper - Class in org.apache.gobblin.util

SystemPropertiesWrapper() - Constructor for class org.apache.gobblin.util.SystemPropertiesWrapper

T

t - Variable in class org.apache.gobblin.util.Either

tar(FileSystem, Path, Path) - Static method in class org.apache.gobblin.util.io.StreamUtils

Creates a tar gzip file using a given Path as input and a given Path as a destination.

tar(FileSystem, FileSystem, Path, Path) - Static method in class org.apache.gobblin.util.io.StreamUtils

Similiar to StreamUtils.tar(FileSystem, Path, Path) except the source and destination FileSystem can be different.

TarGpgPathFilter - Class in org.apache.gobblin.util.filters

A PathFilter to accept encrypted tarball files.

TarGpgPathFilter() - Constructor for class org.apache.gobblin.util.filters.TarGpgPathFilter

TaskScheduler<K,T extends ScheduledTask<K>> - Class in org.apache.gobblin.util.concurrent

Base class for a task scheduler that can run ScheduledTasks periodically.

TaskScheduler() - Constructor for class org.apache.gobblin.util.concurrent.TaskScheduler

TaskSchedulerFactory - Class in org.apache.gobblin.util.concurrent

A factory which can be used to get an instance of TaskScheduler.

TaskSchedulerType - Enum in org.apache.gobblin.util.concurrent

The types of supported TaskSchedulers.

TemplateUtils - Class in org.apache.gobblin.util

TemplateUtils() - Constructor for class org.apache.gobblin.util.TemplateUtils

TestingSource - Class in org.apache.gobblin.util.test

A trivial implementation of Source to be used to testing.

TestingSource() - Constructor for class org.apache.gobblin.util.test.TestingSource

TestingSource.Extract - Class in org.apache.gobblin.util.test

TestIOUtils - Class in org.apache.gobblin.util.test

Test utils to read from and write records from a file

TestIOUtils() - Constructor for class org.apache.gobblin.util.test.TestIOUtils

TExtractor() - Constructor for class org.apache.gobblin.util.request_allocation.AllocatedRequestsIteratorBase.TExtractor

ThrottledFileSystem - Class in org.apache.gobblin.util.filesystem

A FileSystemInstrumentation for throttling calls to the underlying FileSystem using the input Limiter.

ThrottledFileSystem(FileSystem, Limiter, String) - Constructor for class org.apache.gobblin.util.filesystem.ThrottledFileSystem

ThrottledFileSystem.Factory<S extends ScopeType<S>> - Class in org.apache.gobblin.util.filesystem

Factory for ThrottledFileSystem.

ThrottledInputStream - Class in org.apache.gobblin.util.io

A throttled InputStream.

ThrottledInputStream(InputStream, Limiter, MeteredInputStream) - Constructor for class org.apache.gobblin.util.io.ThrottledInputStream

Builds a ThrottledInputStream.

TIME_MILLIS - Static variable in class org.apache.gobblin.util.HiveAvroTypeConstants

TimeBasedLimiter - Class in org.apache.gobblin.util.limiter

An implementation of Limiter that limits the time elapsed for some events.

TimeBasedLimiter(long) - Constructor for class org.apache.gobblin.util.limiter.TimeBasedLimiter

TimeBasedLimiter(long, TimeUnit) - Constructor for class org.apache.gobblin.util.limiter.TimeBasedLimiter

TimeBasedLimiter.Factory - Class in org.apache.gobblin.util.limiter

TIMEOUT_MS_KEY - Static variable in class org.apache.gobblin.util.service.StandardServiceConfig

TimeRangeChecker - Class in org.apache.gobblin.util

Utility class to check if a given time is in a pre-defined range.

TimeRangeChecker() - Constructor for class org.apache.gobblin.util.TimeRangeChecker

toConfigurationKey() - Method in class org.apache.gobblin.broker.EmptyKey

toConfigurationKey() - Method in class org.apache.gobblin.util.eventbus.EventBusKey

toConfigurationKey() - Method in class org.apache.gobblin.util.filesystem.FileSystemKey

toConfigurationKey() - Method in class org.apache.gobblin.util.limiter.broker.SharedLimiterKey

TOKEN_RENEWER - Static variable in class org.apache.gobblin.util.hadoop.TokenUtils

TokenUtils - Class in org.apache.gobblin.util.hadoop

A utility class for obtain Hadoop tokens and Hive metastore tokens for Azkaban jobs.

TokenUtils() - Constructor for class org.apache.gobblin.util.hadoop.TokenUtils

toString() - Method in class org.apache.gobblin.util.callbacks.Callback

toString() - Method in class org.apache.gobblin.util.guid.Guid

Serializes the guid into a hex string.

toString() - Method in class org.apache.gobblin.util.Id

toString() - Method in enum org.apache.gobblin.util.limiter.BaseLimiterType

toString() - Method in class org.apache.gobblin.util.limiter.broker.SharedLimiterKey

toString() - Method in class org.apache.gobblin.util.request_allocation.ResourcePool

toStringMap(Object) - Static method in class org.apache.gobblin.util.AvroUtils

Given a map: key -> value, return a map: key.toString() -> value.toString().

TOTAL_MULTI_WORK_UNIT_WEIGHT - Static variable in class org.apache.gobblin.util.binpacking.WorstFitDecreasingBinPacking

totalResourcesUsed() - Method in interface org.apache.gobblin.util.request_allocation.AllocatedRequestsIterator

totalResourcesUsed() - Method in class org.apache.gobblin.util.request_allocation.AllocatedRequestsIteratorBase

toUriPath(Path) - Static method in class org.apache.gobblin.util.HadoopUtils

Get the path as a string without schema or authority.

transitionIfAllowed(T) - Method in class org.apache.gobblin.fsm.FiniteStateMachine

Transition immediately to the given end state if the transition is allowed.

transitionImmediately(T) - Method in class org.apache.gobblin.fsm.FiniteStateMachine

Transition immediately to the given end state.

TrustManagerSecureSocketFactory - Class in org.apache.gobblin.util

TrustManagerSecureSocketFactory() - Constructor for class org.apache.gobblin.util.TrustManagerSecureSocketFactory

TrustManagerSecureSocketFactory(String[], String) - Constructor for class org.apache.gobblin.util.TrustManagerSecureSocketFactory

TrustManagerSocketFactory - Class in org.apache.gobblin.util

A SSL socket wrapper used to create sockets with a

TrustManagerSocketFactory() - Constructor for class org.apache.gobblin.util.TrustManagerSocketFactory

TTLResourceEntry<T> - Class in org.apache.gobblin.broker

A ResourceEntry that automatically expires after a given number of milliseconds.

TTLResourceEntry(T, long, boolean) - Constructor for class org.apache.gobblin.broker.TTLResourceEntry

U

UnallowedTransitionException(Object, Object) - Constructor for exception org.apache.gobblin.fsm.FiniteStateMachine.UnallowedTransitionException

uncaughtException(Thread, Throwable) - Method in class org.apache.gobblin.util.LoggingUncaughtExceptionHandler

unpackFilterInputStream(FilterInputStream) - Static method in class org.apache.gobblin.util.io.FilterStreamUnpacker

Finds the underlying InputStream to a FilterInputStream.

unpackFilterOutputStream(FilterOutputStream) - Static method in class org.apache.gobblin.util.io.FilterStreamUnpacker

Finds the underlying OutputStream to a FilterOutputStream.

unsafeRenameIfNotExists(FileSystem, Path, Path) - Static method in class org.apache.gobblin.util.HadoopUtils

Renames from to to if to doesn't exist in a non-thread-safe way.

updateLog4jConfiguration(Class<?>, String, String) - Static method in class org.apache.gobblin.util.logs.Log4jConfigurationHelper

Update the log4j configuration.

url(String) - Method in class org.apache.gobblin.util.jdbc.DataSourceBuilder

URL_TO_NAME_MAP_RESOURCE_NAME - Static variable in class org.apache.gobblin.util.ClustersNames

useCurrentLogFileName(String) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the current log file name

useDestFileSystem(FileSystem) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the destination FileSystem for writing the destination log file.

useDestFsSupplier(FileSystemSupplier) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the FileSystemSupplier used for generating new Dest FileSystem later when token been updated.

useExcludingRegexPatterns(String) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the regex patterns used to filter logs that should not be copied.

useIncludingRegexPatterns(String) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the regex patterns used to filter logs that should be copied.

useLinesWrittenBeforeFlush(int) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the number of lines written before they are flushed to disk.

useLogFileNamePrefix(String) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the log file name prefix at the destination.

useNumCopyThreads(int) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the number of threads to use for copying container log files to dest FS.

userName(String) - Method in class org.apache.gobblin.util.jdbc.DataSourceBuilder

USERNAME - Static variable in class org.apache.gobblin.util.jdbc.DataSourceProvider

useSourceLogFileMonitorInterval(long) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the interval between two checks for the source log file monitor.

useSrcFileSystem(FileSystem) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the source FileSystem for reading the source log file.

useSrcFsSupplier(FileSystemSupplier) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the FileSystemSupplier used for generating new source FileSystem later when token been updated.

useStrongEncryption(boolean) - Method in class org.apache.gobblin.util.jdbc.DataSourceBuilder

useTimeUnit(TimeUnit) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the TimeUnit used for the source log file monitor interval.

V

valueOf(String) - Static method in enum org.apache.gobblin.fsm.FiniteStateMachine.FailedCallback

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.util.concurrent.TaskSchedulerType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.util.DatePartitionType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.util.filesystem.DataFileVersionStrategy.Characteristic

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.util.Id.Parts

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.util.limiter.BaseLimiterType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.util.limiter.broker.SharedLimiterKey.GlobalLimiterPolicy

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.util.ParallelRunner.FailPolicy

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.util.ProxiedFileSystemUtils.AuthType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.util.ProxiedFileSystemWrapper.AuthType

Deprecated.

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.util.request_allocation.RequestAllocatorConfig.StoreRejectedRequestsConfig

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.util.retry.RetryerFactory.RetryType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum org.apache.gobblin.util.WriterUtils.WriterFilePathType

Returns the enum constant of this type with the specified name.

values() - Static method in enum org.apache.gobblin.fsm.FiniteStateMachine.FailedCallback

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.util.concurrent.TaskSchedulerType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.util.DatePartitionType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.util.filesystem.DataFileVersionStrategy.Characteristic

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.util.Id.Parts

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.util.limiter.BaseLimiterType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.util.limiter.broker.SharedLimiterKey.GlobalLimiterPolicy

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.util.ParallelRunner.FailPolicy

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.util.ProxiedFileSystemUtils.AuthType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.util.ProxiedFileSystemWrapper.AuthType

Deprecated.

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.util.request_allocation.RequestAllocatorConfig.StoreRejectedRequestsConfig

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.util.retry.RetryerFactory.RetryType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum org.apache.gobblin.util.WriterUtils.WriterFilePathType

Returns an array containing the constants of this enum type, in the order they are declared.

verifyAllSuccessful(List<Either<T, ExecutionException>>) - Static method in class org.apache.gobblin.util.executors.IteratorExecutor

Utility method that checks whether all tasks succeeded from the output of IteratorExecutor.executeAndGetResults().

verifySubset(Config, Config) - Static method in class org.apache.gobblin.util.ConfigUtils

Check that every key-value in superConfig is in subConfig

W

waitForTasks(long) - Method in class org.apache.gobblin.util.ParallelRunner

Wait for all submitted tasks to complete.

waitForTasks() - Method in class org.apache.gobblin.util.ParallelRunner

Wait until default timeout(infinite long, if not specified) for all tasks under this parallel runner.

waitOne() - Method in class org.apache.gobblin.util.concurrent.AutoResetEvent

waitOne(int, TimeUnit) - Method in class org.apache.gobblin.util.concurrent.AutoResetEvent

weight(WorkUnit) - Method in class org.apache.gobblin.util.binpacking.FieldWeighter

WHITELIST - Static variable in class org.apache.gobblin.data.management.copy.hive.WhitelistBlacklist

WhitelistBlacklist - Class in org.apache.gobblin.data.management.copy.hive

A whitelist / blacklist implementation for filtering Hive tables.

WhitelistBlacklist(Config) - Constructor for class org.apache.gobblin.data.management.copy.hive.WhitelistBlacklist

WhitelistBlacklist(String, String) - Constructor for class org.apache.gobblin.data.management.copy.hive.WhitelistBlacklist

WhitelistBlacklist(String, String, boolean) - Constructor for class org.apache.gobblin.data.management.copy.hive.WhitelistBlacklist

withAdditionalParentBroker(SharedResourcesBroker<S>) - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl.SubscopedBrokerBuilder

Specify additional ancestor SharedResourcesBrokerImpl.

withBufferSize(int) - Method in class org.apache.gobblin.util.io.StreamCopier

Set the size in bytes of the buffer used to copy.

withCopySpeedMeter(Meter) - Method in class org.apache.gobblin.util.io.StreamCopier

Set a Meter where copy speed will be reported.

withLimitedScopeConfig(Config) - Method in class org.apache.gobblin.util.request_allocation.RequestAllocatorConfig.Builder

withoutLeadingSeparator(Path) - Static method in class org.apache.gobblin.util.PathUtils

Removes the leading slash if present.

withOverridingConfig(Config) - Method in class org.apache.gobblin.broker.SharedResourcesBrokerImpl.SubscopedBrokerBuilder

Specify Config overrides.

withPrioritizer(Comparator<T>) - Method in class org.apache.gobblin.util.request_allocation.RequestAllocatorConfig.Builder

WorkUnitStateSuccess() - Constructor for class org.apache.gobblin.util.PublisherUtils.WorkUnitStateSuccess

WorstFitDecreasingBinPacking - Class in org.apache.gobblin.util.binpacking

Implements a bin packing algorithm similar to worst-fit decreasing bin packing.

WorstFitDecreasingBinPacking() - Constructor for class org.apache.gobblin.util.binpacking.WorstFitDecreasingBinPacking

WritableShimSerialization - Class in org.apache.gobblin.util

A serializer that understands how to write objects that implement WritableShim out to a Hadoop stream.

WritableShimSerialization() - Constructor for class org.apache.gobblin.util.WritableShimSerialization

write(int) - Method in class org.apache.gobblin.util.io.MeteredOutputStream

write(byte[], int, int) - Method in class org.apache.gobblin.util.io.MeteredOutputStream

write(D) - Method in class org.apache.gobblin.util.test.StdoutWriter

WRITER_ENCRYPTED_CONFIG_PATH - Static variable in class org.apache.gobblin.util.WriterUtils

WriterUtils - Class in org.apache.gobblin.util

Utility class for use with the DataWriter class.

WriterUtils() - Constructor for class org.apache.gobblin.util.WriterUtils

WriterUtils.WriterFilePathType - Enum in org.apache.gobblin.util

writeSchemaToFile(Schema, Path, FileSystem, boolean) - Static method in class org.apache.gobblin.util.AvroUtils

writeSchemaToFile(Schema, Path, Path, FileSystem, boolean) - Static method in class org.apache.gobblin.util.AvroUtils

writeSchemaToFile(Schema, Path, FileSystem, boolean, FsPermission) - Static method in class org.apache.gobblin.util.AvroUtils

writeSchemaToFile(Schema, Path, Path, FileSystem, boolean, FsPermission) - Static method in class org.apache.gobblin.util.AvroUtils

Write a schema to a file

writeTo(Path) - Method in class org.apache.gobblin.util.logs.LogCopier.Builder

Set the path of the destination log file directory to write to.

Z

zero() - Method in class org.apache.gobblin.util.request_allocation.ResourceRequirement.Builder

Set all resource requirements to 0.

_

_fileStatus - Variable in class org.apache.gobblin.util.filesystem.FileStatusEntry

_workunits - Variable in class org.apache.gobblin.util.test.TestingSource

A B C D E F G H I J K L M N O P Q R S T U V W Z _