org.apache.gobblin.util.logs

Class LogCopier.Builder

java.lang.Object

org.apache.gobblin.util.logs.LogCopier.Builder

Enclosing class:

LogCopier

public static class LogCopier.Builder
extends Object

A builder class for LogCopier.

Constructor Summary

Constructors

Constructor and Description
Builder()

Method Summary

Modifier and Type	Method and Description
LogCopier.Builder	acceptsLogFileExtensions(Set<String> logFileExtensions) Set the set of acceptable log file extensions.
LogCopier	build() Build a new LogCopier instance.
LogCopier.Builder	readFrom(List<org.apache.hadoop.fs.Path> srcLogDirs) Set the paths of the source log file directories to read from.
LogCopier.Builder	readFrom(org.apache.hadoop.fs.Path srcLogDir) Set the path of the source log file directory to read from.
LogCopier.Builder	useCurrentLogFileName(String currentLogFileName) Set the current log file name
LogCopier.Builder	useDestFileSystem(org.apache.hadoop.fs.FileSystem destFs) Set the destination FileSystem for writing the destination log file.
LogCopier.Builder	useDestFsSupplier(FileSystemSupplier supplier) Set the FileSystemSupplier used for generating new Dest FileSystem later when token been updated.
LogCopier.Builder	useExcludingRegexPatterns(String regexList) Set the regex patterns used to filter logs that should not be copied.
LogCopier.Builder	useIncludingRegexPatterns(String regexList) Set the regex patterns used to filter logs that should be copied.
LogCopier.Builder	useLinesWrittenBeforeFlush(int linesWrittenBeforeFlush) Set the number of lines written before they are flushed to disk.
LogCopier.Builder	useLogFileNamePrefix(String logFileNamePrefix) Set the log file name prefix at the destination.
LogCopier.Builder	useNumCopyThreads(int numCopyThreads) Set the number of threads to use for copying container log files to dest FS.
LogCopier.Builder	useSourceLogFileMonitorInterval(long sourceLogFileMonitorInterval) Set the interval between two checks for the source log file monitor.
LogCopier.Builder	useSrcFileSystem(org.apache.hadoop.fs.FileSystem srcFs) Set the source FileSystem for reading the source log file.
LogCopier.Builder	useSrcFsSupplier(FileSystemSupplier supplier) Set the FileSystemSupplier used for generating new source FileSystem later when token been updated.
LogCopier.Builder	useTimeUnit(TimeUnit timeUnit) Set the TimeUnit used for the source log file monitor interval.
LogCopier.Builder	writeTo(org.apache.hadoop.fs.Path destLogDir) Set the path of the destination log file directory to write to.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

public Builder()

Method Detail

useSourceLogFileMonitorInterval

public LogCopier.Builder useSourceLogFileMonitorInterval(long sourceLogFileMonitorInterval)

Set the interval between two checks for the source log file monitor.

Parameters:

sourceLogFileMonitorInterval - the interval between two checks for the source log file monitor

Returns:

this LogCopier.Builder instance

useTimeUnit

public LogCopier.Builder useTimeUnit(TimeUnit timeUnit)

Set the TimeUnit used for the source log file monitor interval.

Parameters:

timeUnit - the TimeUnit used for the log file monitor interval

Returns:

this LogCopier.Builder instance

useDestFsSupplier

public LogCopier.Builder useDestFsSupplier(FileSystemSupplier supplier)

Set the FileSystemSupplier used for generating new Dest FileSystem later when token been updated.

Parameters:

supplier - the FileSystemSupplier used for generating new Dest FileSystem

Returns:

this LogCopier.Builder instance

useSrcFsSupplier

public LogCopier.Builder useSrcFsSupplier(FileSystemSupplier supplier)

Set the FileSystemSupplier used for generating new source FileSystem later when token been updated.

Parameters:

supplier - the FileSystemSupplier used for generating new source FileSystem

Returns:

this LogCopier.Builder instance

acceptsLogFileExtensions

public LogCopier.Builder acceptsLogFileExtensions(Set<String> logFileExtensions)

Set the set of acceptable log file extensions.

Parameters:

logFileExtensions - the set of acceptable log file extensions

Returns:

this LogCopier.Builder instance

useIncludingRegexPatterns

public LogCopier.Builder useIncludingRegexPatterns(String regexList)

Set the regex patterns used to filter logs that should be copied.

Parameters:

regexList - a comma-separated list of regex patterns

Returns:

this LogCopier.Builder instance

useExcludingRegexPatterns

public LogCopier.Builder useExcludingRegexPatterns(String regexList)

Set the regex patterns used to filter logs that should not be copied.

Parameters:

regexList - a comma-separated list of regex patterns

Returns:

this LogCopier.Builder instance

useSrcFileSystem

public LogCopier.Builder useSrcFileSystem(org.apache.hadoop.fs.FileSystem srcFs)

Set the source FileSystem for reading the source log file.

Parameters:

srcFs - the source FileSystem for reading the source log file

Returns:

this LogCopier.Builder instance

useDestFileSystem

public LogCopier.Builder useDestFileSystem(org.apache.hadoop.fs.FileSystem destFs)

Set the destination FileSystem for writing the destination log file.

Parameters:

destFs - the destination FileSystem for writing the destination log file

Returns:

this LogCopier.Builder instance

readFrom

public LogCopier.Builder readFrom(org.apache.hadoop.fs.Path srcLogDir)

Set the path of the source log file directory to read from.

Parameters:

srcLogDir - the path of the source log file directory to read from

Returns:

this LogCopier.Builder instance

readFrom

public LogCopier.Builder readFrom(List<org.apache.hadoop.fs.Path> srcLogDirs)

Set the paths of the source log file directories to read from.

Parameters:

srcLogDirs - the paths of the source log file directories to read from

Returns:

this LogCopier.Builder instance

writeTo

public LogCopier.Builder writeTo(org.apache.hadoop.fs.Path destLogDir)

Set the path of the destination log file directory to write to.

Parameters:

destLogDir - the path of the destination log file directory to write to

Returns:

this LogCopier.Builder instance

useLogFileNamePrefix

public LogCopier.Builder useLogFileNamePrefix(String logFileNamePrefix)

Set the log file name prefix at the destination.

Parameters:

logFileNamePrefix - the log file name prefix at the destination

Returns:

this LogCopier.Builder instance

useLinesWrittenBeforeFlush

public LogCopier.Builder useLinesWrittenBeforeFlush(int linesWrittenBeforeFlush)

Set the number of lines written before they are flushed to disk.

Parameters:

linesWrittenBeforeFlush - the number of lines written before they are flushed to disk

Returns:

this LogCopier.Builder instance

useCurrentLogFileName

public LogCopier.Builder useCurrentLogFileName(String currentLogFileName)

Set the current log file name

useNumCopyThreads

public LogCopier.Builder useNumCopyThreads(int numCopyThreads)

Set the number of threads to use for copying container log files to dest FS.

Parameters:

numCopyThreads -

build

public LogCopier build()
                throws IOException

Build a new LogCopier instance.

Returns:

a new LogCopier instance

Throws:

IOException