alluxio.cli.fs.command

Class DistributedLoadUtils

java.lang.Object

alluxio.cli.fs.command.DistributedLoadUtils

public final class DistributedLoadUtils
extends java.lang.Object

Utilities Loads a file or directory in Alluxio space, makes it resident in memory.

Method Summary

Modifier and Type	Method and Description
static int	distributedLoad(AbstractDistributedJobCommand command, org.apache.commons.cli.CommandLine cl, java.lang.String pathStr) Run the actual distributedLoad command.
static org.apache.commons.cli.Options	getDistLoadCommonOptions() Get the general options that distributed load needs to use.
static java.lang.String	getDistLoadCommonUsage() Get the general usage of distributed load.
static org.apache.commons.cli.Options	getDistLoadFileSemanticsOptions() Get the options of distributed load needs to use in file semantics.
static java.lang.String	getDistLoadFileSemanticsUsage() Get usage of distributed load in file semantics.
static java.lang.Long	runDistLoad(AbstractDistributedJobCommand command, AlluxioURI filePath, int replication, int batchSize, java.util.Set<java.lang.String> workerSet, java.util.Set<java.lang.String> excludedWorkerSet, java.util.Set<java.lang.String> localityIds, java.util.Set<java.lang.String> excludedLocalityIds, boolean directCache) Run the actual distributedLoad command.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getDistLoadCommonOptions

public static org.apache.commons.cli.Options getDistLoadCommonOptions()

Get the general options that distributed load needs to use. Suitable for file semantics and structured data semantics.

Returns:

general options

getDistLoadFileSemanticsOptions

public static org.apache.commons.cli.Options getDistLoadFileSemanticsOptions()

Get the options of distributed load needs to use in file semantics.

Returns:

options in file semantics

getDistLoadCommonUsage

public static java.lang.String getDistLoadCommonUsage()

Get the general usage of distributed load. Suitable for file semantics and structured data semantics.

Returns:

usage message

getDistLoadFileSemanticsUsage

public static java.lang.String getDistLoadFileSemanticsUsage()

Get usage of distributed load in file semantics.

Returns:

usage message

distributedLoad

public static int distributedLoad(AbstractDistributedJobCommand command,
                                  org.apache.commons.cli.CommandLine cl,
                                  java.lang.String pathStr)
                           throws AlluxioException,
                                  java.io.IOException

Run the actual distributedLoad command.

Parameters:

command - Distributed Load command instance

cl - CommandLine instance of command

pathStr - path to be loaded

Returns:

the result of running the command

Throws:

AlluxioException

java.io.IOException

runDistLoad

public static java.lang.Long runDistLoad(AbstractDistributedJobCommand command,
                                         AlluxioURI filePath,
                                         int replication,
                                         int batchSize,
                                         java.util.Set<java.lang.String> workerSet,
                                         java.util.Set<java.lang.String> excludedWorkerSet,
                                         java.util.Set<java.lang.String> localityIds,
                                         java.util.Set<java.lang.String> excludedLocalityIds,
                                         boolean directCache)

Run the actual distributedLoad command.

Parameters:

command - Distributed Load command instance

filePath - file path to load

replication - Number of block replicas of each loaded file

batchSize - Batch size for loading

workerSet - A set of worker hosts to load data

excludedWorkerSet - A set of worker hosts can not to load data

localityIds - The locality identify set

excludedLocalityIds - A set of worker locality identify can not to load data

directCache - use direct cache request or cache through read

Returns:

job Control ID