Package alluxio.cli.fs.command

Class DistributedLoadUtils

  • public final class DistributedLoadUtils
extends java.lang.Object
    Utilities Loads a file or directory in Alluxio space, makes it resident in memory.

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static int distributedLoad​(AbstractDistributedJobCommand command, org.apache.commons.cli.CommandLine cl, java.lang.String pathStr)
      Run the actual distributedLoad command.
      static org.apache.commons.cli.Options getDistLoadCommonOptions()
      Get the general options that distributed load needs to use.
      static java.lang.String getDistLoadCommonUsage()
      Get the general usage of distributed load.
      static org.apache.commons.cli.Options getDistLoadFileSemanticsOptions()
      Get the options of distributed load needs to use in file semantics.
      static java.lang.String getDistLoadFileSemanticsUsage()
      Get usage of distributed load in file semantics.
      static long runDistLoad​(AbstractDistributedJobCommand command, alluxio.AlluxioURI filePath, int replication, int batchSize, java.util.Set<java.lang.String> workerSet, java.util.Set<java.lang.String> excludedWorkerSet, java.util.Set<java.lang.String> localityIds, java.util.Set<java.lang.String> excludedLocalityIds, boolean directCache)
      Run the actual distributedLoad command.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • getDistLoadCommonOptions

        public static org.apache.commons.cli.Options getDistLoadCommonOptions()
        Get the general options that distributed load needs to use. Suitable for file semantics and structured data semantics.
        Returns:
        general options

      • getDistLoadFileSemanticsOptions

        public static org.apache.commons.cli.Options getDistLoadFileSemanticsOptions()
        Get the options of distributed load needs to use in file semantics.
        Returns:
        options in file semantics

      • getDistLoadCommonUsage

        public static java.lang.String getDistLoadCommonUsage()
        Get the general usage of distributed load. Suitable for file semantics and structured data semantics.
        Returns:
        usage message

      • getDistLoadFileSemanticsUsage

        public static java.lang.String getDistLoadFileSemanticsUsage()
        Get usage of distributed load in file semantics.
        Returns:
        usage message

      • distributedLoad

        public static int distributedLoad​(AbstractDistributedJobCommand command,
                                  org.apache.commons.cli.CommandLine cl,
                                  java.lang.String pathStr)
                           throws alluxio.exception.AlluxioException,
                                  java.io.IOException
        Run the actual distributedLoad command.
        Parameters:
        command - Distributed Load command instance
        cl - CommandLine instance of command
        pathStr - path to be loaded
        Returns:
        the result of running the command
        Throws:
        alluxio.exception.AlluxioException
        java.io.IOException

      • runDistLoad

        public static long runDistLoad​(AbstractDistributedJobCommand command,
                               alluxio.AlluxioURI filePath,
                               int replication,
                               int batchSize,
                               java.util.Set<java.lang.String> workerSet,
                               java.util.Set<java.lang.String> excludedWorkerSet,
                               java.util.Set<java.lang.String> localityIds,
                               java.util.Set<java.lang.String> excludedLocalityIds,
                               boolean directCache)
        Run the actual distributedLoad command.
        Parameters:
        command - Distributed Load command instance
        filePath - file path to load
        replication - Number of block replicas of each loaded file
        batchSize - Batch size for loading
        workerSet - A set of worker hosts to load data
        excludedWorkerSet - A set of worker hosts can not to load data
        localityIds - The locality identify set
        excludedLocalityIds - A set of worker locality identify can not to load data
        directCache - use direct cache request or cache through read
        Returns:
        job Control ID