org.apache.tez.mapreduce.hadoop
Class InputSplitInfoDisk

java.lang.Object
  org.apache.tez.mapreduce.hadoop.InputSplitInfoDisk

All Implemented Interfaces:

InputSplitInfo

public class InputSplitInfoDisk
extends Object
implements InputSplitInfo

Information obtained by using the InputFormat class to generate the required InputSplit information files that in turn can be used to setup a DAG. The splitsFile and splitsMetaInfoFile need to be provided as LocalResources to the vertex in question. The numTasks represents the parallelism for the vertex and the taskLocationHints define the possible nodes on which the tasks should be run based on the location of the splits that will be processed by each task. The getSplitsProto method is not supported by this implementation.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.tez.mapreduce.hadoop.InputSplitInfo

InputSplitInfo.Type

Constructor Summary

InputSplitInfoDisk(org.apache.hadoop.fs.Path splitsFile, org.apache.hadoop.fs.Path splitsMetaInfoFile, int numTasks, List<org.apache.tez.dag.api.VertexLocationHint.TaskLocationHint> taskLocationHints, org.apache.hadoop.security.Credentials credentials)

Method Summary

org.apache.hadoop.security.Credentials	getCredentials() Get Credentials which may be required to access the splits.
int	getNumTasks() Get the number of splits that were generated.
org.apache.hadoop.fs.Path	getSplitsFile() Get the path to the splits file
org.apache.hadoop.fs.Path	getSplitsMetaInfoFile() Get the path to the splits meta info file
MRRuntimeProtos.MRSplitsProto	getSplitsProto() Get the splits proto
List<org.apache.tez.dag.api.VertexLocationHint.TaskLocationHint>	getTaskLocationHints() Get the TaskLocationHints for each task
InputSplitInfo.Type	getType() Get the InputSplitInfo.Type of the InputSplitInfo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

InputSplitInfoDisk

public InputSplitInfoDisk(org.apache.hadoop.fs.Path splitsFile,
                          org.apache.hadoop.fs.Path splitsMetaInfoFile,
                          int numTasks,
                          List<org.apache.tez.dag.api.VertexLocationHint.TaskLocationHint> taskLocationHints,
                          org.apache.hadoop.security.Credentials credentials)

Method Detail

getTaskLocationHints

public List<org.apache.tez.dag.api.VertexLocationHint.TaskLocationHint> getTaskLocationHints()

Description copied from interface: InputSplitInfo

Get the TaskLocationHints for each task

Specified by:

getTaskLocationHints in interface InputSplitInfo

getSplitsMetaInfoFile

public org.apache.hadoop.fs.Path getSplitsMetaInfoFile()

Description copied from interface: InputSplitInfo

Get the path to the splits meta info file

Specified by:

getSplitsMetaInfoFile in interface InputSplitInfo

getSplitsFile

public org.apache.hadoop.fs.Path getSplitsFile()

Description copied from interface: InputSplitInfo

Get the path to the splits file

Specified by:

getSplitsFile in interface InputSplitInfo

getNumTasks

public int getNumTasks()

Description copied from interface: InputSplitInfo

Get the number of splits that were generated. Same as number of tasks that should be run for the vertex processing these set of splits.

Specified by:

getNumTasks in interface InputSplitInfo

getType

public InputSplitInfo.Type getType()

Description copied from interface: InputSplitInfo

Get the InputSplitInfo.Type of the InputSplitInfo

Specified by:

getType in interface InputSplitInfo

getSplitsProto

public MRRuntimeProtos.MRSplitsProto getSplitsProto()

Description copied from interface: InputSplitInfo

Get the splits proto

Specified by:

getSplitsProto in interface InputSplitInfo

getCredentials

public org.apache.hadoop.security.Credentials getCredentials()

Description copied from interface: InputSplitInfo

Get Credentials which may be required to access the splits.

Specified by:

getCredentials in interface InputSplitInfo

Returns:

Credentials which may be required to access the splits.