org.apache.tez.mapreduce.input
Class MRInput

java.lang.Object
  org.apache.tez.mapreduce.input.MRInput

All Implemented Interfaces:

org.apache.tez.runtime.api.Input, org.apache.tez.runtime.api.LogicalInput

Direct Known Subclasses:

MRInputLegacy

public class MRInput
extends Object
implements org.apache.tez.runtime.api.LogicalInput

MRInput is an Input which provides key/values pairs for the consumer. It is compatible with all standard Apache Hadoop MapReduce InputFormat implementations. This class is not meant to be extended by external projects.

Field Summary

protected org.apache.hadoop.mapreduce.InputSplit	newInputSplit
protected org.apache.hadoop.mapred.InputSplit	oldInputSplit
protected org.apache.hadoop.mapred.RecordReader	oldRecordReader
protected org.apache.hadoop.mapreduce.split.JobSplit.TaskSplitIndex	splitMetaInfo

Constructor Summary

MRInput()

Method Summary

List<org.apache.tez.runtime.api.Event>	close()
org.apache.hadoop.conf.Configuration	getConfigUpdates() MRInput sets some additional parameters like split location when using the new API.
static org.apache.hadoop.mapreduce.InputSplit	getNewSplitDetailsFromEvent(MRRuntimeProtos.MRSplitProto splitProto, org.apache.hadoop.conf.Configuration conf)
static org.apache.hadoop.mapred.InputSplit	getOldSplitDetailsFromEvent(MRRuntimeProtos.MRSplitProto splitProto, org.apache.hadoop.conf.Configuration conf)
float	getProgress()
org.apache.tez.runtime.library.api.KeyValueReader	getReader()
void	handleEvents(List<org.apache.tez.runtime.api.Event> inputEvents)
List<org.apache.tez.runtime.api.Event>	initialize(org.apache.tez.runtime.api.TezInputContext inputContext)
protected org.apache.hadoop.mapreduce.split.JobSplit.TaskSplitMetaInfo[]	readSplits(org.apache.hadoop.conf.Configuration conf)
void	setNumPhysicalInputs(int numInputs)
void	start()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

newInputSplit

protected org.apache.hadoop.mapreduce.InputSplit newInputSplit

oldRecordReader

protected org.apache.hadoop.mapred.RecordReader oldRecordReader

oldInputSplit

protected org.apache.hadoop.mapred.InputSplit oldInputSplit

splitMetaInfo

protected org.apache.hadoop.mapreduce.split.JobSplit.TaskSplitIndex splitMetaInfo

Constructor Detail

MRInput

public MRInput()

Method Detail

initialize

public List<org.apache.tez.runtime.api.Event> initialize(org.apache.tez.runtime.api.TezInputContext inputContext)
                                                  throws IOException

Specified by:

initialize in interface org.apache.tez.runtime.api.Input

Throws:

IOException

start

public void start()

Specified by:

start in interface org.apache.tez.runtime.api.Input

getReader

public org.apache.tez.runtime.library.api.KeyValueReader getReader()
                                                            throws IOException

Specified by:

getReader in interface org.apache.tez.runtime.api.Input

Throws:

IOException

handleEvents

public void handleEvents(List<org.apache.tez.runtime.api.Event> inputEvents)
                  throws Exception

Specified by:

handleEvents in interface org.apache.tez.runtime.api.Input

Throws:

Exception

setNumPhysicalInputs

public void setNumPhysicalInputs(int numInputs)

Specified by:

setNumPhysicalInputs in interface org.apache.tez.runtime.api.LogicalInput

close

public List<org.apache.tez.runtime.api.Event> close()
                                             throws IOException

Specified by:

close in interface org.apache.tez.runtime.api.Input

Throws:

IOException

getConfigUpdates

public org.apache.hadoop.conf.Configuration getConfigUpdates()

MRInput sets some additional parameters like split location when using the new API. This methods returns the list of additional updates, and should be used by Processors using the old MapReduce API with MRInput.

Returns:

the additional fields set by MRInput

getProgress

public float getProgress()
                  throws IOException,
                         InterruptedException

Throws:

IOException

InterruptedException

getOldSplitDetailsFromEvent

@InterfaceAudience.Private
public static org.apache.hadoop.mapred.InputSplit getOldSplitDetailsFromEvent(MRRuntimeProtos.MRSplitProto splitProto,
                                                                                                        org.apache.hadoop.conf.Configuration conf)
                                                                       throws IOException

Throws:

IOException

getNewSplitDetailsFromEvent

@InterfaceAudience.Private
public static org.apache.hadoop.mapreduce.InputSplit getNewSplitDetailsFromEvent(MRRuntimeProtos.MRSplitProto splitProto,
                                                                                                           org.apache.hadoop.conf.Configuration conf)
                                                                          throws IOException

Throws:

IOException

readSplits

protected org.apache.hadoop.mapreduce.split.JobSplit.TaskSplitMetaInfo[] readSplits(org.apache.hadoop.conf.Configuration conf)
                                                                             throws IOException

Throws:

IOException