org.apache.tez.runtime.library.common.shuffle.impl
Class Shuffle

java.lang.Object
  org.apache.tez.runtime.library.common.shuffle.impl.Shuffle

@InterfaceAudience.Private
@InterfaceStability.Unstable
public class Shuffle
extends Object

Usage: Create instance, setInitialMemoryAllocated(long), run()

Nested Class Summary

static class

Shuffle.ShuffleError

Constructor Summary

Shuffle(org.apache.tez.runtime.api.TezInputContext inputContext, org.apache.hadoop.conf.Configuration conf, int numInputs, long initialMemoryAvailable)

Method Summary

static long	getInitialMemoryRequirement(org.apache.hadoop.conf.Configuration conf, long maxAvailableTaskMemory)
void	handleEvents(List<org.apache.tez.runtime.api.Event> events)
boolean	isInputReady() Indicates whether the Shuffle and Merge processing is complete.
void	reportException(Throwable t)
void	run()
TezRawKeyValueIterator	waitForInput() Waits for the Shuffle and Merge to complete, and returns an iterator over the input.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Shuffle

public Shuffle(org.apache.tez.runtime.api.TezInputContext inputContext,
               org.apache.hadoop.conf.Configuration conf,
               int numInputs,
               long initialMemoryAvailable)
        throws IOException

Throws:

IOException

Method Detail

handleEvents

public void handleEvents(List<org.apache.tez.runtime.api.Event> events)

isInputReady

public boolean isInputReady()

Indicates whether the Shuffle and Merge processing is complete.

Returns:

false if not complete, true if complete or if an error occurred.

waitForInput

public TezRawKeyValueIterator waitForInput()
                                    throws IOException,
                                           InterruptedException

Waits for the Shuffle and Merge to complete, and returns an iterator over the input.

Returns:

an iterator over the fetched input.

Throws:

IOException

InterruptedException

run

public void run()
         throws IOException

Throws:

IOException

reportException

@InterfaceAudience.Private
public void reportException(Throwable t)

getInitialMemoryRequirement

@InterfaceAudience.Private
public static long getInitialMemoryRequirement(org.apache.hadoop.conf.Configuration conf,
                                                                         long maxAvailableTaskMemory)