org.apache.tez.runtime.library.common.sort.impl
Class PipelinedSorter

java.lang.Object
  org.apache.tez.runtime.library.common.sort.impl.ExternalSorter
      org.apache.tez.runtime.library.common.sort.impl.PipelinedSorter

public class PipelinedSorter
extends ExternalSorter

Nested Class Summary

protected class

PipelinedSorter.InputByteBuffer

Nested classes/interfaces inherited from class org.apache.tez.runtime.library.common.sort.impl.ExternalSorter

ExternalSorter.MapBufferTooSmallException

Field Summary

static int

MAP_OUTPUT_INDEX_RECORD_LENGTH The size of each record in the index file for the map-outputs.

Fields inherited from class org.apache.tez.runtime.library.common.sort.impl.ExternalSorter

additionalSpillBytesRead, additionalSpillBytesWritten, availableMemoryMb, codec, combiner, comparator, conf, fileOutputByteCounter, ifileBufferSize, ifileReadAhead, ifileReadAheadLength, keyClass, keySerializer, mapOutputByteCounter, mapOutputFile, mapOutputRecordCounter, nullProgressable, numAdditionalSpills, outputBytesWithOverheadCounter, outputContext, partitioner, partitions, rfs, serializationFactory, sorter, spilledRecordsCounter, valClass, valSerializer

Constructor Summary

PipelinedSorter(org.apache.tez.runtime.api.TezOutputContext outputContext, org.apache.hadoop.conf.Configuration conf, int numOutputs, long initialMemoryAvailable)

Method Summary

void	close()
void	flush()
void	sort()
void	spill()
void	write(Object key, Object value)

Methods inherited from class org.apache.tez.runtime.library.common.sort.impl.ExternalSorter

getInitialMemoryRequirement, getMapOutput, getShuffleHeader, getSortedStream, runCombineProcessor, sameVolRename

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAP_OUTPUT_INDEX_RECORD_LENGTH

public static final int MAP_OUTPUT_INDEX_RECORD_LENGTH

The size of each record in the index file for the map-outputs.

See Also:

Constant Field Values

Constructor Detail

PipelinedSorter

public PipelinedSorter(org.apache.tez.runtime.api.TezOutputContext outputContext,
                       org.apache.hadoop.conf.Configuration conf,
                       int numOutputs,
                       long initialMemoryAvailable)
                throws IOException

Throws:

IOException

Method Detail

sort

public void sort()
          throws IOException

Throws:

IOException

write

public void write(Object key,
                  Object value)
           throws IOException

Specified by:

write in class ExternalSorter

Throws:

IOException

spill

public void spill()
           throws IOException

Throws:

IOException

flush

public void flush()
           throws IOException

Specified by:

flush in class ExternalSorter

Throws:

IOException

close

public void close()

Specified by:

close in class ExternalSorter