com.hazelcast.mapreduce

Class Combiner<ValueIn,ValueOut>

java.lang.Object

com.hazelcast.mapreduce.Combiner<ValueIn,ValueOut>

Type Parameters:

ValueIn - value type of the incoming values

ValueOut - value type of the reduced values

@Beta
public abstract class Combiner<ValueIn,ValueOut>
extends Object

The abstract Combiner class is used to build combiners for the Job.
Those Combiners are distributed inside of the cluster and are running alongside the Mapper implementations in the same node.
Combiners are called in a threadsafe way so internal locking is not required.

Combiners are normally used to build intermediate results on the mapping nodes to lower the traffic overhead between the different nodes before the reducing phase.
Combiners need to be capable of combining data in multiple chunks to create a more streaming like internal behavior.

A simple Combiner implementation in combination with a Reducer could look like this avg-function implementation.

 public class AvgCombiner implements Combiner<Integer, Tuple<Long, Long>>
 {
   private long count;
   private long amount;
   public void combine(Integer value)
   {
     count++;
     amount += value;
   }

   public Tuple<Long, Long> finalizeChunk()
   {
     Tuple<Long, Long> tuple = new Tuple<>( count, amount );
     count = 0;
     amount = 0;
     return tuple;
   }
 }

 public class SumReducer implements Reducer<Tuple<Long, Long>, Integer>
 {
   private long count;
   private long amount;
   public void reduce( Tuple<Long, Long> value )
   {
     count += value.getFirst();
     amount += value.getSecond();
   }

   public Integer finalizeReduce()
   {
     return amount / count;
   }
 }
 

Since:

3.2

Constructor Summary

Constructors

Constructor and Description
Combiner()

Method Summary

Methods

Modifier and Type	Method and Description
void	beginCombine() This method is called before the first value is submitted to this Combiner instance.
abstract void	combine(ValueIn value) This method is called to supply values to be combined into an intermediate result chunk. The combine method might be called multiple times so the combined chunk needs to be held internally in a member state of the Combiner. After this method is called you need to reset() the internal state to prepare for combining of the next chunk.
abstract ValueOut	finalizeChunk() Creates a chunk of ValueOut to be sent to the Reducer for the according key.
void	finalizeCombine() This method is called after the mapping phase on the local node is over.
void	reset() This method is always called after a chunk of data is retrieved.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Combiner

public Combiner()

Method Detail

beginCombine

public void beginCombine()

This method is called before the first value is submitted to this Combiner instance. You can use it to set up any internal needed state before combining the actual values.
The method is called only one time; it is not called again when starting a new chunk.

combine

public abstract void combine(ValueIn value)

This method is called to supply values to be combined into an intermediate result chunk.
The combine method might be called multiple times so the combined chunk needs to be held internally in a member state of the Combiner.
After this method is called you need to reset() the internal state to prepare for combining of the next chunk.

Parameters:

value - value to be reduced (combined into an intermediate result chunk)

finalizeChunk

public abstract ValueOut finalizeChunk()

Creates a chunk of ValueOut to be sent to the Reducer for the according key.

Returns:

chunk of intermediate data

reset

public void reset()

This method is always called after a chunk of data is retrieved. It resets the internal state of the Combiner. It is equivalent to resetting the state inside of finalizeChunk(), as with the last version of the API.

finalizeCombine

public void finalizeCombine()

This method is called after the mapping phase on the local node is over. No further combining runs will take place after this call. You override this method to clean up the internal state and free possibly acquired external resources.