org.apache.flink.streaming.connectors.kafka.partitioner

Class FixedPartitioner

java.lang.Object

org.apache.flink.streaming.connectors.kafka.partitioner.KafkaPartitioner

org.apache.flink.streaming.connectors.kafka.partitioner.FixedPartitioner

All Implemented Interfaces:

Serializable, kafka.producer.Partitioner

public class FixedPartitioner
extends KafkaPartitioner
implements Serializable

A partitioner ensuring that each internal Flink partition ends up in one Kafka partition. Note, one Kafka partition can contain multiple Flink partitions. Cases: # More Flink partitions than kafka partitions

                Flink Sinks:            Kafka Partitions
                        1       ---------------->    1
                        2   --------------/
                        3   -------------/
                        4       ------------/
 

Some (or all) kafka partitions contain the output of more than one flink partition # Fewer Flink partitions than Kafka

                Flink Sinks:            Kafka Partitions
                        1       ---------------->    1
                        2       ---------------->    2
                                                                                3
                                                                                4
                                                                                5
 

Not all Kafka partitions contain data To avoid such an unbalanced partitioning, use a round-robin kafka partitioner. (note that this will cause a lot of network connections between all the Flink instances and all the Kafka brokers

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
FixedPartitioner()

Method Summary

Modifier and Type	Method and Description
void	open(int parallelInstanceId, int parallelInstances, int[] partitions) Initializer for the Partitioner.
int	partition(Object element, int numPartitions)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FixedPartitioner

public FixedPartitioner()

Method Detail

open

public void open(int parallelInstanceId,
                 int parallelInstances,
                 int[] partitions)

Description copied from class: KafkaPartitioner

Initializer for the Partitioner.

Overrides:

open in class KafkaPartitioner

Parameters:

parallelInstanceId - 0-indexed id of the parallel instance in Flink

parallelInstances - the total number of parallel instances

partitions - an array describing the partition IDs of the available Kafka partitions.

partition

public int partition(Object element,
                     int numPartitions)

Specified by:

partition in interface kafka.producer.Partitioner