org.apache.flink.streaming.connectors.kafka

Class FlinkKafkaConsumer081<T>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.source.RichParallelSourceFunction<T>

org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer<T>

org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer081<T>

Type Parameters:

T - The type of elements produced by this consumer.

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.api.java.typeutils.ResultTypeQueryable<T>, org.apache.flink.streaming.api.checkpoint.CheckpointCommitter, org.apache.flink.streaming.api.checkpoint.Checkpointed<long[]>, org.apache.flink.streaming.api.checkpoint.CheckpointedAsynchronously<long[]>, org.apache.flink.streaming.api.functions.source.ParallelSourceFunction<T>, org.apache.flink.streaming.api.functions.source.SourceFunction<T>

public class FlinkKafkaConsumer081<T>
extends FlinkKafkaConsumer<T>

Creates a Kafka consumer compatible with reading from Kafka 0.8.1.x brokers. The consumer will internally use the old low-level Kafka API, and manually commit offsets partition offsets to ZooKeeper.

The following additional configuration values are available:

• socket.timeout.ms
• socket.receive.buffer.bytes
• fetch.message.max.bytes
• auto.offset.reset with the values "latest", "earliest" (unlike 0.8.2 behavior)
• fetch.wait.max.ms

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer

FlinkKafkaConsumer.FetcherType, FlinkKafkaConsumer.OffsetStore

Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.functions.source.SourceFunction

org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<T>

Field Summary

Fields inherited from class org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer

DEFAULT_GET_PARTITIONS_RETRIES, GET_PARTITIONS_RETRIES_KEY, MAX_NUM_PENDING_CHECKPOINTS, OFFSET_NOT_SET

Constructor Summary

Constructors

Constructor and Description
FlinkKafkaConsumer081(String topic, org.apache.flink.streaming.util.serialization.DeserializationSchema<T> valueDeserializer, Properties props) Creates a new Kafka 0.8.1.x streaming source consumer.

Method Summary

Methods inherited from class org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer

assignPartitions, cancel, close, commitCheckpoint, getPartitionsForTopic, getProducedType, open, restoreState, run, snapshotState, validateZooKeeperConfig

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FlinkKafkaConsumer081

public FlinkKafkaConsumer081(String topic,
                             org.apache.flink.streaming.util.serialization.DeserializationSchema<T> valueDeserializer,
                             Properties props)

Creates a new Kafka 0.8.1.x streaming source consumer.

Parameters:

topic - The name of the topic that should be consumed.

valueDeserializer - The de-/serializer used to convert between Kafka's byte messages and Flink's objects.

props - The properties used to configure the Kafka consumer client, and the ZooKeeper client.